{
    "info": {
        "author": "Javier Llopis",
        "author_email": "javier@llopis.me",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Environment :: Console",
            "Intended Audience :: End Users/Desktop",
            "Intended Audience :: System Administrators",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3.7",
            "Topic :: Internet",
            "Topic :: Office/Business",
            "Topic :: System",
            "Topic :: System :: Monitoring",
            "Topic :: System :: Networking :: Monitoring",
            "Topic :: Utilities"
        ],
        "description": "## UrlMonitor\n\nThis program is intended to be run periodically from cron or a systemd ``.timer`` service on GNU/Linux, a Periodic Task on Windows, or whatever you are using.\n\nWhen run, the program will check the URLs from its list, and, if they have changed since the last time they were checked, will run a series of actions as a result.\n\nThe list of URLs and actions to be run if changed are specified in a YAML file (by default called ``urllist.yml``). This file consist on a LIST of OBJECTS, each containing the fields name with a descriptive string, url with the url to be checked, and actions, which is a LIST of OBJECTS, where the key is the action name and the value are the arguments, which can be a STRING or a LIST of strings.\n\nAn example of urllist.yml file:\n\n```yaml\n---\n- name: Check for new versions of my favourite software\n  url: https://www.coolproject.org/download/files\n  actions:\n    - email_notify:\n        - joe@domain.com\n        - jill@somewhere.else.com\n\n- name: Get latest version of the software\n  url: https://www.coolproject.org/latest\n  actions:\n    - re_match:\n        - '[Ss]ource [Tt]arball.*<a href=\\\"([^\"]+)'\n    - run_script: download_and_build https://www.coolproject.org/{re_match[1]}\n```\n\nAs you can see on the last line, the arguments to the actions are expanded with the variables from previous actions, the url, the contents, the HTTP status code obtained while fetching the URL, etc.\n\n### Installation\n\nJust run the provided ``setup.py`` program:\n\n```\n$ python3 setup.py install\n```\n\n### Running\n\nYou can call the program with ``--help`` for the program options::\n\n\n    $ urlmonitor --help\n    usage: urlmonitor [-h] [--config FILE] [--persist-file FILE]\n                      [ymlfile [ymlfile ...]]\n\n    positional arguments:\n      ymlfile               Url check specification.\n\n    optional arguments:\n      -h, --help            show this help message and exit\n      --config FILE, -c FILE\n                            Configuration file.\n      --persist-file FILE, -p FILE\n                            Persistence file to store URL status. Default\n                            ./persistence.sqlite\n\n\n* The ``yml`` file(s) contain the URLs to check, as discussed above.\n* The ``--persist-file`` is a database that will be created in order to track whether the URLs have changed and when.\n* The ``--config`` file is a YAML file that specifies where the options are located. It is discussed on the next section.\n\n### Configuration\n\nUrlMonitor is configured using a YAML file containing an OBJECT with the different sections where you can specify:\n\n* Global variables (``set_vars``): available to all the actions.\n* Actions (``actions``): Each additional action individually, specifying its name, the Python module where it is located, and perhaps a set of configuration parameters.\n* Action directories (``action_dir``): Specify a directory containing Python files. Each Python file contains an action named like the file.\n* Action configurations (``action_config``): Contains an OBJECT whose entries are the names of the actions and the values are the configuration parameters for that action. Any action can be configured in this way, whether it has been specified using ``action`` or an ``action_dir``. Some predefined actions also need to be configured this way, for example the ``email_notify`` predefined action needs the parameter ``smtp_server`` to be set.\n* Logging configuration (``logging_config``): Contains the settings needed to call `logging.config.dictConfig()` as per the [Python standard library documentation](https://docs.python.org/3/library/logging.config.html#logging-config-dictschema). The logger name that will be used is `URLMONITOR`. If this section is not present, sensible defaults will be used.\n\nAn example of the configuration follows::\n\n~~~yaml\n---\n\n# define global variables for all the actions\nset_vars:\n    var1: 5.4\n    var2:\n        - list\n        - of\n        - \"strings\"\n    var3:\n        keys: of\n        a: dict\n\n# load all the actions in a directory\n#   all the .py files in the directory will be loaded\n#   if the module is action_name.py, the action name will be 'action_name'\n#   and will be initialised with the objects in its entry in an\n#   an 'actions_config' section\n#\naction_dir: /path/to/dir1\n\naction_dir:\n    # specify a list of paths\n    - /path/to/dir2\n    - /another/path/dir3\n\n# actions and configuration objects:\n# configuration objects are mappings that the actions need to initialise\n# themselves, like the smtp host in a 'mail_send' action or a database name\n# and connection parameters to look up/store stuff etc.\n#\n# The actions will be called (or not) at runtime with the url that's changed,\n# the contents of that url, and the variables with their values at that point.\n\nactions_config:\n        # configuration objects for actions defined on action_dir\n        # as opposed to actions defined in actions:\n        action_in_dir1:\n            update_url: https://www.someserver.com\n            something_or_another: 34\n\n        some_action_in_dir3:\n            parameter: value\n            another_parameter: another_value\n\n\nactions:\n    # always a list containing mappings of actions and configuration values\n\n                # should always have a name and module\n    - name: add_to_database\n      module: /path/to/module.py\n                # ad_hoc configuration parameters, defined and expected by\n                # each particular action\n      dbhost: host1.mynetwork\n      dbname: documents\n      dbuser: albert\n      dbpassword: hello0011\n\n    - name: another_action:\n      module: /the/path/to/another_action.py\n      foo: bar\n\nlogging_config:\n     # the following is passed directly to logging.config.dictConfig()\n     loggers:\n        URLMONITOR:\n            level: DEBUG\n            handlers:\n                - console\n     handlers:\n         console:\n             class: logging.StreamHandler\n             stream: ext://sys.stdout\n             level: DEBUG\n~~~\n\n### Predefined Actions\n\nThe following actions are available out of the box:\n\n#### set_vars\nSet global variables. The variables defined using this action are global and can be used by all the URL objects and their actions.\n\nExample:\n```yaml\nset_vars:\n   value_1: 34\n   list_var:\n      - item 1\n      - item 2\n      - another item\n```\n\n#### list_vars\nThis action lists all the variables and their values to the log. The log level must be debug.\n\n#### no_action\nThis is a dummy action. It does nothing.\n\n#### re_match\nSearch the content of the URL using one or more regular expressions passed as arguments. The regular expressions are applied using ``re.MULTILINE`` (see [Python's ``re`` module documentation](https://docs.python.org/3/library/re.html#re.MULTILINE) ) i.e. '^' matches the beginning of every line and '$' matches every end of line.\n\nAs a result, it creates two variables:\n\n* ``re_match``: A list of all the group matches in the first regular expression.\n* ``re_match_all``: A list containing the list of all group matches of all the regular expressions in the argument list, in the same order as they appear.\n\nExample:\n\n```yaml\n  - name: Get amount and code\n    url: http://www.interesting.com/amounts_codes/\n    actions:\n       - re_match:\n             - 'The amount is:\\s*([0-9]+)'\n             - 'Code\\s*(\\w+)'\n       - set_vars:\n            amount: {re_match[1]}\n            code: {re_match_all[1][1]}\n```\n\n#### email_notify\nSend an email with a notification of the URL change to the recipient email addresses specified in the argument list.\n\nExample:\n```yaml\n- name: Check for new versions of my favourite software\n  url: https://www.coolproject.org/download/files\n  actions:\n    - email_notify:\n        - joe@domain.com\n        - jill@somewhere.else.com\n```\n\nBefore this action can be used it must be configured in the yaml configuration file with the following parameters:\n\nName            | Value                   | Default value\n----------------|-------------------------|-------------\nsmtp_server     | server name             |\nfrom_address    | email address           | urlmonitor@hostname\nsmtp_port       | port number             | 25 for SMTP, 587 for SSL\nsmtp_encryption | SSL or STARTTLS         | empty, no encryption\nsmtp_user       | username                | empty, no smtp authentication\nsmtp_password   | password base64 encoded |\n\nConfiguration example:\n\n```yaml\n- actions_config:\n     email_notify:\n         smtp_server: mailhost.server.com\n         from_address: urlbot@mydomain.com\n         smtp_encryption: STARTTLS\n         smtp_port: 465\n         smtp_user: phil\n         smtp_password: aGVsbG8wMDExMjI=\n```\n\n#### run_script\nThis action runs a program whose command line is its argument list.\n\nExample:\n\n```yaml\n- name: Download the new file\n  url: https://www.filesite.net/targetfiles.html\n  actions:\n      - re_match: '<a href=\\\"(.*)\\\">The file'\n      - run_script: wget {re_match[1]}\n```\nThis action sets the following variables:\n\nVariable    | Value\n------------|-----------------------------------\nreturn_code | The exit code of the program\nerror       | The error encountered, if any\nstdout      | Whatever the program has written to its standard output\nstderr      | Whatever the program has written to its error output\n\n\n### Writing Custom Actions\nYou can write a custom action easily. You just need to write a Python module that exports a callable called ``action_object`` with the following signature:\n\n```python\ndef action_object(name, arglst, url, content, variables, log):\n    pass\n```\n\nWhere the arguments are:\n\nArgument  | Description\n----------|--------\nname      | The action name\narglst    | The list of arguments.\nurl       | The url that changed\ncontent   | The content (possibly HTML) of the url\nvariables | A mapping containing the variables and their values\nlog       | A logging object\n\nIf the arguments in ``arglst`` will be expanded by the calling routine and should contain the final value.\n\nIf your action requires configuration parameters, you can create a class that has a ``__call__`` method and instantiate into ``action_object``. At configuration time, the program will call the ``initialise`` method with a dictionary containing the configuration parameters as argument.\n\nIn order to simplify the process, the module ``actionbase`` provides the class ``Action`` from which your action can inherit. This already provides an ``initialise`` method which will install the configuration parameters, which you can access as instance attributes of ``action_object``. In addition, you should provide a list of expected parameters and a default value for those that can have one in the class variables ``check_cfg_vars`` and ``default_vars``. An exception is raised if a parameter that has been specified in ``check_cfg_vars`` is not found and there is no default value.\n\nAs an example, you can check the code for the ``email_notify`` action.\n\n```python\nfrom urlmonitor.actionbase import Action\n\nclass _EmailAction(Action):\n\n    check_cfg_vars = [ \"smtp_server\", \"from_address\", \"smtp_encryption\",\n            \"smtp_user\", \"smtp_password\", \"smtp_port\"]\n    default_vars = {\n        \"from_address\": \"urlmonitor@{}\".format(_NODE),\n        \"smtp_encryption\": None,\n        \"smtp_user\": None,\n        \"smtp_password\": \"\",\n        \"smtp_port\": 0,\n        }\n\n    ...\n\n    def __call__(self, name, arglst, url, content, variables, log):\n        ...\n        \naction_object = _EmailAction()\n```\n\n\n### License\n\nThis software is released under the **MIT License**",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/destrangis/urlmonitor",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "urlmonitor",
        "package_url": "https://pypi.org/project/urlmonitor/",
        "platform": "",
        "project_url": "https://pypi.org/project/urlmonitor/",
        "project_urls": {
            "Homepage": "https://github.com/destrangis/urlmonitor"
        },
        "release_url": "https://pypi.org/project/urlmonitor/0.0.8/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Check url and run actions if changed",
        "version": "0.0.8"
    },
    "last_serial": 5958860,
    "releases": {
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5d23b035db08fe02b2fdb3900718c29a",
                    "sha256": "0a57bc7d650ad69a5bea8f225a519cb207af9d948c9f7d243ac0b528a6115b29"
                },
                "downloads": -1,
                "filename": "urlmonitor-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "5d23b035db08fe02b2fdb3900718c29a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17753,
                "upload_time": "2019-05-26T19:44:21",
                "url": "https://files.pythonhosted.org/packages/6c/d1/5480daed713e68b54b23ebcca720995bcb97bc85b1ca02f9b0892eac0eb1/urlmonitor-0.0.4.tar.gz"
            }
        ],
        "0.0.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7d0f9d764808a4636015de94dd777b9a",
                    "sha256": "810277b5b9c2a3746ca7ef2704679f2c9b2f0f1fe742ccf7adeeffe503d58ddd"
                },
                "downloads": -1,
                "filename": "urlmonitor-0.0.8.tar.gz",
                "has_sig": false,
                "md5_digest": "7d0f9d764808a4636015de94dd777b9a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17799,
                "upload_time": "2019-10-11T07:50:43",
                "url": "https://files.pythonhosted.org/packages/2b/a5/6af81eb046edc29f785641cdc476cd98d70f5b16d5e8232dea81c6faf285/urlmonitor-0.0.8.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "7d0f9d764808a4636015de94dd777b9a",
                "sha256": "810277b5b9c2a3746ca7ef2704679f2c9b2f0f1fe742ccf7adeeffe503d58ddd"
            },
            "downloads": -1,
            "filename": "urlmonitor-0.0.8.tar.gz",
            "has_sig": false,
            "md5_digest": "7d0f9d764808a4636015de94dd777b9a",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 17799,
            "upload_time": "2019-10-11T07:50:43",
            "url": "https://files.pythonhosted.org/packages/2b/a5/6af81eb046edc29f785641cdc476cd98d70f5b16d5e8232dea81c6faf285/urlmonitor-0.0.8.tar.gz"
        }
    ]
}