{ "info": { "author": "Jan Seifert (Seznam.cz, a.s.)", "author_email": "jan.seifert@firma.seznam.cz", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Environment :: Web Environment", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "Shelter\n=======\n\nShelter is a *Python's Tornado* wrapper which provides classes and helpers\nfor creation new application skeleton, writing management commands (like a\n*Django*), service processes and request handlers. It was tested with\n*Python 2.7* and *Python 3.4* or higher and *Tornado 3.2* or higher.\n\nInstalation\n-----------\n\n::\n\n cd shelter/\n python setup.py install\n\nAfter instalation **shelter-admin** command is available. For help type:\n\n::\n\n shelter-admin -h\n\nThe most important argument is ``-s/--settings``, which joins Shelter library\nand your application. Format is Python's module path, eg. `myapp.settings`.\nSecond option how to handle `settings` module is ``SHELTER_SETTINGS_MODULE``\nenvironment variable. If both are handled, command line argument has higher\npriority than environment variable.\n\n::\n\n shelter-admin -s myapp.settings\n\nor\n\n::\n\n SHELTER_SETTINGS_MODULE=myapp.settings shelter-admin\n\nUsage\n------\n\n::\n\n shelter-admin startproject myproject\n\nSkeleton of the new application will be created in current working directory.\nProject name has the same rules as Python's module name. Entry point into new\napplication is a script **manage.py**.\n\n::\n\n cd myproject/\n ./manage.py -h\n ./manage.py devserver\n\n`settings.py` is included in the new skeleton. See comments in file how to\ndefine interfaces, management commands, service processes, ...\n\nList of the management commands which provides Shelter library:\n\n+ **runserver** runs production HTTP, multi-process server. Number of the\n processes are detected according to ``INTERFACES`` setting in the\n ``settings`` module. Service processes are run in separated processes.\n Parent process checks child processes and when child process exits (due to\n crash or signal), it is run again. Crashes are counted, maximum amount of\n the crashes are 100, after it whole application will be exited. If child\n stops with exit code 0, crash counter is not incremented, so you can exit\n the worker deliberately and it will be run again.\n+ **devserver** runs development HTTP server, which autoreloads application\n when source files are changes. Server is run only in one process.\n+ **shell** runs interactive Python's shell. First it tries to run *IPython*,\n then standard *Python* shell.\n+ **showconfig** shows effective configuration.\n+ **startproject** will generate new apllication skeleton.\n\nPay attention, service processes are run only in **runserver** command!\n\nConfig class\n------------\n\nLibrary provides base configuration class ``shelter.core.config.Config``\nwhich holds all configuration. Public attributes are **settings** which\nis ``settings`` module of the application and **args_parser** which is\ninstance of the ``argparse.ArgumentParser`` from *Python's standard library*.\n\nYou can override this class in the `settings` module::\n\n CONFIG_CLASS = 'myapp.core.config.AppConfig'\n\nYour own `AppConfig` class can contain additional *properties* with\napplication's settings, e.g. database connection arguments. Way how the value\nis found is only on you - either only in **settings** or **args_parser** or\nin both. You can define additional arguments of the command line.\n\n::\n\n import time\n\n from shelter.core.config import Config, argument\n\n class AppConfig(Config):\n\n arguments = (\n argument(\n '-k', '--secret-key',\n dest='secret_key', action='store',\n type=str, default='',\n help='configuration file'\n ),\n )\n\n Database = collections.namedtuple('Database', ['host', 'db'])\n\n def initialize(self):\n # initialize() is called after instance is created. If you want\n # add some instance attributes, use this method instead of\n # override __init__().\n self._server_started_time = time.time()\n\n def get_config_items(self):\n # Override get_config_items() method if you want to add\n # your options into showconfig management command.\n base_items = super(AppConfig, self).get_config_items()\n app_items = (\n ('secret_key', self.secret_key),\n ('database', self.database),\n )\n return base_items + app_items\n\n @property\n def secret_key(self):\n # If command-line argument -k/--secret-key exists, it will\n # override settings.SECRET_KEY value.\n return self.args_parser.secret_key or self.settings.SECRET_KEY\n\n @property\n def database(self):\n return self.Database(\n db=self.settings.DATABASE_NAME,\n host=self.settings.DATABASE_HOST,\n passwd=getattr(self.settings, DATABASE_PASSWORD, '')\n )\n\nContext class\n-------------\n\nIn all handlers, management commands and service processes is available\ninstance of the ``shelter.core.context.Context`` which holds resources for\nyour appllication. Bundled class ``Context`` contains only one property\n**config** with ``Config`` instance (see previous chapter).\n\nYou can define own class in ``settings`` module::\n\n CONTEXT_CLASS = 'myapp.core.context.Context'\n\nOverrided ``Context`` can contain additional *properties*, e.g. database\nconnection pool.\n\n**It is necesary to initialize shared resources (sockets, open files, ...)\nlazy!** The reason is that subprocesses (Tornado HTTP workers, service\nprocesses) have to get uninitialized ``Context``, because forked resources\ncan cause a lot of nights without dreams... **Also it is necessary to known\nthat Context is shared among coroutines!** So you are responsible for\nlocking shared resources (be careful, it is blocking operation) or use\nanother mechanism, e.g. database connection pool.\n\n``Context`` class contains two methods, ``initialize()`` and\n``initialize_child()``.\n\n``initialize()`` is called from constructor during instance is initialized.\nSo it is the best place where you can initialize attributes which can be\nshared among processes.\n\n``initialize_child()`` is called when service processes or Tornado workers\nare initialized. So it is the best place where you can safely initialize\nshared resources like a database connection. *process_type* argument contains\ntype of the child \u2013 **shelter.core.constants.SERVICE_PROCESS** or\n**shelter.core.constants.TORNADO_WORKER**. *kwargs* contains additional data\naccording to *process_type*:\n\n+ for **SERVICE_PROCESS** contains *process* key which is instance of the\n service process.\n+ for **TORNADO_WORKER** contains *process* key which is instance of the\n HTTP worker.\n\n::\n\n class Context(shelter.core.context.Context):\n\n def initialize(self):\n self._database = None\n\n def initialize_child(self, process_type, **kwargs):\n # Initialize database in the subprocesses when child is created\n self._init_database(max_connections=10)\n\n def _init_database(self, max_connections):\n self._database = ConnectionPool(\n self.config.database.host,\n self.config.database.db,\n max_connections=max_connections,\n connect_on_init=True)\n\n @property\n def database(self):\n # Lazy property if you need database connection in\n # the main process (e.g. management command)\n if self._database is None:\n self._init_database(max_connections=1)\n return self._database\n\nHooks\n-----\n\nYou can define several hooks in the ``settings`` module - when application\nis launched, on **SIGUSR1** and **SIGUSR2** signals and when instance of the\nTornado application is created.\n\n::\n\n APP_SETTINGS_HANDLER = 'myapp.core.app.get_app_settings'\n INIT_HANDLER = 'myapp.core.app.init_handler'\n SIGUSR1_HANDLER = 'myapp.core.app.sigusr1_handler'\n SIGUSR2_HANDLER = 'myapp.core.app.sigusr2_handler'\n\n``INIT_HANDLER`` is allowed to contain multiple values.\n\n::\n\n INIT_HANDLER = [\n 'myapp.core.app.init_handler1', 'myapp.core.app.init_handler2']\n\nHandler is common *Python's* function which takes only one argument\n*context* with ``Context`` instance (see previous chapter).\n\n::\n\n def init_handler(context):\n do_something(context.config)\n\n+ **INIT_HANDLER** is called during the application starts, before workers\n or service processes are run.\n+ **SIGUSR1_HANDLER** is called on **SIGUSR1** signal. When signal receives\n worker/child process, it is processed only in this process. When signal\n receives main/parent process, signal is propagated into all workers.\n+ **SIGUSR2_HANDLER** is called on **SIGUSR2** signal. Signal is processed\n only in process which received signal. It is not propagated anywhere.\n+ **APP_SETTINGS_HANDLER** is called on when instance of the Tornado\n application is created. Function have to return *dict*, which is passed as\n *\\*\\*settings* argument into ``tornado.web.Application`` constructor. Do not\n pass *debug*, *context* and *interface* keys.\n\nService processes\n-----------------\n\nService process are tasks which are repeatedly launched in adjusted interval,\ne.g. warms cache data before they expire. Library provides base class\n``shelter.core.process.BaseProcess``. For new service process\nyou must inherit ``BaseProcess``, adjust ``interval`` attribute and override\n``loop()`` method.\n\n::\n\n from shelter.core.processes import BaseProcess\n\n class WarmCache(BaseProcess)\n\n interval = 30.0\n\n def initialize(self):\n self.db_conn = self.context.db.conn_pool\n self.cache = self.context.cache\n\n def loop(self):\n self.logger.info(\"Warn cached data\")\n with self.db_conn.get() as db:\n self.cache.set('key', db.get_data(), timeout=60)\n\n+ **interval** is a time in seconds. After this time ``loop()`` method is\n repeatedly called.\n\nService process has to be registered in the ``settings`` module.\n\n::\n\n SERVICE_PROCESSES = (\n ('myapp.processes.WarmCache', True, 15.0),\n )\n\nEach service process definition is list/tuple in format\n``('path.to.ClassName', wait_unless_ready, timeout)``. If *wait_unless_ready*\nis ``True``, wait maximum *timeout* seconds unless process is successfully\nstarted, otherwise raise ``shelter.core.exceptions.ProcessError`` exception.\n\nManagement commands\n-------------------\n\nClass ``shelter.core.commands.BaseCommand`` is an ancestor for user\ndefined managemend commands, e.g. export/import database data. For new\nmanagement command you must inherit ``BaseCommand`` and override ``command()``\nmethod and/or ``initialize()`` method.\n\n::\n\n import sys\n\n from gettext import gettext as _\n\n from shelter.core.commands import BaseCommand, argument\n\n class Export(BaseCommand)\n\n name = 'export'\n help = 'export data from database'\n arguments = (\n argument(\n '-o', dest=output_file, type=str, default='-',\n help=_('output filename')),\n )\n\n def initialize(self):\n filename = self.conntext.config.args_parser.output_file\n if filename == '-':\n self.output_file = sys.stdout\n else:\n self.output_file = open(filename, 'wt')\n\n def command(self):\n self.logger.info(\"Exporting data\")\n with self.context.db.get_connection_from_pool() as db:\n data = db.get_data()\n self.output_file.write(data)\n self.output_file.flush()\n\n+ **name** is a name of the management command. This name is used from command\n line, e.g. ``./manage.py export``.\n+ **help** is a short description of the management command. This help is\n printed onto console when you type ``./manage.py command -h``.\n+ **arguments** are arguments of the command line parser. ``argument()``\n function has the same meaning as ``ArgumentParser.add_argument()``\n from *Python's standard library*.\n+ **settings_required** If it is ``False``, `settings` module will not be\n required for command. However, only internal ``shelter.core.config.Config``\n and ``shelter.core.context.Context`` will be available, not your own defined\n in settings. For example, internal **startprocest** command sets this flag\n to ``False``. **It is not public API, do not use this attribute unless you\n really know what you are doing**!\n\nManagement command has to be registered in the ``settings`` module.\n\n::\n\n MANAGEMENT_COMMANDS = (\n 'myapp.commands.Export',\n )\n\nInterfaces\n----------\n\n*Tornado's HTTP server* can be run in multiple instances. Interface are\ndefined in ``settings`` module. Interfaces can be set as either TCP/IP sockets\n(``LISTEN`` directive) or unix sockets (``UNIX_SOCKET`` directive) or both.\n\n::\n\n INTERFACES = {\n 'default': {\n # IP/hostname (not required) and port where the interface\n # listens to requests\n 'LISTEN': ':8000',\n\n # Path to desired unix socket\n 'UNIX_SOCKET': '/run/myapp.sock',\n\n # Amount of the server processes if application is run\n # using runserver command. Positive integer, 0 will\n # detect amount of the CPUs\n 'PROCESSES': 0,\n\n # Path in format 'path.to.module.variable_name' where\n # urls patterns are defined\n 'URLS': 'myapp.urls.default_urls',\n },\n }\n\nURL path to HTTP handler routing\n--------------------------------\n\nIt is the same as in *Python's Tornado* application.\n\n::\n\n from tornado.web import URLSpec\n\n from myapp.handlers import HomepageHandler, AboutHandler\n\n urls_default = (\n URLSpec(r'/', HomepageHandler),\n URLSpec(r'/about/', AboutHandler),\n )\n\nTuple/list **urls_default** is handled into relevant interface in the\n``settings`` module, see previous chapter.\n\nHTTP handler is a subclass of the ``shelter.core.web.BaseRequestHandler``\nwhich enhances ``tornado.web.RequestHandler``. Provides additional instance\nattributes/properties **logger**, **context** and **interface**.\n\n+ **logger** is an instance of the ``logging.Logger`` from *Python's standard\n library*. Logger name is derived from handlers's name, e.g\n ``myapp.handlers.HomepageHandler``.\n+ **context** is an instance of the ``Context``, see *Context* paragraph.\n+ **interface** is a namedtuple with informations about current interface.\n Named attributes are **name**, **host**, **port**, **processes** and\n **urls**.\n\n::\n\n from shelter.core.web import BaseRequestHandler\n\n class DummyHandler(BaseRequestHandler):\n\n def get(self):\n self.write(\n \"Interface '%s' works!\\n\" % self.interface.name)\n self.set_header(\n \"Content-Type\", 'text/plain; charset=UTF-8')\n\nLogging\n-------\n\nStandard *Python's logging* is used. ``Config.configure_logging()`` method\nis responsible for setting the logging. Default ``Config`` class reads\nlogging's configuration from ``settings`` module::\n\n LOGGING = {\n 'version': 1,\n 'disable_existing_loggers': False,\n 'handlers': {\n 'console': {\n 'class': 'logging.StreamHandler',\n 'level': 'INFO',\n 'formatter': 'default',\n },\n },\n 'root': {\n 'handlers': ['console'],\n 'level': 'INFO',\n },\n }\n\nContrib\n-------\n\nshelter.contrib.config.iniconfig.IniConfig\n``````````````````````````````````````````\n\nDescendant of the ``shelter.core.config.Config``, provides **INI** files\nconfiguration. Adds additional public attribute **config_parser** which is\ninstance of the ``RawConfigParser`` from *Python's standard library*.\nInterfaces and application's name can be overrided in configuration file,\n*Python's logging* must be defined.\n\nConfiguration file is specified either by ``SHELTER_CONFIG_FILENAME``\nenvironment variable or ``-f/--config-file`` command line argument. First,\nmain configuration file is read. Then all configuration files from\n``file.conf.d`` subdirectory are read in alphabetical order. E.g. if\n``-f conf/myapp.conf`` is handled, first ``conf/myapp.conf`` file is read\nand then all ``conf/myapp.conf.d/*.conf`` files. Value in later\nconfiguration file overrides previous defined value.\n\n::\n\n [application]\n name = MyApp\n\n [interface_http]\n Listen=:4444\n Processes=8\n Urls=tests.urls1.urls_http\n\n [formatters]\n keys=default\n\n [formatter_default]\n class=logging.Formatter\n format=%(asctime)s %(name)s %(levelname)s: %(message)s\n\n [handlers]\n keys=console\n\n [handler_console]\n class=logging.StreamHandler\n args=()\n level=NOTSET\n\n [loggers]\n keys=root\n\n [logger_root]\n level=INFO\n handlers=console\n\nLicense\n-------\n\n3-clause BSD", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/seznam/shelter", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "shelter", "package_url": "https://pypi.org/project/shelter/", "platform": "any", "project_url": "https://pypi.org/project/shelter/", "project_urls": { "Homepage": "https://github.com/seznam/shelter" }, "release_url": "https://pypi.org/project/shelter/2.1.2/", "requires_dist": null, "requires_python": "", "summary": "Simple Python's Tornado wrapper which provides helpers for creationa new project, writing management commands, service processes, ...", "version": "2.1.2" }, "last_serial": 5480648, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "aa11494013879d94e23d4cfc64a82d60", "sha256": "88aed4fb5fb81a83770e751d2ad3d3ceaa10763e2f172c28766f2ce6f191e106" }, "downloads": -1, "filename": "shelter-1.0.0.tar.gz", "has_sig": false, "md5_digest": "aa11494013879d94e23d4cfc64a82d60", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 28718, "upload_time": "2016-07-06T19:08:58", "url": "https://files.pythonhosted.org/packages/80/fc/241295c0caa250c3f29e18be1c7fa087d1546b422498f213ff79cff4e0ce/shelter-1.0.0.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "0336f274221513e06697ea3b9b3368a1", "sha256": "c8c6abed3ead2d83986d224f42785971a2bdf697459bcc6d867603b09a051952" }, "downloads": -1, "filename": "shelter-1.1.0.tar.gz", "has_sig": false, "md5_digest": "0336f274221513e06697ea3b9b3368a1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 56928, "upload_time": "2016-07-29T12:43:18", "url": "https://files.pythonhosted.org/packages/84/c0/968cb6636124079ef6ae047eb6008d1f2345eddb139574ade06008f68314/shelter-1.1.0.tar.gz" } ], "1.1.1": [ { "comment_text": "", "digests": { "md5": "6ecd8c4ee9168d372b0804623216e9aa", "sha256": "099549277bb6e762e115855de448837f05a39667628e353505b42bd773eadb88" }, "downloads": -1, "filename": "shelter-1.1.1.tar.gz", "has_sig": false, "md5_digest": "6ecd8c4ee9168d372b0804623216e9aa", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 57527, "upload_time": "2016-08-01T14:39:23", "url": "https://files.pythonhosted.org/packages/50/0d/d315d3abc1803e9da22eced0a589f2133d9e285c760c484860def3ea15f7/shelter-1.1.1.tar.gz" } ], "2.0.0": [ { "comment_text": "", "digests": { "md5": "2e65b86580d10352f051ae9400abde9a", "sha256": "089af8739a7002792b2da6dccfc9cd9ce16f7f210c68a2c148efb147d53a891f" }, "downloads": -1, "filename": "shelter-2.0.0.tar.gz", "has_sig": false, "md5_digest": "2e65b86580d10352f051ae9400abde9a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 48639, "upload_time": "2019-03-12T14:08:20", "url": "https://files.pythonhosted.org/packages/e4/04/844c49d244ed9550f59fe5f0f8c0842bc3c95ab8703c37088ad5c9a4d0c8/shelter-2.0.0.tar.gz" } ], "2.1.0": [ { "comment_text": "", "digests": { "md5": "24feaecdb0209f40678b45653bd3ef73", "sha256": "051d5ac31689f5da3ce049c9cf4fd8a1cdc6bd6a111f9ea79d1027ed043b8cf2" }, "downloads": -1, "filename": "shelter-2.1.0.tar.gz", "has_sig": false, "md5_digest": "24feaecdb0209f40678b45653bd3ef73", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 51604, "upload_time": "2019-04-02T06:32:21", "url": "https://files.pythonhosted.org/packages/a2/ff/e202cde998ef88181f0f9bd1e0c875f19a1711126e5bc71e2a709aa173b5/shelter-2.1.0.tar.gz" } ], "2.1.1": [ { "comment_text": "", "digests": { "md5": "ce25e4a04d2722a03cf7cab2f98eeb1b", "sha256": "cf482b4370fa119852d793575cd77d80d1cad6f72a24e4d9da4a8f59a5c6c410" }, "downloads": -1, "filename": "shelter-2.1.1.tar.gz", "has_sig": false, "md5_digest": "ce25e4a04d2722a03cf7cab2f98eeb1b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34812, "upload_time": "2019-05-16T13:47:43", "url": "https://files.pythonhosted.org/packages/1e/2a/acac60abcda6ba26a730b295e2e001216c806e5536eff9f5110faf380ec6/shelter-2.1.1.tar.gz" } ], "2.1.2": [ { "comment_text": "", "digests": { "md5": "0bc199dda087f938bda478cb8c83f3a8", "sha256": "4788905a665c7a5082d05c9102b09f39cbe6b7f087443248efbf5c1b7304f25e" }, "downloads": -1, "filename": "shelter-2.1.2.tar.gz", "has_sig": false, "md5_digest": "0bc199dda087f938bda478cb8c83f3a8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34819, "upload_time": "2019-07-03T10:34:12", "url": "https://files.pythonhosted.org/packages/5c/9d/0a5e0e0ef61ba7eeed4fea6f81a9751aa487fbb44f8c0e5a8ca0b19bfd29/shelter-2.1.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "0bc199dda087f938bda478cb8c83f3a8", "sha256": "4788905a665c7a5082d05c9102b09f39cbe6b7f087443248efbf5c1b7304f25e" }, "downloads": -1, "filename": "shelter-2.1.2.tar.gz", "has_sig": false, "md5_digest": "0bc199dda087f938bda478cb8c83f3a8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34819, "upload_time": "2019-07-03T10:34:12", "url": "https://files.pythonhosted.org/packages/5c/9d/0a5e0e0ef61ba7eeed4fea6f81a9751aa487fbb44f8c0e5a8ca0b19bfd29/shelter-2.1.2.tar.gz" } ] }