{ "info": { "author": "saaj", "author_email": "mail@saaj.me", "bugtrack_url": null, "classifiers": [ "Framework :: CherryPy", "Intended Audience :: Developers", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Database" ], "description": ".. image:: https://img.shields.io/badge/dynamic/json.svg?url=https://api.bitbucket.org/2.0/repositories/saaj/chronologer/pipelines/?page=1%26pagelen=1%26sort=-created_on%26target.ref_name=backend&label=build&query=$.values[0].state.result.name&colorB=blue\n :target: https://bitbucket.org/saaj/chronologer/addon/pipelines/home\n.. image:: https://codecov.io/bitbucket/saaj/chronologer/branch/backend/graph/badge.svg\n :target: https://codecov.io/bitbucket/saaj/chronologer/branch/backend\n.. image:: https://badge.fury.io/py/Chronologer.svg\n :target: https://pypi.org/project/Chronologer/\n.. image:: https://images.microbadger.com/badges/image/saaj/chronologer.svg\n :target: https://microbadger.com/images/saaj/chronologer\n\n===========\nChronologer\n===========\n\n.. figure:: https://bitbucket.org/saaj/chronologer/raw/8b437413ac3ecf50a5f422394332b7d921ce6804/chronologer/static/resource/clui/image/logo/logo-alt240.png\n :alt: Chronologer\n\nChronologer is a counterpart of Python stdlib's logging ``HTTPHandler`` [1]_ and more.\nIt provides RESTful API for accepting Python logging HTTP POST requests and optional\nUI for browsing and searching the logs. The idea is the same as for database backends\nof logging software, like ``rsyslog-mysql`` [2]_.\n\nUI features are described in the `frontend`_ branch and released as\n``ChronologerUI`` [17]_ package.\n\n.. contents::\n\nScope\n=====\nChronologer is meant for small- and medium-sized logging workloads and audit logging.\nPractically it's limited by its backend's write throughput and capacity. In case of\nMySQL backend vertical scaling can suffice many types of applications.\n\nEspecially it's useful for microservice architecture where file logging is no longer\npractical.\n\nInstallation\n============\nChronologer is available as a Python package [3]_ and as a Docker image [4]_ (includes UI).\nThe former can be installed like ``pip install chronologer[server,mysql,ui]``. The latter\ncan be used like in the following ``docker-compose.yml`` for deployment with MySQL database.\n\n.. sourcecode:: yaml\n\n version: '2.4'\n services:\n web:\n image: saaj/chronologer\n environment:\n CHRONOLOGER_STORAGE_DSN: mysql://chronologer:pass@mysql/chronologer\n CHRONOLOGER_SECRET: some_long_random_string\n CHRONOLOGER_USERNAME: logger\n CHRONOLOGER_PASSWORD: another_long_random_string\n depends_on:\n mysql:\n condition: service_healthy\n ports:\n - 8080:8080\n\n mysql:\n image: mysql:5.7\n environment:\n MYSQL_DATABASE: chronologer\n MYSQL_USER: chronologer\n MYSQL_PASSWORD: pass\n MYSQL_ROOT_PASSWORD: pass\n healthcheck:\n test: mysqladmin ping --protocol=tcp --password=pass --silent\n interval: 5s\n retries: 4\n\nIt can be run like the following. The second line applies database migrations.\n\n.. sourcecode:: bash\n\n docker-compose up -d\n docker-compose run --rm web python -m chronologer -e production migrate\n\nTo open the UI navigate to ``http://localhost:8080/``.\n\nChronologer's configuration can be fine-tuned with several environment variables\ndefined in ``envconf`` [5]_. Default ``envconf`` can be overridden completely, see\n``python -m chronologer --help``.\n\nFor examples of scaling the application server with ``docker-compose`` see\n``perftest/stack`` directory [22]_. There are examples for Nginx and Traefik.\n\nQuickstart\n==========\nHaving Chronologer server running as described above, client logging configuration\nmay look like the following. It requires ``chronologer`` package installed on the\nclient as well (i.e. ``pip install chronologer``).\n\n.. sourcecode:: python\n\n import logging.config\n\n\n config = {\n 'version' : 1,\n 'disable_existing_loggers' : False,\n 'handlers' : {\n 'http' : {\n 'class' : 'chronologer.client.QueueProxyHandler',\n 'queue' : {'()': 'queue.Queue', 'maxsize': 10 ** 4},\n 'target' : 'ext://chronologer.client.BatchJsonHandler',\n 'prefix' : 'appname',\n 'capacity' : 128,\n 'host' : 'chronologer_host:8080',\n 'url' : '/api/v1/record',\n 'credentials' : ('logger', 'another_long_random_string'),\n 'flushLevel' : 'ERROR',\n 'flushTimeout' : 30,\n },\n },\n 'root' : {\n 'handlers' : ['http'],\n 'level' : 'INFO'\n }\n }\n logging.config.dictConfig(config)\n\nThe ``http`` handler buffers records for efficiency. It flushes its buffer to\nthe server when one of the following occurs:\n\n* the buffer, of 128 records, has been filled,\n* a logging record with ``level`` ``ERROR`` or above has been logged,\n* while logging a record there's a record in the buffer created earlier\n then 30 seconds ago.\n\n``chronologer.client`` itself doesn't have dependencies but Python standard library.\nFor working only with standard library ``logging.handlers.HTTPHandler`` read below.\n\nPath of the logging handler\n===========================\nThis section starts with ``logging.handlers.HTTPHandler`` and explains why\n``chronologer.client`` builds on it and beyond. The naive imperative logging\nconfiguration looks like:\n\n.. sourcecode:: python\n\n import logging.handlers\n\n chrono = logging.handlers.HTTPHandler(\n 'localhost:8080', '/api/v1/record', 'POST', credentials = ('logger', ''))\n handlers = [logging.StreamHandler(), chrono]\n logging.basicConfig(level = logging.DEBUG, handlers = handlers)\n\nThe same can be expressed declaratively:\n\n.. sourcecode:: python\n\n import logging.config\n\n conf = {\n 'version' : 1,\n 'disable_existing_loggers' : False,\n 'handlers' : {\n 'console' : {\n 'class' : 'logging.StreamHandler',\n },\n 'http' : {\n 'class' : 'logging.handlers.HTTPHandler',\n 'host' : 'localhost:8080',\n 'url' : '/api/v1/record',\n 'method' : 'POST',\n 'credentials' : ('logger', ''),\n 'secure' : False\n },\n },\n 'root' : {\n 'handlers' : ['console', 'http'],\n 'level' : 'DEBUG'\n }\n }\n logging.config.dictConfig(conf)\n\nThis configuration is called naive because the handler is blocking. It may\nwork in trivial cases but generally it's discouraged because the network is\nnot reliable [6]_. Instead Python provides logging queueing in stdlib [7]_:\n\n Along with ``QueueHandler`` class, ``QueueListener`` is used to let\n handlers do their work on a separate thread. This is important for web and\n other applications where threads serving clients need to respond as\n quickly as possible, while any potentially slow, and especially\n complementary operations are done in background.\n\nHere follows imperative configuration with memory queueing.\n\n.. sourcecode:: python\n\n chrono = logging.handlers.HTTPHandler(\n 'localhost:8080', '/api/v1/record', 'POST', credentials = ('logger', ''))\n q = queue.Queue(maxsize = 4096)\n qh = logging.handlers.QueueHandler(q)\n ql = logging.handlers.QueueListener(q, chrono)\n ql.start()\n handlers = [logging.StreamHandler(), qh]\n logging.basicConfig(level = logging.DEBUG, handlers = handlers)\n\n # somewhere on shutdown\n ql.stop()\n\nBecause the queue listener's shutdown procedure is inconvenient this way and it's\nhard to express declaratively, ``QueueProxyHandler`` is suggested.\n\n.. sourcecode:: python\n\n import logging.handlers\n import logging.config\n\n\n class QueueProxyHandler(logging.handlers.QueueHandler):\n '''Queue handler which creates its own ``QueueListener`` to\n proxy log records via provided ``queue`` to ``target`` handler.'''\n\n _listener = None\n '''Queue listener'''\n\n\n def __init__(self, queue, target = logging.handlers.HTTPHandler, **kwargs):\n # user-supplied factory is not converted by default\n if isinstance(queue, logging.config.ConvertingDict):\n queue = queue.configurator.configure_custom(queue)\n\n super().__init__(queue)\n self._listener = logging.handlers.QueueListener(queue, target(**kwargs))\n self._listener.start()\n\n def close(self):\n super().close()\n self._listener.stop()\n\n conf = {\n 'version' : 1,\n 'disable_existing_loggers' : False,\n 'handlers' : {\n 'console' : {\n 'class' : 'logging.StreamHandler',\n },\n 'http' : {\n 'class' : 'somemodule.QueueProxyHandler',\n 'queue' : {'()': 'queue.Queue', 'maxsize': 4096},\n 'host' : 'localhost:8080',\n 'url' : '/api/v1/record',\n 'method' : 'POST',\n 'credentials' : ('logger', ''),\n 'secure' : False\n },\n },\n 'root' : {\n 'handlers' : ['console', 'http'],\n 'level' : 'DEBUG'\n }\n }\n logging.config.dictConfig(conf)\n\n.. warning::\n Always set reasonable ``maxsize`` for the underlying queue to avoid\n unbound memory growth. ``logging.handlers.QueueHandler`` uses\n non-blocking ``put_nowait`` to enqueue records and in case the queue\n is full, it raises and the exception is handled by\n ``logging.Handler.handleError``. Alternatively a file-based queue, for\n instance, ``pqueue`` [8]_, can used to allow more capacity in\n memory-restricted environments.\n\nClient\n======\nFor convenience reasons, the above is available as\n``chronologer.client.QueueProxyHandler``.\n\nIn addition it has logger name prefixing and suffixing capability, and some\nedge case resilience. ``prefix`` is passed to ``QueueProxyHandler`` on creation.\nIt allows many applications logging into the same Chronologer instance to have\nseparate logger namespaces (e.g. including ``aiohttp`` logging whose namespace\nis fixed). ``suffix`` is an extra attribute of ``LogRecord`` which allows to\nfine-tune the logger namespace for easier search of the records.\n\n.. sourcecode:: python\n\n import logging.config\n\n\n conf = {\n 'version' : 1,\n 'disable_existing_loggers' : False,\n 'handlers' : {\n 'console' : {\n 'class' : 'logging.StreamHandler',\n },\n 'http' : {\n 'class' : 'chronologer.client.QueueProxyHandler',\n 'queue' : {'()': 'queue.Queue', 'maxsize': 4096},\n 'prefix' : 'appname',\n 'host' : 'localhost:8080',\n 'url' : '/api/v1/record',\n 'method' : 'POST',\n 'credentials' : ('logger', ''),\n 'secure' : False\n },\n },\n 'root' : {\n 'handlers' : ['console', 'http'],\n 'level' : 'DEBUG'\n }\n }\n logging.config.dictConfig(conf)\n\n logging.getLogger('some').info(\n 'Chronologer!', extra = {'suffix': 'important.transfer'})\n\nThe ``LogRecord`` corresponding to the last line will have ``name`` equal to\n``'appname.some.important.transfer'``. If ``name`` is modified the original is\nsaved as ``origname``.\n\nBut this is unfortunately not it. Looking at ``logging.handlers.HTTPHandler``\ncarefully we can see a few flaws, including but not limited to:\n\n* it doesn't validate response codes, say ``403 Forbidden``, and will silently\n ignore the error, i.e. not calling ``logging.Handler.handleError``, will\n leads to data loss,\n* it doesn't support request retries,\n* it doesn't support buffering to improve throughput,\n* it doesn't support other serialisation formats but\n ``application/x-www-form-urlencoded``.\n\n``chronologer.client.BatchJsonHandler`` tries to address these issues, see\n`Quickstart`_.\n\nJSON input support\n==================\nBesides ``application/x-www-form-urlencoded`` of ``HTTPHandler`` Chronologer\nsupports ``application/json`` of the same structure. It also supports\n``application/x-ndjson`` [19]_ for bulk ingestion.\n\nJSON of arbitrary structure can be ingested in the *raw mode*. In the mode\nChronologer will not classify input into logging ``meta``, ``data`` and\n``error`` and will not insist on presence of Python ``logging``-specific keys.\nFor example, a file containing newline separated JSON entries can be sent to\nChronologer like:\n\n.. sourcecode:: bash\n\n curl -H \"content-type: application/x-ndjson\" --user logger: \\\n --data-binary @/path/to/some/file.ndjson localhost:8080/api/v1/record?raw=1\n\nRecord retention\n================\nWhen ``CHRONOLOGER_RETENTION_DAYS`` is set, daily, around midnight a background\nthread will purge records older than given number of days.\n\nAuthentication\n==============\nChronologer does not provide (neither intends to) a user management. The intent\nis to delegate authentication. The credentials and roles used by the server can\nbe provided by the following environment variables:\n\n* ``CHRONOLOGER_USERNAME``\n* ``CHRONOLOGER_PASSWORD``\n* ``CHRONOLOGER_ROLES`` \u00ad\u2013 space separated role list (see below)\n\nAlternatively a JSON file located by ``CHRONOLOGER_AUTHFILE`` of the following\nstructure can be used to authenticate multiple users:\n\n.. sourcecode:: json\n\n [\n {\n \"username\": \"bob\",\n \"pbkdf2\": \"f57ef1e3e8f90cb367dedd44091f251b5b15c9c36ddd7923731fa7ee41cbaa82\",\n \"hashname\": \"sha256\",\n \"salt\": \"c0139cff\",\n \"iterations\": 32,\n \"roles\": [\"writer\"]\n }, {\n \"username\": \"obo\",\n \"pbkdf2\": \"ff680a9237549f698da5345119dec1ed314eb4fdefe59837d0724d747c3169089ae45...\",\n \"hashname\": \"sha384\",\n \"salt\": \"9230dbdd5a13f009\",\n \"iterations\": 4096,\n \"roles\": [\"basic-reader\", \"query-reader\"]\n }\n ]\n\nThe value of ``pbkdf2`` and keys ``hashname``, ``salt``, ``iterations`` correspond to\nPython ``hashlib.pbkdf2_hmac`` [21]_.\n\n.. warning::\n Note that the auth-scheme is ``Basic`` which means that the password hash is calculated\n per request. Thus ``iterations`` should be a low value (especially for writing\n users). To compensate that it is possible to choose passwords with enough entropy.\n\nAuthorisation\n=============\nChronologer defines the following roles:\n\n* ``basic-reader`` allows ``HEAD`` and ``GET`` to ``/api/v1/record``\n* ``query-reader`` in combination with ``basic-reader`` allows the use\n ``query``, SQL expression, to (further) filter the records\n* ``writer`` allows ``POST`` to ``/api/v1/record``\n\nThe UI (in case ``chronologerui`` is installed) is available to every\nauthenticated user.\n\nAPI\n===\nBy default Chronologer listens port 8080 and is protected by HTTP Basic\nAuthentication, username \"logger\" without password (see environment\nvariables to override these).\n\nChronologer provides *Record* resource.\n\nCreate record\n-------------\n======================== ===============================================\nURL ``/api/v1/record``\n------------------------ -----------------------------------------------\nMethod ``POST``\n------------------------ -----------------------------------------------\nRequest content-type ``application/x-www-form-urlencoded``,\n ``application/json``, ``application/x-ndjson``\n------------------------ -----------------------------------------------\nRequest body Representation of ``logging.LogRecord``\n------------------------ -----------------------------------------------\nResponse content-type ``application/json``\n------------------------ -----------------------------------------------\nResponse body Representation of created ``model.Record``,\n except for ``application/x-ndjson`` input\n where only a list of insert record identifiers\n is returned\n------------------------ -----------------------------------------------\nSuccessful response code ``201 Created``\n======================== ===============================================\n\nOptional *raw* mode, accepting arbitrary JSON documents, is supported by\npassing ``raw=1`` into the query string.\n\n``application/x-ndjson`` request body can produce ``207 Multi-Status``\nresponse when a successful chunk is followed by a failed chunk,\nsay that contained malformed a JSON line. Multi-status body looks like:\n\n.. sourcecode:: json\n\n {\n \"multistatus\": [\n {\"status\": 201, \"body\": [1, 2, \"...\"]},\n {\"status\": 400, \"body\": \"Invalid JSON document on line 2012\"},\n ]\n }\n\nRetrieve record count\n---------------------\n======================== ===============================================\nURL ``/api/v1/record``\n------------------------ -----------------------------------------------\nMethod ``HEAD``\n------------------------ -----------------------------------------------\nQuery string Optional filtering fields (see details below):\n\n * ``after`` \u2013 ISO8601 timestamp\n * ``before`` \u2013 ISO8601 timestamp\n * ``level`` \u2013 integer logging level\n * ``name`` \u2013 logging record prefix(es)\n * ``query`` \u2013 storage-specific expression\n------------------------ -----------------------------------------------\nResponse headers * ``X-Record-Count: 42``\n------------------------ -----------------------------------------------\nSuccessful response code ``200 OK``\n======================== ===============================================\n\nRetrieve record timeline\n------------------------\n======================== ===============================================\nURL ``/api/v1/record``\n------------------------ -----------------------------------------------\nMethod ``HEAD``\n------------------------ -----------------------------------------------\nQuery string Required fields:\n\n * ``group`` \u2013 \"day\" or \"hour\"\n * ``timezone`` \u2013 ``pytz``-compatible one\n\n Optional filtering fields (see details below):\n\n * ``after`` \u2013 ISO8601 timestamp\n * ``before`` \u2013 ISO8601 timestamp\n * ``level`` \u2013 integer logging level\n * ``name`` \u2013 logging record prefix(es)\n * ``query`` \u2013 storage-specific expression\n------------------------ -----------------------------------------------\nResponse headers * ``X-Record-Count: 90,236``\n * ``X-Record-Group: 1360450800,1360537200``\n------------------------ -----------------------------------------------\nSuccessful response code ``200 OK``\n======================== ===============================================\n\nRetrieve record range\n---------------------\n======================== ===============================================\nURL ``/api/v1/record``\n------------------------ -----------------------------------------------\nMethod ``GET``\n------------------------ -----------------------------------------------\nQuery string Required fields:\n\n * ``left`` \u2013 left offset in the result set\n * ``right`` \u2013 right offset in the result set\n\n Optional filtering fields (see details below):\n\n * ``after`` \u2013 ISO8601 timestamp\n * ``before`` \u2013 ISO8601 timestamp\n * ``level`` \u2013 integer logging level\n * ``name`` \u2013 logging record prefix(es)\n * ``query`` \u2013 storage-specific expression\n------------------------ -----------------------------------------------\nResponse content-type ``application/json``\n------------------------ -----------------------------------------------\nResponse body .. sourcecode:: json\n\n [\n {\n \"name\": \"some.module\",\n \"ts\": \"2018-05-10 16:36:53.377493+00:00\",\n \"message\": \"Et quoniam eadem...\",\n \"id\": 177260,\n \"level\": 20\n },\n \"...\"\n ]\n------------------------ -----------------------------------------------\nSuccessful response code ``200 OK``\n======================== ===============================================\n\nRetrieve record\n---------------\n======================== ===============================================\nURL ``/api/v1/record/{id}``\n------------------------ -----------------------------------------------\nMethod ``GET``\n------------------------ -----------------------------------------------\nResponse content-type ``application/json``\n------------------------ -----------------------------------------------\nResponse body .. sourcecode:: json\n\n {\n \"name\": \"some.module\",\n \"logrec\": {\n \"data\": {\n \"foo\": 387\n },\n \"meta\": {\n \"process\": 29406,\n \"module\": \"some.module\",\n \"relativeCreated\": 103.23762893676758,\n \"msecs\": 376.4379024505615,\n \"pathname\": \"logtest.py\",\n \"msg\": \"Et quoniam eadem...\",\n \"stack_info\": null,\n \"processName\": \"MainProcess\",\n \"filename\": \"logtest.py\",\n \"thread\": 140312867051264,\n \"threadName\": \"MainThread\",\n \"lineno\": 20,\n \"funcName\": \"main\",\n \"args\": null\n }\n },\n \"id\": 177260,\n \"level\": 20,\n \"message\": \"Et quoniam eadem...\",\n \"ts\": \"2018-05-10 16:36:53.377493+00:00\"\n }\n\n ``logrec`` has two nested dictionaries.\n ``data`` has what was passed to ``extra`` [16]_\n and ``meta`` has internal fields of\n ``logging.LogRecord``.\n------------------------ -----------------------------------------------\nSuccessful response code ``200 OK``\n======================== ===============================================\n\nError representation\n--------------------\nErrors for HTTP method requests that allow a response body are represented like:\n\n.. sourcecode:: json\n\n {\n \"error\" : {\n \"type\" : \"HTTPError\",\n \"message\" : \"Nothing matches the given URI\"\n }\n }\n\nErrors for HTTP method requests that don't allow a response body are represented in the headers:\n\n* ``X-Error-Type: StorageQueryError``\n* ``X-Error-Message: Make sure the query filter is a valid WHERE expression``\n\nResponse encoding\n-----------------\nChronologer supports Gzip and Brotli response body encoding. The latter takes precedence because\nit provides significant improvement for verbose logging records.\n\n.. note::\n Modern browsers don't advertise, via ``Accept-Encoding``, Brotli support on non-HTTPS\n connections (due to broken intermediary software concerns). In Firefox it can be forced\n by appending ``br`` to ``network.http.accept-encoding`` in ``about:config``.\n\nFiltering\n=========\nFilter fields have the following semantics:\n\n* ``after`` \u2013 ISO8601 timestamp.\n The predicate is true for a record which was created after given timestamp.\n* ``before`` \u2013 ISO8601 timestamp.\n The predicate is true for a record which was created before given timestamp.\n* ``level`` \u2013 integer logging level.\n The predicate is true for a record whose severity level is greater or equal to given level.\n* ``name`` \u2013 logging record prefix. Optionally can be a comma-separated list of prefixes.\n The predicate is true for a record whose logger name starts with any of given prefixes.\n* ``query`` \u2013 storage-specific expression.\n Requires the user to have ``query-reader`` role. See JSON path description below.\n\n.. warning::\n Each user who has access to Chronologer with ``query-reader`` role (default user\n does not have it) effectively has full access to its database, because ``query``\n expressions are put into the SQL queries directly as there's no intent to\n abstract native database search features.\n\nMySQL\n=====\nChronologer relies on a compressed InnoDB table which provides good compromise\nbetween reliability, data modelling, search features, performance and size of\nlogged data. The data of logging records are written into ``logrec`` JSON\nfield (see the initial migration [9]_ and examples above).\n\nIt is a good idea to have dedicated MySQL instance for Chronologer. Then, for\ninstance, it is possible to fine-tune MySQL's ACID guarantees, namely\n``innodb_flush_log_at_trx_commit = 0`` allow MySQL to write 1-second batches\n[10]_. Disabling performance schema [11]_ by setting ``performance_schema = 0``\nis also recommended, because it has significant overhead. Basic InnoDB settings\nshould be reasonably configured:\n\n* ``innodb_buffer_pool_size`` [12]_\n* ``innodb_log_buffer_size`` [13]_\n* ``innodb_log_file_size`` [14]_\n\nJSON path query\n---------------\n``query`` passes a storage-specific expression. Particularly, it's useful\nto write post-filtering conditions for ``logrec`` JSON field using\nJSONPath expressions and ``->`` operator [15]_. It may look like the following,\nthough arbitrary ``WHERE`` clause expressions are possible.\n\n* ``\"logrec->'$.data.foo' = 387 AND logrec->'$.meta.lineno' = 20\"``\n* ``\"logrec->'$.meta.threadName' != 'MainThread'\"``\n\nNote that connection to MySQL works in ``ANSI_QUOTES`` mode [18]_, so ``\"``\ncannot be used to form string literals. ``'`` must be used instead.\n\nCompression tuning\n------------------\nInitial migration [9]_ sets ``KEY_BLOCK_SIZE = 4``. It may be sub-optimal for\nthe shape of your log records. MySQL provides guidelines for choosing\n``KEY_BLOCK_SIZE`` [23]_ and monitoring \"compression failures\"\nat runtime [24]_.\n\nIf you want to change ``KEY_BLOCK_SIZE`` for ``record`` table, you can provide\nyour own database migration. Chronologer uses yoyo-migrations [25]_ for\ndatabase migrations. For example, to switch to ``KEY_BLOCK_SIZE = 8``\nmigration file, named ``20190803T1404_key_size.py``, will look like:\n\n.. sourcecode:: python\n\n from yoyo import step\n\n step('ALTER TABLE record KEY_BLOCK_SIZE = 8')\n\nIt can be mounted into the migration directory of Chonologer's container\nin your ``docker-compose.yml`` like:\n\n.. sourcecode:: yaml\n\n volumes:\n - ./20190803T1404_key_size.py:/opt/chronologer/chronologer/migration/mysql/20190803T1404_key_size.py\n\nThen re-apply migrations with ``migrate`` or run ``serve`` with ``-m`` command\nline flag.\n\nSQLite\n======\nSQLite is supported for very simple, one-off or evaluation cases. Also it doesn't\nsupport compression. ``JSON1`` extension [20]_ is required for JSON Path queries.\n\n* ``\"json_extract(logrec, '$.data.foo') = 387 AND json_extract(logrec, '$.meta.lineno') = 20\"``\n* ``\"json_extract(logrec, '$.meta.threadName') = 'MainThread'\"``\n\nA one-off Chronologer container with SQLite storage can be run on port 8080 like::\n\n docker run --rm -it -p 8080:8080 -v /tmp/db \\\n -e CHRONOLOGER_STORAGE_DSN=sqlite:////tmp/db/chrono.sqlite \\\n -e CHRONOLOGER_SECRET=some_long_random_string \\\n saaj/chronologer \\\n python3.7 -m chronologer -e production serve -u www-data -g www-data -m\n\nTwo things to note:\n\n1. ``-m`` to ``serve`` runs migrations before starting the server,\n2. SQLite needs permissions to the directory where a database file\n resides, to write its temporary files.\n\nR&D roadmap\n===========\nSee the `roadmap`_ issue.\n\nCredits\n=======\nLogo is contributed by `lightypaints`_.\n\n____\n\n.. _frontend: https://bitbucket.org/saaj/chronologer/src/frontend\n.. _roadmap: https://bitbucket.org/saaj/chronologer/issues/1\n.. _lightypaints: https://www.behance.net/lightypaints\n.. [1] https://docs.python.org/3/library/logging.handlers.html#httphandler\n.. [2] https://packages.debian.org/sid/rsyslog-mysql\n.. [3] https://pypi.org/project/Chronologer/\n.. [4] https://hub.docker.com/r/saaj/chronologer/\n.. [5] https://bitbucket.org/saaj/chronologer/src/backend/chronologer/envconf.py\n.. [6] https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing\n.. [7] https://docs.python.org/3/library/logging.handlers.html#queuelistener\n.. [8] https://pypi.org/project/pqueue/\n.. [9] https://bitbucket.org/saaj/chronologer/src/d5d4daa/chronologer/migration/mysql/20171026T1428_initial.py\n.. [10] https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit\n.. [11] https://dev.mysql.com/doc/refman/5.7/en/performance-schema.html\n.. [12] https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_buffer_pool_size\n.. [13] https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_log_buffer_size\n.. [14] https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_log_file_size\n.. [15] https://dev.mysql.com/doc/refman/5.7/en/json-search-functions.html#operator_json-column-path\n.. [16] https://docs.python.org/3/library/logging.html#logging.debug\n.. [17] https://pypi.org/project/ChronologerUI/\n.. [18] https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sqlmode_ansi_quotes\n.. [19] https://github.com/ndjson/ndjson-spec\n.. [20] https://www.sqlite.org/json1.html\n.. [21] https://docs.python.org/3/library/hashlib.html#hashlib.pbkdf2_hmac\n.. [22] https://bitbucket.org/saaj/chronologer/src/backend/perftest/\n.. [23] https://dev.mysql.com/doc/refman/5.7/en/innodb-compression-tuning.html\n.. [24] https://dev.mysql.com/doc/refman/5.7/en/innodb-compression-tuning-monitoring.html\n.. [25] https://pypi.python.org/pypi/yoyo-migrations\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://bitbucket.org/saaj/chronologer/src/backend", "keywords": "python logging http json", "license": "GPL-3.0", "maintainer": "", "maintainer_email": "", "name": "Chronologer", "package_url": "https://pypi.org/project/Chronologer/", "platform": "Any", "project_url": "https://pypi.org/project/Chronologer/", "project_urls": { "Homepage": "https://bitbucket.org/saaj/chronologer/src/backend" }, "release_url": "https://pypi.org/project/Chronologer/0.4.3/", "requires_dist": null, "requires_python": "", "summary": "Python HTTP logging server", "version": "0.4.3" }, "last_serial": 5628442, "releases": { "0.1.4": [ { "comment_text": "", "digests": { "md5": "19f9a3de42dcef476bc1234787302d95", "sha256": "a7ef1c6139d05c8cc139f18216cef076fd14ccd6595dba877c8763ce69d7102f" }, "downloads": -1, "filename": "Chronologer-0.1.4.tar.gz", "has_sig": false, "md5_digest": "19f9a3de42dcef476bc1234787302d95", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 625343, "upload_time": "2018-05-16T20:41:29", "url": "https://files.pythonhosted.org/packages/19/87/707fffd6b7061bb27988978914bec8d913ecd141caceab8dc88c40341ce1/Chronologer-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "b62211fda1f601b78c7c1a73f64405d0", "sha256": "4900ea6872ee7b7f7fe44e328f86d8921394fc9185a0b1de9c2b29689836b22d" }, "downloads": -1, "filename": "Chronologer-0.1.5.tar.gz", "has_sig": false, "md5_digest": "b62211fda1f601b78c7c1a73f64405d0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 626726, "upload_time": "2018-05-21T17:05:21", "url": "https://files.pythonhosted.org/packages/28/6c/ce75fde520afc8b6f580414cce895673e6e9ed0493cd2dacdb6fefa870b9/Chronologer-0.1.5.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "b6b01ea88f90168d656a0ad5b39f0dbd", "sha256": "38dd640b1d9230c88f8f62520e0907114e20abe87e2d89c60e3a13c517eb7e09" }, "downloads": -1, "filename": "Chronologer-0.2.1.tar.gz", "has_sig": false, "md5_digest": "b6b01ea88f90168d656a0ad5b39f0dbd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21306, "upload_time": "2018-09-30T20:50:05", "url": "https://files.pythonhosted.org/packages/51/49/9d70c17f2a2dbd0143228b7753e5429ba5c2788b248918072b117094eb4f/Chronologer-0.2.1.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "f334f8c2cd0430aa8fe73aefa8c6329f", "sha256": "02ef482c7061470f66ae29d86b4e5015ed6eca916672e8842d50099a49fdfab5" }, "downloads": -1, "filename": "Chronologer-0.3.0.tar.gz", "has_sig": false, "md5_digest": "f334f8c2cd0430aa8fe73aefa8c6329f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38635, "upload_time": "2019-01-13T20:56:26", "url": "https://files.pythonhosted.org/packages/d7/bb/0e08a79d4a6cfb3394c92219d0e4b36310fe01c3dc979d3b8ec80565c466/Chronologer-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "67f5798134f62b2c75496f38b343bbf8", "sha256": "2a2ef10cbe2dc3cc3b9519d5f2468168c7bc624a544d66840fca561340bfb85c" }, "downloads": -1, "filename": "Chronologer-0.3.1.tar.gz", "has_sig": false, "md5_digest": "67f5798134f62b2c75496f38b343bbf8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42309, "upload_time": "2019-01-26T14:23:06", "url": "https://files.pythonhosted.org/packages/3a/87/7157fd126a7088bbbe1e559bb65746f79b22e4bca8f1e40d83d1d7e40841/Chronologer-0.3.1.tar.gz" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "23f374574ae0c2323908201dcd01b33e", "sha256": "761ea0d464115a72d6cc4f0176d8707880014e849f59ef110705f7bc87057bea" }, "downloads": -1, "filename": "Chronologer-0.3.2.tar.gz", "has_sig": false, "md5_digest": "23f374574ae0c2323908201dcd01b33e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42386, "upload_time": "2019-01-26T15:38:04", "url": "https://files.pythonhosted.org/packages/01/48/f406094517d23ef74c48e052bf17474591b29964227886c63b5ff95fa23e/Chronologer-0.3.2.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "fee798532571140e4fa855a60da62214", "sha256": "b3f04621575b190e0a3aa8967628af32a6ea9469935f27127d67616d6e50d129" }, "downloads": -1, "filename": "Chronologer-0.4.0.tar.gz", "has_sig": false, "md5_digest": "fee798532571140e4fa855a60da62214", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55589, "upload_time": "2019-04-28T18:49:32", "url": "https://files.pythonhosted.org/packages/7c/95/883d265bb7731fb3876eb2abeba9baa1241277c86628c6f336480f320bc6/Chronologer-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "6990b7c79bf3ed1cfb367a69a7732b85", "sha256": "cb4ff6baf26ad18537c7d302d0c97089bf17e7249961c76f61829b0544a1bf9d" }, "downloads": -1, "filename": "Chronologer-0.4.1.tar.gz", "has_sig": false, "md5_digest": "6990b7c79bf3ed1cfb367a69a7732b85", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55607, "upload_time": "2019-04-28T19:00:15", "url": "https://files.pythonhosted.org/packages/e0/3d/b53b4e9a6d83bdace3296b65e6c2b3d9a30be63a1088ff1b3c62c3e1a165/Chronologer-0.4.1.tar.gz" } ], "0.4.2": [ { "comment_text": "", "digests": { "md5": "2668671aa106860329bd18cbc4755145", "sha256": "d09ca576c2e33766d2ced03b5add3a399c1ae6b69048119a6343e8a0ec65960d" }, "downloads": -1, "filename": "Chronologer-0.4.2.tar.gz", "has_sig": false, "md5_digest": "2668671aa106860329bd18cbc4755145", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55608, "upload_time": "2019-04-28T19:54:05", "url": "https://files.pythonhosted.org/packages/8e/22/8aad0f5a8b2ac29aacaffc0fae9ddce73aa6dca473267bbcd67c179a955b/Chronologer-0.4.2.tar.gz" } ], "0.4.3": [ { "comment_text": "", "digests": { "md5": "1f8919fc1fe813a59d61c29688f8be0a", "sha256": "4c1073ba79025b15d80702108413ae0c2ea24bf7b21cc322cf13ad09405ff76f" }, "downloads": -1, "filename": "Chronologer-0.4.3.tar.gz", "has_sig": false, "md5_digest": "1f8919fc1fe813a59d61c29688f8be0a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 57373, "upload_time": "2019-08-03T15:17:58", "url": "https://files.pythonhosted.org/packages/c0/29/decbccf31b18600b81c8b527957eaae8743d2bf69e600391a06375443e54/Chronologer-0.4.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1f8919fc1fe813a59d61c29688f8be0a", "sha256": "4c1073ba79025b15d80702108413ae0c2ea24bf7b21cc322cf13ad09405ff76f" }, "downloads": -1, "filename": "Chronologer-0.4.3.tar.gz", "has_sig": false, "md5_digest": "1f8919fc1fe813a59d61c29688f8be0a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 57373, "upload_time": "2019-08-03T15:17:58", "url": "https://files.pythonhosted.org/packages/c0/29/decbccf31b18600b81c8b527957eaae8743d2bf69e600391a06375443e54/Chronologer-0.4.3.tar.gz" } ] }