{ "info": { "author": "Gavin Huttley", "author_email": "gavin.huttley@anu.edu.au", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Science/Research", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Topic :: Scientific/Engineering :: Bio-Informatics", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "##################\nAbout ``scitrack``\n##################\n\nOne of the critical challenges in scientific analysis is to track all the elements involved. This includes the arguments provided to a specific application, input data files referenced by those arguments and output data generated by the application. In addition to this, tracking a minimal set of system specific information.\n\n``scitrack`` is a library aimed at application developers writing scientific software to support this tracking of scientific computation. The library provides elementary functionality to support logging. The primary capabilities concern generating checksums on input and output files and facilitating logging of the computational environment.\n\n**********\nInstalling\n**********\n\nFor the released version::\n\n $ pip install scitrack\n\nFor the very latest version::\n\n $ pip install git+https://github.com/HuttleyLab/scitrack\n\nOr clone it::\n\n $ git clone git@github.com:HuttleyLab/scitrack.git\n\nAnd then install::\n\n $ pip install ~/path/to/scitrack\n\n*****************\n``CachingLogger``\n*****************\n\nThere is a single object provided by ``scitrack``, ``CachingLogger``. This object is basically a wrapper around the ``logging`` module, but on invocation, captures basic information regarding the system and the command line call that was made to invoke the application.\n\nIn addition, the class provides convenience methods for logging both the path and the md5 hexdigest checksum of input/output files. A method is also provided for producing checksums of text data. The latter is useful for the case when data are from a stream or a database, for instance.\n\nAll logging calls are cached until a path for a logfile is provided. The logger can also, optionally, create directories.\n\nWhen run in parallel using ``mpirun``, the process ID is appended to the hostname to help identify processors.\n\n**********************************\nSimple instantiation of the logger\n**********************************\n\nCreating the logger. Setting ``create_dir=True`` means on creation of the logfile, the directory path will be created also.\n\n.. code:: python\n\n from scitrack import CachingLogger\n LOGGER = CachingLogger(create_dir=True)\n LOGGER.log_file_path = \"somedir/some_path.log\"\n\n\nThe last assignment triggers creation of ``somedir/some_path.log``.\n\n******************************************\nCapturing a programs arguments and options\n******************************************\n\n``scitrack`` will write the contents of ``sys.argv`` to the log file, prefixed by ``command_string``. However, this only captures arguments specified on the command line. Tracking the value of optional arguments not specified, which may have default values, is critical to tracking the full command set. Doing this is your responsibility as a developer.\n\nHere's one approach when using the ``click`` `command line interface library `_. Below we create a simple ``click`` app and capture the required and optional argument values.\n\n.. code:: python\n\n from scitrack import CachingLogger\n import click\n\n LOGGER = CachingLogger()\n\n @click.group()\n def main():\n \"\"\"the main command\"\"\"\n pass\n\n @main.command()\n @click.option('-i', '--infile', type=click.Path(exists=True))\n @click.option('-t', '--test', is_flag=True, help='Run test.')\n def my_app(infile, test):\n # capture the local variables, at this point just provided arguments\n LOGGER.log_args()\n LOGGER.log_versions('numpy')\n LOGGER.input_file(infile)\n LOGGER.log_file_path = \"some_path.log\"\n\n if __name__ == \"__main__\":\n my_app()\n\n\nThe ``CachingLogger.write()`` method takes a message and a label. All other logging methods wrap ``log_message()``, providing a specific label. For instance, the method ``input_file()`` writes out two lines in the log.\n\n- input_file_path, the absolute path to the intput file\n- input_file_path md5sum, the hex digest of the file\n\n``output_file()`` behaves analogously. An additional method ``text_data()`` is useful for other data input/output sources (e.g. records from a database). For this to have value for arbitrary data types requires a systematic approach to ensuring the text conversion is robust across platforms.\n\nThe ``log_args()`` method captures all local variables within a scope.\n\nThe ``log_versions()`` method captures versions for the current file and that of a list of named packages, e.g. ``LOGGER.log_versions(['numpy', 'sklearn'])``.\n\n\nSome sample output\n==================\n\n::\n\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO system_details : system=Darwin Kernel Version 18.2.0: Fri Oct 5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO python : 3.7.1\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO user : gavin\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO command_string : /Users/gavin/miniconda3/envs/py37/bin/py.test -s\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO input_file_path : /Users/gavin/repos/SciTrack/tests/sample.fasta\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO input_file_path md5sum : 96eb2c2632bae19eb65ea9224aaafdad\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO version : test_logging==0.1.5\n 2018-11-28 11:33:30 yourmachine.com:71779 INFO version : numpy==1.15.1\n\n**********************\nOther useful functions\n**********************\n\nTwo other useful functions are ``get_file_hexdigest`` and ``get_text_hexdigest``. The latter can take either unicode or ascii strings.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/HuttleyLab/scitrack", "keywords": "science,logging,parallel", "license": "['BSD']", "maintainer": "", "maintainer_email": "", "name": "scitrack", "package_url": "https://pypi.org/project/scitrack/", "platform": "any", "project_url": "https://pypi.org/project/scitrack/", "project_urls": { "Homepage": "https://github.com/HuttleyLab/scitrack" }, "release_url": "https://pypi.org/project/scitrack/0.1.8.1/", "requires_dist": null, "requires_python": "", "summary": "scitrack", "version": "0.1.8.1" }, "last_serial": 5797760, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "2653d94ec24180a2db69029d3cbe8a14", "sha256": "8c456c216833cd98a73698fbf86ff7d908f76ca652217ee9e2593c897c5d6442" }, "downloads": -1, "filename": "scitrack-0.1-py2-none-any.whl", "has_sig": false, "md5_digest": "2653d94ec24180a2db69029d3cbe8a14", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 4181, "upload_time": "2016-05-22T02:49:12", "url": "https://files.pythonhosted.org/packages/72/b0/ae4a3a34290b2c61494820568c0c0705e2febc7186def5c4c765a51f58a9/scitrack-0.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e5cd9784400b66e837d89c2857119b1a", "sha256": "5f70a2d5da94d26016d2afe06c29dd3e166e958c75c576ef732a458a0972c09d" }, "downloads": -1, "filename": "scitrack-0.1.tar.gz", "has_sig": false, "md5_digest": "e5cd9784400b66e837d89c2857119b1a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5087, "upload_time": "2016-05-22T02:49:18", "url": "https://files.pythonhosted.org/packages/cb/27/ba9a06ae4e9b1b4dd315de0bc54c4d82c4dbd3b58ad1cfd8b93e855e2bff/scitrack-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "6fa68d754f7a7c9387c75b26b53ad994", "sha256": "0fb81213b9222dd8217d3e37c0ebda8ed40d4166b20d4b73091feb1ab17b8b89" }, "downloads": -1, "filename": "scitrack-0.1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "6fa68d754f7a7c9387c75b26b53ad994", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4205, "upload_time": "2016-08-18T23:29:11", "url": "https://files.pythonhosted.org/packages/e9/f4/0d4c93489625e812822085832b4500a0da17c59dce33170a7f9c59e650a6/scitrack-0.1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8917d78794cf0633bc79331bb9682609", "sha256": "337a9fbb3633a02073356edf417bceb3b8673c7a043495d6eae654e6a904ef34" }, "downloads": -1, "filename": "scitrack-0.1.1.tar.gz", "has_sig": false, "md5_digest": "8917d78794cf0633bc79331bb9682609", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5118, "upload_time": "2016-08-18T23:29:14", "url": "https://files.pythonhosted.org/packages/31/64/4c898643ceb56b6c84c6566bd06d687910ee870cdc0e773538fb8e6698d4/scitrack-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "f4397118978b569b2cbbbc937966a761", "sha256": "f511702ebbe455aa549003ab36b0447602da522cdbf7a41905ba245a19f172d8" }, "downloads": -1, "filename": "scitrack-0.1.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f4397118978b569b2cbbbc937966a761", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4265, "upload_time": "2016-11-25T03:50:45", "url": "https://files.pythonhosted.org/packages/64/ee/df6d9db756030f90c0099459841fcd288163a4c76e6c62fde542829abf7a/scitrack-0.1.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e1b905c2bf9e4422a27b70d853de3336", "sha256": "be83dd943d867d26c87bf223beef49d57a4df92b91aac43f5c3aa088f51688b4" }, "downloads": -1, "filename": "scitrack-0.1.2.tar.gz", "has_sig": false, "md5_digest": "e1b905c2bf9e4422a27b70d853de3336", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5191, "upload_time": "2016-11-25T03:50:48", "url": "https://files.pythonhosted.org/packages/ea/0b/377810e7409b84b03cbd3f508c7e6774307561710b8a21c4a6cc3bc51e88/scitrack-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "15dd6c99508f87487ea3455caf9ad7b5", "sha256": "1ed36d0e97ae8d196cbceb70c7e8d5b7f5a76db44a75f08a80f4079361e8c7dc" }, "downloads": -1, "filename": "scitrack-0.1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "15dd6c99508f87487ea3455caf9ad7b5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4498, "upload_time": "2017-06-01T01:19:48", "url": "https://files.pythonhosted.org/packages/bf/e0/7731e6ff0865a9c5bfee507cdca2d2459a05479a03b6d44c4ac48c82a05b/scitrack-0.1.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ba75786d4181c05cdc1efd3e2b42e2eb", "sha256": "bb0a48454e713dba4be98915caec578e9d3385f480009005639711f38b49da88" }, "downloads": -1, "filename": "scitrack-0.1.3.tar.gz", "has_sig": false, "md5_digest": "ba75786d4181c05cdc1efd3e2b42e2eb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5405, "upload_time": "2017-06-01T01:19:51", "url": "https://files.pythonhosted.org/packages/c3/88/ec851541b2d9d1dfcc4f7b95ff5ed953d252486c4f5917aaf4a8e19a3ccb/scitrack-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "c16684e7f628e88312b33d93a03ad9e7", "sha256": "2128a3414003d917705d878fd367be5e8b84e5f42d423963485be54850546cdd" }, "downloads": -1, "filename": "scitrack-0.1.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c16684e7f628e88312b33d93a03ad9e7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4546, "upload_time": "2018-01-29T22:15:20", "url": "https://files.pythonhosted.org/packages/7d/44/aec879e34e9def33cccf7339ed82095ac5b63693bf990d381c26c5a29b86/scitrack-0.1.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ddea94452de5c26c63f1c11de5b8aa0b", "sha256": "6bea0f8f809764e3091d902dbd38318911c77819f4a639f40d18b146d37ffc38" }, "downloads": -1, "filename": "scitrack-0.1.4.tar.gz", "has_sig": false, "md5_digest": "ddea94452de5c26c63f1c11de5b8aa0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5353, "upload_time": "2018-01-29T22:15:22", "url": "https://files.pythonhosted.org/packages/af/a5/fa45ec46473a73e1e10b80bb9bcae6e0540bb097d32fae95e1f60b314380/scitrack-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "bf362ef9e3936b481c3f69fd8208aa87", "sha256": "9c13d151dfb95330edd2950e9fb71755768d43d7d7ec36234808a55c98b231f6" }, "downloads": -1, "filename": "scitrack-0.1.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "bf362ef9e3936b481c3f69fd8208aa87", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4552, "upload_time": "2018-10-20T04:21:59", "url": "https://files.pythonhosted.org/packages/bc/91/076dec48a4c19371d711f52485d3361773f891e4e96dfa36c654cce42b72/scitrack-0.1.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2136c112e2c43649a65d127c5ee95631", "sha256": "777e453068068581849bc0c55acb98c09b12900ab6310e35f8a03fd503a79d0f" }, "downloads": -1, "filename": "scitrack-0.1.5.tar.gz", "has_sig": false, "md5_digest": "2136c112e2c43649a65d127c5ee95631", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5365, "upload_time": "2018-10-20T04:22:01", "url": "https://files.pythonhosted.org/packages/4d/bb/dd40755f47507382216d3c3da8c3576ffa350cb9d885412cb59eaf36f9d4/scitrack-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "d80a1cf5bde65dbde9404140e0301276", "sha256": "34da15cbf072d47e404fcc6fa8200b63dbeb7e657ebd7150f289a18e6a0aa664" }, "downloads": -1, "filename": "scitrack-0.1.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d80a1cf5bde65dbde9404140e0301276", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4085, "upload_time": "2018-11-28T00:43:56", "url": "https://files.pythonhosted.org/packages/73/42/0fb6bc13a40877eb02df8ea539d976af2a16d5b3eba962e0e34763a6c7c3/scitrack-0.1.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "df31aa3664c40d181179501906126001", "sha256": "15110c184e93ba30565b047a1cc2cc26bb7dd7ad2b4b11fca2142fb074167618" }, "downloads": -1, "filename": "scitrack-0.1.6.tar.gz", "has_sig": false, "md5_digest": "df31aa3664c40d181179501906126001", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6027, "upload_time": "2018-11-28T00:43:58", "url": "https://files.pythonhosted.org/packages/e9/80/94afa77630d6ca001eca17f637e34138c82aa365c43e19c6a127402a3547/scitrack-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "b44a28af75a71565a76eba5d939816b8", "sha256": "c14af3ce530c8d1946f47dc0f83e6c66fc642069baba96b4b9b2367fdf1b0b86" }, "downloads": -1, "filename": "scitrack-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b44a28af75a71565a76eba5d939816b8", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4052, "upload_time": "2019-08-09T05:13:40", "url": "https://files.pythonhosted.org/packages/db/8a/390107b2a322719e2f82feb4ac300c59b4e2d0063a303abe920a70d96b74/scitrack-0.1.7-py2.py3-none-any.whl" } ], "0.1.8": [ { "comment_text": "", "digests": { "md5": "4eeaf28f5ac7b7c63fb9ee7e08995d15", "sha256": "b4e0a376babd9eac2a02878a5ae1f67ae7601b59e9237dbd6a25ba6162f2d0fc" }, "downloads": -1, "filename": "scitrack-0.1.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "4eeaf28f5ac7b7c63fb9ee7e08995d15", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6197, "upload_time": "2019-09-07T23:33:02", "url": "https://files.pythonhosted.org/packages/bf/75/db875a97712ad239ea596821d19fdfa6bc3168e8405d097d6b5c6b9ad33b/scitrack-0.1.8-py2.py3-none-any.whl" } ], "0.1.8.1": [ { "comment_text": "", "digests": { "md5": "76efcb49e8bfea4bd8df38d1246eae25", "sha256": "fcebeb8c4c869344e60173d5ce310e067b29a3552255bf2f5ec4bc4dc3c80958" }, "downloads": -1, "filename": "scitrack-0.1.8.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "76efcb49e8bfea4bd8df38d1246eae25", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6172, "upload_time": "2019-09-07T23:36:07", "url": "https://files.pythonhosted.org/packages/ba/9e/6e9ca0402b0c5fedfb19b552a252810ad2b85396ec177663203eba46cec3/scitrack-0.1.8.1-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "76efcb49e8bfea4bd8df38d1246eae25", "sha256": "fcebeb8c4c869344e60173d5ce310e067b29a3552255bf2f5ec4bc4dc3c80958" }, "downloads": -1, "filename": "scitrack-0.1.8.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "76efcb49e8bfea4bd8df38d1246eae25", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6172, "upload_time": "2019-09-07T23:36:07", "url": "https://files.pythonhosted.org/packages/ba/9e/6e9ca0402b0c5fedfb19b552a252810ad2b85396ec177663203eba46cec3/scitrack-0.1.8.1-py2.py3-none-any.whl" } ] }