{
    "info": {
        "author": "Alex Lubbock",
        "author_email": "code@alexlubbock.com",
        "bugtrack_url": null,
        "classifiers": [
            "Intended Audience :: Developers",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python"
        ],
        "description": "# Microbench\n\nMicrobench is a small Python package for benchmarking Python functions, and \noptionally capturing extra runtime/environment information. It is most useful in\nclustered/distributed environments, where the same function runs under different\nenvironments, and is designed to be extensible with new\nfunctionality. In addition to benchmarking, this can help reproducibility by\ne.g. logging the versions of key Python packages, or even all packages loaded\ninto the global environment.\n\n## Requirements\n\nMicrobench by default has no dependencies outside of the Python standard\nlibrary, although [pandas](https://pandas.pydata.org/) is recommended to\nexamine results. However, some mixins (extensions) have specific requirements:\n\n* The [line_profiler](https://github.com/rkern/line_profiler)\n  package needs to be installed for line-by-line code benchmarking.\n* `MBInstalledPackages` requires `setuptools`, which is not a part of the\n  standard library, but is usually available. \n* The CPU cores and total RAM extensions require\n  [psutil](https://pypi.org/project/psutil/).\n* The NVIDIA GPU plugin requires the\n  [nvidia-smi](https://developer.nvidia.com/nvidia-system-management-interface)\n  utility, which usually ships with the NVIDIA graphics card drivers. It needs\n  to be on your `PATH`.\n\n## Installation\n\nTo install using `pip`:\n\n```\npip install microbench\n```\n\n## Usage\n\nMicrobench is designed for benchmarking Python functions. These examples will\nassume you have already defined a Python function `myfunction` that you wish to\nbenchmark:\n\n```python\ndef myfunction(arg1, arg2, ...):\n    ...\n```\n\n### Minimal example\n\nFirst, create a benchmark suite, which specifies the configuration and\ninformation to capture.\n\nHere's a minimal, complete example:\n\n```python\nfrom microbench import MicroBench\n    \nbasic_bench = MicroBench()\n```\n\nTo attach the benchmark to your function, simply use `basic_bench` as a\ndecorator, like this:\n\n```python\n@basic_bench\ndef myfunction(arg1, arg2, ...):\n    ...\n```\n\nThat's it! Benchmark information will be appended to the file specified in\n`outfile`. This example captures the fields `start_time`, `finish_time` and\n`function_name`. See the **Examine results** section for further information.\n\n### Extended examples\n\nHere's a more complete example using mixins (the `MB` prefixed class \nnames) to extend functionality. Note that keyword arguments can be supplied\nto the constructor (in this case `some_info=123`) to specify additional\ninformation to capture. This example also specifies the `outfile` option,\nwhich writes conda\n\n```python\nfrom microbench import *\nimport numpy, pandas\n\nclass MyBench(MicroBench, MBFunctionCall, MBPythonVersion, MBHostInfo):\n    outfile = '/home/user/my-benchmarks'\n    capture_versions = (numpy, pandas)  # Or use MBGlobalPackages/MBInstalledPackages\n    env_vars = ('SLURM_ARRAY_TASK_ID', )\n    \nbenchmark = MyBench(some_info=123)\n```\n\nThe `env_vars` option from the example above specifies a list of environment\nvariables to capture as `env_<variable name>`. In this example,\nthe [slurm](https://slurm.schedmd.com) array task ID will be stored as\n`env_SLURM_ARRAY_TASK_ID`. Where the environment variable is not set, the\nvalue will be `null`.\n\nTo capture package versions, you can either specify them individually (as above), or you can capture the versions of\nevery package in the global environment. In the following example, we would capture the versions of `microbench`,\n`numpy`, and `pandas` automatically.\n\n```python\nfrom microbench import *\nimport numpy, pandas\n\nclass Bench2(MicroBench, MBGlobalPackages):\n    outfile = '/home/user/bench2'\n\nbench2 = Bench2()\n```\n\nIf you want to go even further, and capture the version of every package available for import, there's a\nmixin for that:\n\n```python\nfrom microbench import *\n\nclass Bench3(MicroBench, MBInstalledPackages):\n    pass\n    \nbench3 = Bench3()\n``` \n\n Mixin                 | Fields captured\n-----------------------|----------------\n*(default)*            | `start_time`<br>`finish_time`<br>`function_name`\nMBGlobalPackages       | `package_versions`, with entry for every package in the global environment\nMBInstalledPackages    | `package_versions`, with entry for every package available for import\nMBFunctionCall         | `args` (positional arguments)<br>`kwargs` (keyword arguments)\nMBPythonVersion        | `python_version` (e.g. 3.6.0) and `python_executable` (e.g. `/usr/bin/python`, which should indicate any active virtual environment)\nMBHostInfo             | `hostname`<br>`operating_system`\nMBHostCpuCores         | `cpu_cores_logical` (number of cores, requires `psutil`)\nMBHostRamTotal         | `ram_total` (total RAM in bytes, requires `psutil`)\nMBNvidiaSmi            | Various NVIDIA GPU fields, detailed in a later section\nMBLineProfiler         | `line_profiler` containing line-by-line profile (see section below)\n\n## Examine results\n\nEach result is a [JSON](https://en.wikipedia.org/wiki/JSON) object. When using\nthe `outfile` option, a JSON object for each `@benchmark` call is stored on a\nseparate line in the file. The output from the minimal example above for a\nsingle run will look similar to the following:\n\n```json\n{\"start_time\": \"2018-08-06T10:28:24.806493\", \"finish_time\": \"2018-08-06T10:28:24.867456\", \"function_name\": \"my_function\"}\n```\n\nThe simplest way to examine results in detail is to load them into a\n[pandas](https://pandas.pydata.org/) dataframe:\n\n```python\nimport pandas\nresults = pandas.read_json('/home/user/my-benchmarks', lines=True)\n```\n\nPandas has powerful data manipulation capabilities. For example, to calculate\nthe average runtime by Python version:\n\n```python\n# Calculate runtime for each run\nresults['runtime'] = results['finish_time'] - results['start_time']\n\n# Average runtime by Python version\nresults.groupby('python_version')['runtime'].mean()\n```\n\nMany more advanced operations are available. The\n[pandas tutorial](https://pandas.pydata.org/pandas-docs/stable/tutorials.html)\nis recommended.\n\n## Line profiler support\n\nMicrobench also has support for [line_profiler](https://github.com/rkern/line_profiler), which shows the execution time\nof each line of Python code. Note that this will slow down your code, so only use it if needed, but it's useful for\ndiscovering bottlenecks within a function. Requires the `line_profiler` package to be installed\n(e.g. `pip install line_profiler`).\n\n```python\nfrom microbench import MicroBench, MBLineProfiler\nimport pandas\n\n# Create our benchmark suite using the MBLineProfiler mixin\nclass LineProfilerBench(MicroBench, MBLineProfiler):\n    pass\n\nlpbench = LineProfilerBench()\n\n# Decorate our function with the benchmark suite\n@lpbench\ndef my_function():\n    \"\"\" Inefficient function for line profiler \"\"\"\n    acc = 0\n    for i in range(1000000):\n        acc += i\n\n    return acc\n\n# Call the function as normal\nmy_function()\n\n# Read the results into a Pandas DataFrame\nresults = pandas.read_json(lpbench.outfile.getvalue(), lines=True)\n\n# Get the line profiler report as an object\nlp = MBLineProfiler.decode_line_profile(results['line_profiler'][0])\n\n# Print the line profiler report\nMBLineProfiler.print_line_profile(results['line_profiler'][0])\n```\n\nThe last line of the previous example will print the line profiler report, showing the execution time of each line of\ncode. Example:\n\n```\nTimer unit: 1e-06 s\n\nTotal time: 0.476723 s\nFile: /home/user/my_test.py\nFunction: my_function at line 12\n\nLine #      Hits         Time  Per Hit   % Time  Line Contents\n==============================================================\n    12                                               @lpbench\n    13                                               def my_function():\n    14                                                   \"\"\" Inefficient function for line profiler \"\"\"\n    15         1          2.0      2.0      0.0          acc = 0\n    16   1000001     217874.0      0.2     45.7          for i in range(1000000):\n    17   1000000     258846.0      0.3     54.3              acc += i\n    18\n    19         1          1.0      1.0      0.0          return acc\n```\n\n## NVIDIA GPU support\n\nAttributes about NVIDIA GPUs can be captured using the `MBNvidiaSmi` plugin.\nThis requires the `nvidia-smi` utility to be available in the current `PATH`.\n\nBy default, the `gpu_name` (model number) and `memory.total` attributes are\ncaptured. Extra attributes can be specified using the class or object-level\nvariable `nvidia_attributes`. To see which attributes are available, run\n`nvidia-smi --help-query-gpu`.\n\nBy default, all installed GPUs will be polled. To limit to a specific GPU,\nspecify the `nvidia_gpus` attribute as a tuple of GPU IDs, which can be\nzero-based GPU indexes (can change between reboots, not recommended),\nGPU UUIDs, or PCI bus IDs. You can find out GPU UUIDs by running\n`nvidia-smi -L`.\n\nHere's an example specifying the optional `nvidia_attributes` and\n`nvidia_gpus` fields:\n\n```python\nfrom microbench import MicroBench, MBNvidiaSmi\n\nclass GpuBench(MicroBench, MBNvidiaSmi):\n    outfile = '/home/user/gpu-benchmarks'\n    nvidia_attributes = ('gpu_name', 'memory.total', 'pcie.link.width.max')\n    nvidia_gpus = (0, )  # Usually better to specify GPU UUIDs here instead\n\ngpu_bench = GpuBench()\n```\n\n## Extending microbench\n\nMicrobench includes a few mixins for basic functionality as described in the\nextended example, above.\n\nYou can also add functions to your benchmark suite to capture\nextra information at runtime. These functions must be prefixed with `capture_`\nfor them to run automatically before the function starts. They take\na single argument, `bm_data`, a dictionary to be extended with extra data.\nCare should be taken to avoid overwriting existing key names.\n\nHere's an example to capture the machine type (`i386`, `x86_64` etc.):\n\n```python\nfrom microbench import MicroBench\nimport platform\n\nclass Bench(MicroBench):\n    outfile = '/home/user/my-benchmarks'\n\n    def capture_machine_platform(self, bm_data):\n        bm_data['platform'] = platform.machine()\n        \nbenchmark = Bench()\n```\n\n## Redis support\n\nBy default, microbench appends output to a file, but output can be directed\nelsewhere, e.g. [redis](https://redis.io) - an in-memory, networked data source.\nThis option is useful when a shared filesystem is not available.\n\nRedis support requires [redis-py](https://github.com/andymccurdy/redis-py).\n\nTo use this feature, inherit from `MicroBenchRedis` instead of `MicroBench`,\nand specify the redis connection and key name as in the following example:\n\n```python\nfrom microbench import MicroBenchRedis\n\nclass RedisBench(MicroBenchRedis):\n    # redis_connection contains arguments for redis.StrictClient()\n    redis_connection = {'host': 'localhost', 'port': 6379}\n    redis_key = 'microbench:mykey'\n\nbenchmark = RedisBench()\n```\n\nTo retrieve results, the `redis` package can be used directly:\n\n```python\nimport redis\nimport pandas\n\n# Establish the connection to redis\nrconn = redis.StrictRedis(host=..., port=...)\n\n# Read the redis data from 'myrediskey' into a list of byte arrays\nredis_data = redis.lrange('myrediskey', 0, -1)\n\n# Convert the list into a single string\njson_data = '\\n'.join(r.decode('utf8') for r in redis_data)\n\n# Read the string into a pandas dataframe\nresults = pandas.read_json(json_data, lines=True)\n```\n\n## Feedback\n\nPlease note this is a recently created, experimental package. Please let me know\nyour feedback or feature requests in Github issues.",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/alubbock/microbench",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "microbench",
        "package_url": "https://pypi.org/project/microbench/",
        "platform": "",
        "project_url": "https://pypi.org/project/microbench/",
        "project_urls": {
            "Homepage": "https://github.com/alubbock/microbench"
        },
        "release_url": "https://pypi.org/project/microbench/0.5/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Micro-benchmarking framework. Extensible, with distributed/cluster support.",
        "version": "0.5"
    },
    "last_serial": 4941568,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "35183c81d2ecdb53aa999fcb72891168",
                    "sha256": "462f81aad7e9f94e7f986da07c8042a03ccb39df8aed596e1abf4bd68a437533"
                },
                "downloads": -1,
                "filename": "microbench-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "35183c81d2ecdb53aa999fcb72891168",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19282,
                "upload_time": "2018-08-02T22:54:14",
                "url": "https://files.pythonhosted.org/packages/f3/2e/432b030d171e69b01caf1df57216ebad5fd9771cb0ab86c6f1afdb4a05c8/microbench-0.1.tar.gz"
            }
        ],
        "0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a946bbc9d0d9b20e39295b0956fb164f",
                    "sha256": "a968f1951f3257b07e7e6820e571c0c2189eeb648b5832734dc49bb5899ed510"
                },
                "downloads": -1,
                "filename": "microbench-0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "a946bbc9d0d9b20e39295b0956fb164f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20114,
                "upload_time": "2018-08-03T18:58:58",
                "url": "https://files.pythonhosted.org/packages/c6/aa/0ea8fabae331a4c5eeceaffcadae36cc4bdc8b2dc128824a1c67da474e58/microbench-0.2.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "152a559f242634e788b45c9c140f63ff",
                    "sha256": "65ebb1c8023008d707b966c8884ee6e9c4845b9561f8e0aa09bc55e236a5ef21"
                },
                "downloads": -1,
                "filename": "microbench-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "152a559f242634e788b45c9c140f63ff",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 23728,
                "upload_time": "2018-08-03T22:41:28",
                "url": "https://files.pythonhosted.org/packages/88/1a/1137ea6c03ee0a929e9acec0e0b4760590b4914100e5752614fd04e1b20c/microbench-0.2.1.tar.gz"
            }
        ],
        "0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "73ef0cd5f78e510c8a7c4792a021a6d2",
                    "sha256": "eafcbd3476c73ffa936d39237bb5b06ec55541cb4ed2d91e64318b2b1d627f1d"
                },
                "downloads": -1,
                "filename": "microbench-0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "73ef0cd5f78e510c8a7c4792a021a6d2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 24320,
                "upload_time": "2018-08-06T15:38:45",
                "url": "https://files.pythonhosted.org/packages/e1/e6/cda6d93ba43ab31fec68baa2b209629223601d75c5d216837e5511f9d7fe/microbench-0.3.tar.gz"
            }
        ],
        "0.4.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "50f922e4687c7aa12221b1fcfc1fbbfb",
                    "sha256": "87a1a6fc8d33e8a91feed051341b54761aca19fcc0503632ad59cbadb545a7b4"
                },
                "downloads": -1,
                "filename": "microbench-0.4.1.tar.gz",
                "has_sig": false,
                "md5_digest": "50f922e4687c7aa12221b1fcfc1fbbfb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 23768,
                "upload_time": "2019-03-06T00:08:41",
                "url": "https://files.pythonhosted.org/packages/01/95/ae9f9cb7f748e47fedae2813332b65eb2201b880524bbbfb141c3b762135/microbench-0.4.1.tar.gz"
            }
        ],
        "0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6674606e04c8be55e3c3b717a2136f9e",
                    "sha256": "a4bd186b309f86668733168d69d484acff5a27ca69a8ee40e264701cacb6b299"
                },
                "downloads": -1,
                "filename": "microbench-0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "6674606e04c8be55e3c3b717a2136f9e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25762,
                "upload_time": "2019-03-14T20:53:29",
                "url": "https://files.pythonhosted.org/packages/0f/a1/2a679d7be2be0b8691926b9e6b122e6c386c2f92a1522c71243c1c7e789d/microbench-0.5.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "6674606e04c8be55e3c3b717a2136f9e",
                "sha256": "a4bd186b309f86668733168d69d484acff5a27ca69a8ee40e264701cacb6b299"
            },
            "downloads": -1,
            "filename": "microbench-0.5.tar.gz",
            "has_sig": false,
            "md5_digest": "6674606e04c8be55e3c3b717a2136f9e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 25762,
            "upload_time": "2019-03-14T20:53:29",
            "url": "https://files.pythonhosted.org/packages/0f/a1/2a679d7be2be0b8691926b9e6b122e6c386c2f92a1522c71243c1c7e789d/microbench-0.5.tar.gz"
        }
    ]
}