{ "info": { "author": "Joosep Pata", "author_email": "joosep.pata@cern.ch", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: BSD License", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: CPython" ], "description": "[![Build Status](https://travis-ci.com/hepaccelerate/hepaccelerate.svg?branch=master)](https://travis-ci.com/hepaccelerate/hepaccelerate)\n[![pipeline status](https://gitlab.cern.ch/jpata/hepaccelerate/badges/master/pipeline.svg)](https://gitlab.cern.ch/jpata/hepaccelerate/commits/master)\n[![DOI](https://zenodo.org/badge/191644111.svg)](https://zenodo.org/badge/latestdoi/191644111)\n\n# hepaccelerate\n\n- HEP data analysis with [jagged arrays](https://github.com/scikit-hep/awkward-array) using python + [Numba](http://numba.pydata.org/)\n- Use **any ntuple**, as long as you can open it with [uproot](https://github.com/scikit-hep/uproot)\n- analyze a billion events with systematic to histograms in minutes on a single workstation\n - 1e9 events / (50 kHz x 24 threads) ~ 13 minutes\n- weighted histograms, deltaR matching and [more](https://github.com/hepaccelerate/hepaccelerate#kernels)\n- use a CPU or an nVidia CUDA GPU with the same interface!\n- this is **not** an analysis framework, but rather a small set of helpers for fast jagged array processing\n\n**Under active development and use by a few CMS analyses!**\n\n

\n \"Kernel\n \"Analysis\n

\n\nMore details are available:\n- writeup: https://arxiv.org/abs/1906.06242v2\n- PyHEP 2019 talk: https://indico.cern.ch/event/833895/contributions/3577804/attachments/1927026/3192574/2019_10_15_pyhep.pdf\n\n## Installation\n\nThe library can be installed using `pip` in python >3.6:\n```bash\npip install hepaccelerate\n```\n\nYou may also clone this library as a part of your project, in which case you will need:\n - `pip install uproot numba xxhash lz4`\n\nOptional libraries, which may be easier to install with conda:\n - `cupy` for GPU support\n - `cudatoolkit` for GPU support\n - `dask` for running the large-scale example\n - `xxhash` for LZ4 support\n\n## Documentation\nThis code consists of two parts which can be used independently:\n - the accelerated HEP kernels that run on jagged data:\n - multithreaded CPU: [backend_cpu.py](https://github.com/hepaccelerate/hepaccelerate/blob/master/hepaccelerate/backend_cpu.py)\n - CUDA GPU: [backend_cuda.py](https://github.com/hepaccelerate/hepaccelerate/blob/master/hepaccelerate/backend_cuda.py) \n - JaggedStruct, Dataset and Histogram classes to help with HEP dataset management\n\n## Kernels\n\nThe jagged kernels work on the basis of the `content` and `offsets` arrays based on `awkward.JaggedArray`.\n\nWe have implemented the following kernels for both the CPU and CUDA backends:\n - `ha.min_in_offsets(offsets, content, mask_rows, mask_content)`: retrieve the minimum value in a jagged array, given row and object masks\n - `ha.max_in_offsets(offsets, content, mask_rows, mask_content)`: as above, but find the maximum\n - `ha.prod_in_offsets(offsets, content, mask_rows, mask_content, dtype=None)`: compute the product in a jagged array\n - `ha.set_in_offsets(content, offsets, indices, target, mask_rows, mask_content)`: set the indexed value in a jagged array to a target\n - `ha.get_in_offsets(offsets, content, indices, mask_rows, mask_content)`: retrieve the indexed values in a jagged array, e.g. get the leading jet pT\n - `ha.compute_new_offsets(offsets_old, mask_objects, offsets_new)`: given an awkward offset array and a mask, create an offset array of the unmasked elements\n - `ha.searchsorted(bins, vals, side=\"left\")`: 1-dimensional search in a sorted array\n - `ha.histogram_from_vector(data, weights, bins, mask=None)`: fill a 1-dimensional weighted histogram with arbitrary sorted bins, possibly using a mask\n - `ha.histogram_from_vector_several(variables, weights, mask)`: fill several histograms simultaneously based on `variables=[(data0, bins0), ...]`\n - `ha.get_bin_contents(values, edges, contents, out)`: look up the bin contents of a histogram based on a vector of values \n - `ha.select_opposite_sign(muons, in_mask)`: select the first pair with opposite sign charge\n - `ha.mask_deltar_first(objs1, mask1, objs2, mask2, drcut)`: given two collections of objects defined by eta, phi and offsets, mask the objects in the first collection that satisfy `DeltaR(o1, o2) < drcut)`\n\n## Usage\n\nThis is a minimal example from [examples/simple_hzz.py](../blob/master/examples/simple_hzz.py), which can be run from this repository directly using\n\n```bash\nPYTHONPATH=. python3 examples/simple_hzz.py\n\n#on GPU\nPYTHONPATH=. HEPACCELERATE_CUDA=1 python3 examples/simple_hzz.py\n```\n\n## Example analysis\nFor an example top quark pair analysis on ~200GB of CMS Open Data, please see [full_analysis.py](https://github.com/hepaccelerate/hepaccelerate/blob/master/examples/full_analysis.py). The following methods are implementd using both the CPU and GPU backends:\n- event and object selection\n- on-the-fly variation of jets using mockup jet energy corrections\n- reconstruction of the jet triplet with invariant mass closest to the top quark mass\n- signal-to-background DNN evaluation\n- filling around 20 control histograms with systematic variations\n\n## Recommendations on data locality and remote data\nIn order to make full use of modern CPUs or GPUs, you want to bring the data as close as possible to where the work is done, otherwise you will spend most of the time waiting for the data to arrive rather than actually performing the computations.\n\nWith CMS NanoAOD with event sizes of 1-2 kB/event, 1 million events is approximately 1-2 GB on disk. Therefore, you can fit a significant amount of data used in a HEP analysis on a commodity SSD.\nIn order to copy the data to your local disk, use grid tools such as `gfal-copy` or even `rsync` to fetch it from your nearest Tier2. Preserving the filename structure (`/store/...`) will allow you to easily run the same code on multiple sites.\n\n## Frequently asked questions\n\n - *Why are you doing this array-based analysis business?* Mainly out of curiosity, and I could not find a tool available with which I could do HEP analysis on data on a local disk with MHz rates. It is possible that dask/spark/RDataFrame will soon work well enough for this purpose, but until then, I can justify writing a few functions.\n - *How does this relate to the awkward-array project?* We use the jagged structure provided by the awkward arrays, but implement common HEP functions such as deltaR matching as parallelizable loops or 'kernels' running directly over the array contents, taking into account the event structure. We make these loops fast with Numba, but allow you to debug them by going back to standard python when disabling the compilation.\n - *How does this relate to the coffea/fnal-columnar-analysis-tools project?* It's very similar, you should check out that project! We implement less methods, mostly by explicit loops in Numba, and on GPUs as well as CPUs.\n - *Why don't you use the array operations (`JaggedArray.sum`, `argcross` etc) implemented in awkward-array?* They are great! However, in order to easily use the same code on either the CPU or GPU, we chose to implement the most common operations explicitly, rather than relying on numpy/cupy to do it internally. This also seems to be faster, at the moment.\n - *What if I don't have access to a GPU?* You should still be able to see event processing speeds in the hundreds of kHz to a few MHz for common analysis tasks.\n - *How do I plot my histograms that are saved in the output JSON?* Load the JSON contents and use the `edges` (left bin edges, plus last rightmost edge), `contents` (weighted bin contents) and `contents_w2` (bin contents with squared weights, useful for error calculation) to access the data directly.\n - *I'm a GPU programming expert, and I worry your CUDA kernels are not optimized. Can you comment?* Good question! At the moment, they are indeed not very optimized, as we do a lot of control flow (`if` statements) in them. However, the GPU analysis is still about 2x faster than a pure CPU analysis, as the CPU is more free to work on loading the data, and this gap is expected to increase as the analysis becomes more complicated (more systematics, more templates). At the moment, we see pure GPU processing speeds of about 8-10 MHz for in-memory data, and data loading from cache at about 4-6 MHz. Have a look at the nvidia profiler results [nvprof1](profiling/nvprof1.png), [nvprof2](profiling/nvprof2.png) to see what's going on under the hood. Please give us a hand to make it even better!\n - *What about running this code on multiple machines?* You can do that, currently just using usual batch tools, but we are looking at other ways (dask, joblib, spark) to distribute the analysis across multiple machines. \n - *What about running this code on data that is remote (XROOTD)?* You can do that thanks to the `uproot` library, but then you gain very little benefit from having a fast CPU or GPU, as you will spend most of your time just waiting for input.\n - *How much RAM is needed?* The amount of RAM determines how much data can be preloaded to memory. You can either process data in memory all at once, which makes rerunning very fast, or set up a batched pipeline. In case of the batched pipeline, no more than a few GB/thread of RAM is needed, and overall processing speeds are still around the MHz-level.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/hepaccelerate/hepaccelerate", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "hepaccelerate", "package_url": "https://pypi.org/project/hepaccelerate/", "platform": "", "project_url": "https://pypi.org/project/hepaccelerate/", "project_urls": { "Homepage": "https://github.com/hepaccelerate/hepaccelerate" }, "release_url": "https://pypi.org/project/hepaccelerate/0.2.3/", "requires_dist": [ "uproot", "awkward", "numba", "xxhash", "lz4", "requests", "cupy; extra == 'cuda'", "cudatoolkit; extra == 'cuda'", "dask; extra == 'dask'", "distributed; extra == 'dask'" ], "requires_python": ">=3.6.0", "summary": "Fast kernels for analyzing jagged columnar data common in high-energy physics.", "version": "0.2.3", "yanked": false, "yanked_reason": null }, "last_serial": 6255480, "releases": { "0.2.1": [ { "comment_text": "", "digests": { "md5": "9ec19db9a499deea6fc0389a144c7a66", "sha256": "6e0eea83a9c1424c9b9703000125c793c2a2e25be77b12f40d3561b8bd709ba8" }, "downloads": -1, "filename": "hepaccelerate-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9ec19db9a499deea6fc0389a144c7a66", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 23441, "upload_time": "2019-10-24T18:43:04", "upload_time_iso_8601": "2019-10-24T18:43:04.635599Z", "url": "https://files.pythonhosted.org/packages/db/ca/9d0fd068ee6fbe344eb57a0514083123a731101730801e8b9c7e43ebce8a/hepaccelerate-0.2.1-py2.py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "9d1fa2a527e294d7f6e2c0e56a0b8435", "sha256": "3f47b06d95d34609bcf357390c0dccc138db1c5a1de352bff2aecb274400568f" }, "downloads": -1, "filename": "hepaccelerate-0.2.1.tar.gz", "has_sig": false, "md5_digest": "9d1fa2a527e294d7f6e2c0e56a0b8435", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 22974, "upload_time": "2019-10-24T18:43:06", "upload_time_iso_8601": "2019-10-24T18:43:06.973757Z", "url": "https://files.pythonhosted.org/packages/b3/34/ec96807879e153b7029e848d4836077a364e6a45c48ce69f574ecaa7609f/hepaccelerate-0.2.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "d450fd305d0f8b370b61c1e0a5b4bbf6", "sha256": "f28f3e8f3be139a6f811d530cc6814ad880842e5b32e87c4f3b380b16dc66e06" }, "downloads": -1, "filename": "hepaccelerate-0.2.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d450fd305d0f8b370b61c1e0a5b4bbf6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 25383, "upload_time": "2019-10-25T02:21:31", "upload_time_iso_8601": "2019-10-25T02:21:31.591496Z", "url": "https://files.pythonhosted.org/packages/d4/63/4d27c9448357f399b4dad0e18bdc0800b263a0a9286d99a1d75c790c91de/hepaccelerate-0.2.2-py2.py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "08952bc0ac83fc70d8623fbdbf73ebfe", "sha256": "edf1d137eb4d67816745056cbaf7db6c3357df811d68623e92e705eea9a966cf" }, "downloads": -1, "filename": "hepaccelerate-0.2.2.tar.gz", "has_sig": false, "md5_digest": "08952bc0ac83fc70d8623fbdbf73ebfe", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 24140, "upload_time": "2019-10-25T02:21:33", "upload_time_iso_8601": "2019-10-25T02:21:33.080248Z", "url": "https://files.pythonhosted.org/packages/54/1b/b1296e0bcda8b54b2fa9a3d65016f2cf2da267c14499bbb721ae8d0dddf7/hepaccelerate-0.2.2.tar.gz", "yanked": false, "yanked_reason": null } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "ed2e072b155664256fb66b716ff0d60a", "sha256": "ec15a750b1c328bc157bfd32b61ab17800349228aede3a92cca99fd096e14d0d" }, "downloads": -1, "filename": "hepaccelerate-0.2.3-py2-none-any.whl", "has_sig": false, "md5_digest": "ed2e072b155664256fb66b716ff0d60a", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": ">=3.6.0", "size": 39989, "upload_time": "2019-12-06T18:25:03", "upload_time_iso_8601": "2019-12-06T18:25:03.400191Z", "url": "https://files.pythonhosted.org/packages/c3/31/1501346c8d820b342f14fe7d0f93e4639ed8f6f4245b9ef019403f7c799d/hepaccelerate-0.2.3-py2-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "89cc9b5e8eaf2c89b8b76fa0282b7015", "sha256": "83f2af9d812d5a8c59e1625eb4e6ace2a53f45e39ca83878f643229536987ff3" }, "downloads": -1, "filename": "hepaccelerate-0.2.3.tar.gz", "has_sig": false, "md5_digest": "89cc9b5e8eaf2c89b8b76fa0282b7015", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 35431, "upload_time": "2019-12-06T18:25:05", "upload_time_iso_8601": "2019-12-06T18:25:05.308172Z", "url": "https://files.pythonhosted.org/packages/bc/eb/f84a57efc354cb80242d809879d06d4af20e590b5fd05ea98e30a93cac80/hepaccelerate-0.2.3.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ed2e072b155664256fb66b716ff0d60a", "sha256": "ec15a750b1c328bc157bfd32b61ab17800349228aede3a92cca99fd096e14d0d" }, "downloads": -1, "filename": "hepaccelerate-0.2.3-py2-none-any.whl", "has_sig": false, "md5_digest": "ed2e072b155664256fb66b716ff0d60a", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": ">=3.6.0", "size": 39989, "upload_time": "2019-12-06T18:25:03", "upload_time_iso_8601": "2019-12-06T18:25:03.400191Z", "url": "https://files.pythonhosted.org/packages/c3/31/1501346c8d820b342f14fe7d0f93e4639ed8f6f4245b9ef019403f7c799d/hepaccelerate-0.2.3-py2-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "89cc9b5e8eaf2c89b8b76fa0282b7015", "sha256": "83f2af9d812d5a8c59e1625eb4e6ace2a53f45e39ca83878f643229536987ff3" }, "downloads": -1, "filename": "hepaccelerate-0.2.3.tar.gz", "has_sig": false, "md5_digest": "89cc9b5e8eaf2c89b8b76fa0282b7015", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 35431, "upload_time": "2019-12-06T18:25:05", "upload_time_iso_8601": "2019-12-06T18:25:05.308172Z", "url": "https://files.pythonhosted.org/packages/bc/eb/f84a57efc354cb80242d809879d06d4af20e590b5fd05ea98e30a93cac80/hepaccelerate-0.2.3.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }