{
    "info": {
        "author": "U.S. Government",
        "author_email": "",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7",
            "Topic :: Scientific/Engineering"
        ],
        "description": "# Arkouda (\u03b1\u03c1\u03ba\u03bf\u03cd\u03b4\u03b1): NumPy-like arrays at massive scale backed by Chapel.\n## _NOTE_: Arkouda is under the MIT license.\n\n## Gitter channels\n[Arkouda Gitter channel](https://gitter.im/ArkoudaProject/community)\n\n[Chapel Gitter channel](https://gitter.im/chapel-lang/chapel)\n\n## Talks on Arkouda\n[Mike Merrill's CHIUW 2019 talk](https://chapel-lang.org/CHIUW/2019/Merrill.pdf)\n\n[Bill Reus' CLSAC 2019 talk](http://www.clsac.org/uploads/5/0/6/3/50633811/2019-reus-arkuda.pdf)\n\n(PAW-ATM) [talk](https://github.com/sourceryinstitute/PAW/raw/gh-pages/PAW-ATM19/presentations/PAW-ATM2019_talk11.pdf) \nand [abstract](https://github.com/sourceryinstitute/PAW/raw/gh-pages/PAW-ATM19/extendedAbstracts/PAW-ATM2019_abstract5.pdf)\n\n## Abstract:\nExploratory data analysis (EDA) is a prerequisite for all data\nscience, as illustrated by the ubiquity of Jupyter notebooks, the\npreferred interface for EDA among data scientists. The operations\ninvolved in exploring and transforming the data are often at least as\ncomputationally intensive as downstream applications (e.g. machine\nlearning algorithms), and as datasets grow, so does the need for HPC-enabled\nEDA. However, the inherently interactive and open-ended nature of\nEDA does not mesh well with current HPC usage models. Meanwhile, several\nexisting projects from outside the traditional HPC space attempt to\ncombine interactivity and\ndistributed computation using programming paradigms and tools from\ncloud computing, but none of these projects have come close to meeting\nour needs for high-performance EDA.\n\nTo fill this gap, we have\ndeveloped a software package, called Arkouda, which allows a user to\ninteractively issue massively parallel computations on distributed\ndata using functions and syntax that mimic NumPy, the underlying\ncomputational library used in the vast majority of Python data science\nworkflows. The computational heart of Arkouda is a Chapel interpreter\nthat\naccepts a pre-defined set of commands from a client (currently\nimplemented in Python) and\nuses Chapel's built-in machinery for multi-locale and multithreaded\nexecution. Arkouda has benefited greatly from Chapel's distinctive\nfeatures and has also helped guide the development of the language.\n\nIn early applications, users of Arkouda have tended to iterate rapidly\nbetween multi-node execution with Arkouda and single-node analysis in\nPython, relying on Arkouda to filter a large dataset down to a smaller\ncollection suitable for analysis in Python, and then feeding the results\nback into Arkouda computations on the full dataset. This paradigm has\nalready proved very fruitful for EDA. Our goal is to enable users to\nprogress seamlessly from EDA to specialized algorithms by making Arkouda\nan integration point for HPC implementations of expensive kernels like\nFFTs, sparse linear algebra, and graph traversal. With Arkouda serving\nthe role of a shell, a data scientist could explore, prepare, and call\noptimized HPC libraries on massive datasets, all within the same\ninteractive session.\n\n## Installation\n\n### Requirements:\n * requires chapel 1.22.0\n * requires zeromq version >= 4.2.5, tested with 4.2.5 and 4.3.1\n * requires hdf5 \n * requires python 3.6 or greater\n * requires numpy\n * requires Sphinx and sphinx-argparse to build python documentation\n * requires pytest and pytest-env to execute the Python test harness\n\n### MacOS Environment Installation\n\nIt is usually very simple to get things going on a mac:\n\n```bash\nbrew install zeromq\n\nbrew install hdf5\n\nbrew install chapel\n\n# Although not required, is is highly recommended to install Anaconda to provide a \n# Python 3 environment and manage Python dependencies:\nwget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh\nsh Anaconda3-2020.02-Linux-x86_64.sh\nsource ~/.bashrc\n\n# Otherwise, Python 3 can be installed with brew\nbrew install python3\n# !!! the standard way of installing through pip3 installs an old version of arkouda\n# !!! the arkouda python client is available via pip\n# !!! pip will automatically install python dependencies (zmq and numpy)\n# !!! however, pip will not build the arkouda server (see below)\n# !!!pip3 install arkouda\n#\n# install the version of the arkouda python package which came with the arkouda_server\n# if you plan on editing the arkouda python package use the -e flag\n# from the local arkouda repo/directory run...\npip3 install -e .\n#\n# these packages are nice but not a requirement\npip3 install pandas\npip3 install jupyter\n```\n\nIf it is preferred to build Chapel instead of using the brew install, the process is as follows:\n\n```bash\n\n# on my mac build chapel in my home directory with these settings...\nexport CHPL_HOME=~/chapel/chapel-1.22.0\nsource $CHPL_HOME/util/setchplenv.bash\nexport CHPL_COMM=gasnet\nexport CHPL_COMM_SUBSTRATE=smp\nexport CHPL_TARGET_CPU=native\nexport GASNET_QUIET=Y\nexport CHPL_RT_OVERSUBSCRIBED=yes\ncd $CHPL_HOME\nmake\n\n# Build chpldoc to enable generation of Arkouda docs\nmake chpldoc\n\n# Add the Chapel executable (chpl) to PATH either in ~/.bashrc (single user) \n# or /etc/environment (all users):\n\nexport PATH=$CHPL_HOME/bin/linux64-x86_64/:$PATH\n```\n\n### Linux Environment Installation\n\nThere is no Linux Chapel install, so the first two steps in the Linux Arkouda install are \nto install the Chapel dependencies followed by downloading and building Chapel:\n\n```\n# Update Linux kernel and install Chapel dependencies\nsudo apt-get update\nsudo apt-get install gcc g++ m4 perl python python-dev python-setuptools bash make mawk git pkg-config\n\n# Download latest Chapel release, explode archive, and navigate to source root directory\nwget https://github.com/chapel-lang/chapel/releases/download/1.22.0/chapel-1.22.0.tar.gz\ntar xvf chapel-1.22.0.tar.gz\ncd chapel-1.22.0/\n\n# Set CHPL_HOME\nexport CHPL_HOME=$PWD\n\n# Add chpl to PATH\nsource $CHPL_HOME/util/setchplenv.bash\n\n# Set remaining env variables and execute make\nexport CHPL_COMM=gasnet\nexport CHPL_COMM_SUBSTRATE=smp\nexport CHPL_TARGET_CPU=native\nexport GASNET_QUIET=Y\nexport CHPL_RT_OVERSUBSCRIBED=yes\ncd $CHPL_HOME\nmake\n\n# Build chpldoc to enable generation of Arkouda docs\nmake chpldoc\n\n# Optionally add the Chapel executable (chpl) to the PATH for all users: /etc/environment\nexport PATH=$CHPL_HOME/bin/linux64-x86_64/:$PATH\n```\n\nAs is the case with the MacOS install, it is highly recommended to install Anaconda to provide a Python environment \nand manage Python dependencies:\n\n```\n wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh\n sh Anaconda3-2020.02-Linux-x86_64.sh\n source ~/.bashrc\n```\n\n## Building Arkouda\n\nDownload, clone, or fork the [arkouda repo](https://github.com/mhmerrill/arkouda). Further instructions assume that \nthe current directory is the top-level directory of the repo.\n\nIf your environment requires non-system paths to find dependencies (e.g.,\nif using the ZMQ and HDF5 bundled with [Anaconda]), append each path to a new file `Makefile.paths` like so:\n\n```make\n# Makefile.paths\n\n# Custom Anaconda environment for Arkouda\n$(eval $(call add-path,/home/user/anaconda3/envs/arkouda))\n#                      ^ Note: No space after comma.\n```\n\nThe `chpl` compiler will be executed with `-I`, `-L` and an `-rpath` to each path.\n\n```\n# If zmq and hdf5 have not been installed previously, execute make install-deps\nmake install-deps\n\n# Run make to build the arkouda_server executable\nmake\n```\n\nNow that the arkouda_server is built and tested, install the Python library\n\n## Installing the Arkouda Python Library\n\n```\n pip3 install -e .\n```\n\n## Testing Arkouda\n\nThere are two unit test suites for Arkouda, one for Python and one for Chapel. As mentioned above, the Arkouda Python \ntest harness leverages the [pytest](https://docs.pytest.org/en/latest/) and [pytest-env](https://pypi.org/project/pytest-env/) \nlibraries, whereas the Chapel test harness does not require any external librares.\n\nFor more details regarding Arkouda testing, please consult the Python test [README](tests/README.md) and Chapel test\n[README](test/README.md), respectively.\n\n## Building the Arkouda documentation\n\nMake sure you've installed the Sphinx and sphinx-argparse packages (e.g. `pip3 install -U Sphinx sphinx-argparse`). _Important: if you've built Chapel, you must execute make chpldoc as detailed above._\n\nRun `make doc` to build both the Arkouda python documentation and the Chapel server documentation\n\nThe output is currently in subdirectories of the `arkouda/doc`\n```\narkouda/doc/python # python frontend documentation\narkouda/doc/server # chapel backend server documentation \n```\n\nTo view the documentation for the Arkouda python client, point your browser to `file:///path/to/arkouda/doc/python/index.html`, \nsubstituting the appropriate path for your configuration.\n\n## Running arkouda_server\n\nThe command-line invocation depends on whether you built a single-locale version (with `CHPL_COMM=none`) or \nmulti-locale version (with `CHPL_COMM` set).\n\nSingle-locale startup:\n\n```bash\n./arkouda_server\n```\n\nMulti-locale startup (user selects the number of locales):\n\n```bash\n./arkouda_server -nl 2\n```\nAlso can run server with memory checking turned on using\n\n```bash\n./arkouda_server --memTrack=true\n```\n\nBy default, the server listens on port `5555` and prints verbose output. These options can be changed with command-line \nflags `--ServerPort=1234` and `--v=false`\n\nMemory checking is turned off by default and turned on by using `--memTrack=true`\n\nLogging messages are turned on by default and turned off by using `--logging=false`\n\nVerbose messages are turned on by default and turned off by using  `--v=false`\n\nOther command line options are available, view them by using `--help`\n\n## Testing arkouda_server\n\nTo sanity check the arkouda server, you can run\n\n```bash\nmake check\n```\n\nThis will start the server, run a few computations, and shut the server down. In addition, the check script can be executed \nagainst a running server by running the following Python command:\n\n```bash\npython3 tests/check.py localhost 5555\n```\n\n## Contributing to Arkouda\n\nIf you'd like to contribute, please see [CONTRIBUTING.md](CONTRIBUTING.md).",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/mhmerrill/arkouda",
        "keywords": "HPC workflow exploratory analysis parallel distribute arrays Chapel",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "arkouda",
        "package_url": "https://pypi.org/project/arkouda/",
        "platform": "",
        "project_url": "https://pypi.org/project/arkouda/",
        "project_urls": {
            "Bug Reports": "https://github.com/mhmerrill/arkouda/issues",
            "Chapel": "https://chapel-lang.org",
            "Homepage": "https://github.com/mhmerrill/arkouda",
            "Source": "https://github.com/mhmerrill/arkouda"
        },
        "release_url": "https://pypi.org/project/arkouda/2020.7.7/",
        "requires_dist": null,
        "requires_python": ">=3.6",
        "summary": "Parallel, distributed NumPy-like arrays backed by Chapel",
        "version": "2020.7.7",
        "yanked": false,
        "yanked_reason": null
    },
    "last_serial": 7648326,
    "releases": {
        "0.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d74227645ed39654ea9299114d1a32fa",
                    "sha256": "23579065b5aea65ce2c6a084e28a4175b4a1c9a5bb5f2ae3728608dcf224c1cd"
                },
                "downloads": -1,
                "filename": "arkouda-0.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "d74227645ed39654ea9299114d1a32fa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 985,
                "upload_time": "2019-03-25T18:21:10",
                "upload_time_iso_8601": "2019-03-25T18:21:10.809977Z",
                "url": "https://files.pythonhosted.org/packages/55/fb/7a8e6b11c4039d50c81d13f8f65fe2758a2f0cbdd7e0da83370c16a4ef15/arkouda-0.0.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "2019.10.22": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b7c2aa7a90a43b428178d296a4d22e1f",
                    "sha256": "524fb7b496ea0be07e503e50f349b372b528eb28654ffd15b7820191dc608aad"
                },
                "downloads": -1,
                "filename": "arkouda-2019.10.22-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b7c2aa7a90a43b428178d296a4d22e1f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5",
                "size": 25676,
                "upload_time": "2019-10-22T18:18:13",
                "upload_time_iso_8601": "2019-10-22T18:18:13.718870Z",
                "url": "https://files.pythonhosted.org/packages/ac/3b/e348fa5574c454c30bec79cc4fff4a52b15b04a1c331e958ecbf817d5899/arkouda-2019.10.22-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "39c654f893a6a2c2967b84f2b8a684c1",
                    "sha256": "d17e2996c236e683385e7707ca2336ac9004cd34de4f79685f30d827ab33b596"
                },
                "downloads": -1,
                "filename": "arkouda-2019.10.22.tar.gz",
                "has_sig": false,
                "md5_digest": "39c654f893a6a2c2967b84f2b8a684c1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5",
                "size": 118693,
                "upload_time": "2019-10-22T18:18:15",
                "upload_time_iso_8601": "2019-10-22T18:18:15.111190Z",
                "url": "https://files.pythonhosted.org/packages/4e/0a/aee212ec2f5978ca679d7d8cf31806a8a49d3017f3feec0355de55a2602d/arkouda-2019.10.22.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "2019.11.12": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e6a17260fbb22ac449ef549589123071",
                    "sha256": "6c748ea7f8b3448eb60696dc69a44b89f250247e07dd684c63401c1ac5a93878"
                },
                "downloads": -1,
                "filename": "arkouda-2019.11.12-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e6a17260fbb22ac449ef549589123071",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 26514,
                "upload_time": "2019-12-06T01:59:01",
                "upload_time_iso_8601": "2019-12-06T01:59:01.433883Z",
                "url": "https://files.pythonhosted.org/packages/5b/64/ff810d6eca7444dd02782aef8252029d2624c4810c9ef2f7ec540ad85134/arkouda-2019.11.12-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3f8cafc0eaa18f2b3feaac5d323a7925",
                    "sha256": "6e571fea209042f90668c04f27f3eb20a1c57bd558e13adf5c3e6a1a121a52b4"
                },
                "downloads": -1,
                "filename": "arkouda-2019.11.12.tar.gz",
                "has_sig": false,
                "md5_digest": "3f8cafc0eaa18f2b3feaac5d323a7925",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 133827,
                "upload_time": "2019-12-06T01:59:02",
                "upload_time_iso_8601": "2019-12-06T01:59:02.713003Z",
                "url": "https://files.pythonhosted.org/packages/a0/bf/929c342cb1e885af3a636f88393174988da9a6cbe1ace35e8b599bb94e61/arkouda-2019.11.12.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "2019.12.10": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2d55c780cb44e13eb2e202de0f3f0cc2",
                    "sha256": "5236b432049a05348b18457929e18cdb596820834a66d164c854dc553b6a1fb0"
                },
                "downloads": -1,
                "filename": "arkouda-2019.12.10-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "2d55c780cb44e13eb2e202de0f3f0cc2",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 30685,
                "upload_time": "2019-12-10T19:16:18",
                "upload_time_iso_8601": "2019-12-10T19:16:18.377398Z",
                "url": "https://files.pythonhosted.org/packages/03/37/f4c58ee046f0b1fe9a98ae51dc0677362f1d908154f606902357bd54b755/arkouda-2019.12.10-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3afe35ce776fe9e8b237b38ae7f84e23",
                    "sha256": "ad83e1767ec800409b5beaf41d5d11e020d7b38ab07021dbd6e9cf9f020b49f3"
                },
                "downloads": -1,
                "filename": "arkouda-2019.12.10.tar.gz",
                "has_sig": false,
                "md5_digest": "3afe35ce776fe9e8b237b38ae7f84e23",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 160056,
                "upload_time": "2019-12-10T19:16:21",
                "upload_time_iso_8601": "2019-12-10T19:16:21.334213Z",
                "url": "https://files.pythonhosted.org/packages/57/e6/df86650f0a45fba684e7657d9876d1bd26b044238b43705b1ad9b015d087/arkouda-2019.12.10.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "2020.2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "899e2cc96d30132f912cb0854cd55270",
                    "sha256": "fb9549224fadf94e61f38287d8928f7e5276196e3ce147488fe5c58f7e872c07"
                },
                "downloads": -1,
                "filename": "arkouda-2020.2.4.tar.gz",
                "has_sig": false,
                "md5_digest": "899e2cc96d30132f912cb0854cd55270",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 10663701,
                "upload_time": "2020-02-18T18:41:03",
                "upload_time_iso_8601": "2020-02-18T18:41:03.363679Z",
                "url": "https://files.pythonhosted.org/packages/8c/c6/2a250112155aa50dbdecb6717d3b5a31e30c3105e954d62e5feacbaa0b2e/arkouda-2020.2.4.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "2020.7.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7a3da8be338039724ff5a111a9f5feeb",
                    "sha256": "ae37d83009c84bd46a16cf71a1b84583e402a419503f5e6d0581fbe2de966aac"
                },
                "downloads": -1,
                "filename": "arkouda-2020.7.7.tar.gz",
                "has_sig": false,
                "md5_digest": "7a3da8be338039724ff5a111a9f5feeb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 10691522,
                "upload_time": "2020-07-07T14:41:31",
                "upload_time_iso_8601": "2020-07-07T14:41:31.745216Z",
                "url": "https://files.pythonhosted.org/packages/7a/7b/debc2432ee27236892f99cd17e8fe0974c2b913f74c12ceacf81559bc37d/arkouda-2020.7.7.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "7a3da8be338039724ff5a111a9f5feeb",
                "sha256": "ae37d83009c84bd46a16cf71a1b84583e402a419503f5e6d0581fbe2de966aac"
            },
            "downloads": -1,
            "filename": "arkouda-2020.7.7.tar.gz",
            "has_sig": false,
            "md5_digest": "7a3da8be338039724ff5a111a9f5feeb",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6",
            "size": 10691522,
            "upload_time": "2020-07-07T14:41:31",
            "upload_time_iso_8601": "2020-07-07T14:41:31.745216Z",
            "url": "https://files.pythonhosted.org/packages/7a/7b/debc2432ee27236892f99cd17e8fe0974c2b913f74c12ceacf81559bc37d/arkouda-2020.7.7.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "vulnerabilities": []
}