{ "info": { "author": "Bryant Moscon", "author_email": "bmoscon@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7" ], "description": "Cryptostore\n===========\n\n|License| |Python| |PyPi|\n\nA storage engine for cryptocurrency market data. You supply the\nexchanges, data type (trade, book, etc), and trading pairs you're\ninterested in and Cryptostore does the rest!\n\nStores data to:\n\n- Parquet\n- `Arctic `__\n- Google Cloud Storage\n- Amazon S3\n- `InfluxDB `__\n- Elasticsearch\n\nRequirements\n~~~~~~~~~~~~\n\nCryptostore currently requires either Kafka or Redis to be installed.\nThe extra dependencies for your backend of choice must be installed as\nwell (eg ``pip install cryptostore[redis]``). Redis requires Redis\nStreams, which is supported in versions >= 5.0.\n\nRunning Cryptostore\n~~~~~~~~~~~~~~~~~~~\n\nOnce installed with pip, an executable is placed on the path, so you can\nsimply run ``cryptostore`` to start the collector. It requires a\n``config.yaml`` file. If its not in the current working directory, you\ncan specify the path to the config with the ``--config`` option.\n\nAn example `config `__, with documentation inline is\nprovided in the root of the repository. The config file is monitored by\ncryptostore, so you can change the options in the file and it will apply\nthem without the need to reload the service (this is experimental. If\nyou encounter issues with it, please raise an issue).\n\nBackfilling Trade Data\n~~~~~~~~~~~~~~~~~~~~~~\n\nCryptstore can backfill trade data - but be aware not all exchanges\nsupport historical trade data, and some only provide a limited amount.\nBackfill fills from the earliest date in data storage until the start\ndate specified in the config. Backfill is restartable.\n\nRunning with other consumers\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nCryptostore can operate with other consumers of the exchange data (eg. a\ntrading engine consuming updates).\n\nFor Redis\n\n- Disable the message removal in the Redis settings in ``config.yaml``.\n The other consumer will need to be responsible for message removal\n (if so desired), and it must ensure messages are not removed before\n cryptostore has had a chance to process them.\n\nFor Kafka\n\n- You need only supply a different consumer group id for the other\n consumers to ensure all consumers receive all messages. Kafka's\n configuration controls the removal of committed messages in a topic\n (typically by time or size).\n\nWith a pass through\n\n- Cryptostore supports forwarding realtime data using ZeroMQ. To\n enable, use the ``pass_through`` option in the config. Data will be\n sent in real time (not subject to aggregation in redis/kafka). This\n can be used with or without data aggregation and storage.\n\nRunning in a container\n~~~~~~~~~~~~~~~~~~~~~~\n\nYou can run Cryptostore in a docker container. A Dockerfile and a\ndocker-compose.yml are provided. It uses the config in\nconfig-docker.yaml, and its set up to use redis and store the data into\nArctic/MongoDB. The port is mapped to 37017 (as opposed to 27017) so\nwhen connecting to Arctic from outside the container make sure you\nspecify the port. Additionally, a volume should be configured in the\ndocker-compose so that the mongoDB data will persist across restarts.\n\nPlanned features\n~~~~~~~~~~~~~~~~\n\n- \u2610 Missing data detection and correction (for exchanges that support\n historical data, typically only trade data)\n- \u2610 Storing data to MongoDB\n- \u2610 Support for enabling computation and storage of diverse metrics in\n parallel with data collection (eg. configurable OHLCV)\n\nContributing\n------------\n\nIssues and PRs are welcomed. If you'd like to discuss ongoing\ndevelopment please join the\n`slack `__\n(use the #cryptostore channel).\n\n.. |License| image:: https://img.shields.io/badge/license-XFree86-blue.svg\n :target: LICENSE\n.. |Python| image:: https://img.shields.io/badge/Python-3.6+-green.svg\n.. |PyPi| image:: https://img.shields.io/badge/PyPi-cryptostore-brightgreen.svg\n :target: https://pypi.python.org/pypi/cryptostore\n\n\n", "description_content_type": "text/x-rst", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/bmoscon/cryptostore", "keywords": "cryptocurrency,bitcoin,btc,market data,data storage,redis,database,kafka", "license": "XFree86", "maintainer": "", "maintainer_email": "", "name": "cryptostore", "package_url": "https://pypi.org/project/cryptostore/", "platform": "", "project_url": "https://pypi.org/project/cryptostore/", "project_urls": { "Homepage": "https://github.com/bmoscon/cryptostore" }, "release_url": "https://pypi.org/project/cryptostore/0.1.0/", "requires_dist": [ "pandas", "cryptofeed (>=1.0.0)", "pyyaml", "pyarrow", "aiohttp", "arctic ; extra == 'arctic'", "boto3 ; extra == 'aws'", "google-cloud-storage ; extra == 'gcs'", "aiokafka ; extra == 'kafka'", "confluent-kafka ; extra == 'kafka'", "redis ; extra == 'redis'", "aioredis ; extra == 'redis'", "pyzmq ; extra == 'zmq'" ], "requires_python": "", "summary": "Storage engine for cryptocurrency data", "version": "0.1.0" }, "last_serial": 5712295, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "f4efe5f5d7f0accefb9191765fcad9c1", "sha256": "9d21ba9a4ae5fe1fe33b9086a386332644da5dcaca5ccf1f2f39be8a5228e6dc" }, "downloads": -1, "filename": "cryptostore-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "f4efe5f5d7f0accefb9191765fcad9c1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6259, "upload_time": "2019-04-14T19:01:12", "url": "https://files.pythonhosted.org/packages/22/1c/6d87a53dd836aecced7352af5b6079c9764cd73473b3297f7c966743ff74/cryptostore-0.0.1-py3-none-any.whl" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "35542263020d6fdb9340f1c49175f51f", "sha256": "4bda46a00677cada410822df7faa1aad60ecfa6d71f72fee35c0c9bcd5107f78" }, "downloads": -1, "filename": "cryptostore-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "35542263020d6fdb9340f1c49175f51f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6252, "upload_time": "2019-04-14T19:01:55", "url": "https://files.pythonhosted.org/packages/b1/96/bcb1a1123c9bb6301f77a877c3019f9e323d2736034647f28b20102d4b5e/cryptostore-0.0.2-py3-none-any.whl" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "aa308fd227e76e45b53604eb421bb315", "sha256": "0ae047a2a2fdee8b2e202e8e8f1e5e9aaf713060ee54e9ac31a3890cdf225d8a" }, "downloads": -1, "filename": "cryptostore-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "aa308fd227e76e45b53604eb421bb315", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 11238, "upload_time": "2019-05-02T23:58:42", "url": "https://files.pythonhosted.org/packages/aa/6f/72ffcbf6c5ad3eb770d7634dc3852a9450607446a32b23ba03563fe3d420/cryptostore-0.0.3-py3-none-any.whl" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "a9c6d64dfeaeee7d3ecb1dcaa8df8571", "sha256": "67369b531f927d794228ad7e70da0fc8334d23b0fc71d944301b05c0937cdc7c" }, "downloads": -1, "filename": "cryptostore-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "a9c6d64dfeaeee7d3ecb1dcaa8df8571", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12065, "upload_time": "2019-05-04T19:31:17", "url": "https://files.pythonhosted.org/packages/a1/57/5f74d48fd37aee6b7584a9aaaf5efafab1b5bcca9f88304282b4f5dcc7f9/cryptostore-0.0.4-py3-none-any.whl" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "7a967c03f94dc097efe1b37106074769", "sha256": "44ec14d874c8d5e66165cbe53d54de8860b0ff139c07993c2db8c804d44bec6f" }, "downloads": -1, "filename": "cryptostore-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "7a967c03f94dc097efe1b37106074769", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14635, "upload_time": "2019-05-08T22:47:15", "url": "https://files.pythonhosted.org/packages/c7/e2/6a17a1905436ea2b94619570bc3d9fb9d5c2b9695b5927e4bbe121f14904/cryptostore-0.0.5-py3-none-any.whl" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "42368c87870e2b717b2cd6e02ca195f4", "sha256": "b2f0ec6760cbc0130bb708deb763312ea6a432f1201b7d6e23bb5e0eb00280f3" }, "downloads": -1, "filename": "cryptostore-0.0.6-py3-none-any.whl", "has_sig": false, "md5_digest": "42368c87870e2b717b2cd6e02ca195f4", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 17243, "upload_time": "2019-05-19T20:32:09", "url": "https://files.pythonhosted.org/packages/5a/c7/3928ae735ede41408d0180ece1f59d06dd28832e20ca16f0a6b58c44f530/cryptostore-0.0.6-py3-none-any.whl" } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "4cb07ba3b2dec1ffd1381448050e6fad", "sha256": "344088bb2372f8f00be72c6a7dc52687b1da1a7fe16d086801462ee25d617ef3" }, "downloads": -1, "filename": "cryptostore-0.0.7-py3-none-any.whl", "has_sig": false, "md5_digest": "4cb07ba3b2dec1ffd1381448050e6fad", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 17947, "upload_time": "2019-05-23T22:18:02", "url": "https://files.pythonhosted.org/packages/a9/0f/c7e1c4ed7f0946f3a18fd7ec0dc72e2073fd99324931d8e96f7af759e3af/cryptostore-0.0.7-py3-none-any.whl" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "a79b4eef4c85a19f6c16b691da27c2cd", "sha256": "f0393e5c474e32cdadf268c8823ba800edf98e496d26385c054f92e2ae00e589" }, "downloads": -1, "filename": "cryptostore-0.0.8-py3-none-any.whl", "has_sig": false, "md5_digest": "a79b4eef4c85a19f6c16b691da27c2cd", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 24153, "upload_time": "2019-07-06T23:24:39", "url": "https://files.pythonhosted.org/packages/e4/67/0323d7b6f913c98cb6c5460c9bd860ef09a145b2ba22da1e0043096c952a/cryptostore-0.0.8-py3-none-any.whl" } ], "0.1.0": [ { "comment_text": "", "digests": { "md5": "aba3c96f65f47230013e34f220a28da9", "sha256": "149bf9159be921d93e54151ba050978b169f741e08771f59b36e9a485895a761" }, "downloads": -1, "filename": "cryptostore-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "aba3c96f65f47230013e34f220a28da9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26188, "upload_time": "2019-08-21T23:29:54", "url": "https://files.pythonhosted.org/packages/e9/31/7b05b237b5c4566b4738764e209c8777d2bd2e1c58bfc2d92e175e6d7de7/cryptostore-0.1.0-py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "aba3c96f65f47230013e34f220a28da9", "sha256": "149bf9159be921d93e54151ba050978b169f741e08771f59b36e9a485895a761" }, "downloads": -1, "filename": "cryptostore-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "aba3c96f65f47230013e34f220a28da9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26188, "upload_time": "2019-08-21T23:29:54", "url": "https://files.pythonhosted.org/packages/e9/31/7b05b237b5c4566b4738764e209c8777d2bd2e1c58bfc2d92e175e6d7de7/cryptostore-0.1.0-py3-none-any.whl" } ] }