{ "info": { "author": "Knight", "author_email": "anonymousknight96@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5" ], "description": "CratesMirror\n============\n\n.. image:: https://img.shields.io/pypi/v/cratesmirror.svg\n :target: https://pypi.python.org/pypi/cratesmirror\n\nAbout\n=====\n\nDownload all crates on `crates.io `__\n\nRequirement\n============\n\n- Python >= 2.7.9\n- `requests `__\n- `GitPython `__\n- At least 4G free disk space for hosting local crates\n\nInstallation\n============\n\n::\n\n # pip install cratesmirror\n\nUsage\n======\n\n.. code-block:: bash\n\n $ cratesmirror -h\n\n usage: cratesmirror [-h] [-i INDEX] [-w CRATES] [-d DBPATH] [-f LOGFILE]\n [-c CHECKDB] [-v]\n\n optional arguments:\n -h, --help show this help message and exit\n -i, --index INDEX registry index directory (default: /srv/git/index)\n -w, --crates CRATES crates directory (default: /srv/www/crates)\n -d, --dbpath DBPATH database file path (default: None)\n -f, --logfile LOGFILE\n log file path (default: None)\n -c, --checkdb CHECKDB\n check database for missing crates (default: False)\n -v, --verbose\n\n Available environment variables: HTTP_PROXY, HTTPS_PROXY, CRATES_DL, CRATES_API\n\n\n Examples:\n # Download all crates only\n $ cratesmirror -d /var/lib/crates/crates.db -f /var/log/crates/debug.log\n\n # Find out missing crates, then update the repository\n $ cratesmirror --checkdb -d /var/lib/crates/crates.db -f /var/log/crates/debug.log\n\n # Update repo and commit custom settings\n $ CRATES_DL='https://crates.mirrors.ustc.edu.cn/api/v1/crates' \\\n cratesmirror -d /var/lib/crates/crates.db -f /var/log/crates/debug.log\n\n # Using proxy\n $ HTTPS_PROXY='https://127.0.0.1:8081' \\\n cratesmirror -d /var/lib/crates/crates.db -f /var/log/crates/debug.log\n\n\nOr use it in script\n\n.. code-block:: python\n\n from cratesmirror import CratesMirror\n\n indexdir = '/srv/git/index'\n cratesdir = '/srv/www/crates'\n config = {'dl': 'https://crates.mirrors.ustc.edu.cn/api/v1/crates',\n 'api': 'https://crates.io'}\n # By default, it will be saved at os.getcwd()/crates.db\n dbpath = '/var/lib/cratesmirror/crates.db'\n\n with CratesMirror(indexdir, cratesdir, config=config, dbpath=dbpath) as mirror:\n mirror.update_repo()\n\n # with proxy\n proxies = {\n \"http\": \"http://10.10.1.10:3128\",\n \"https\": \"http://10.10.1.10:1080\",\n }\n with CratesMirror(indexdir, cratesdir, config=config, proxy=proxies, dbpath=dbpath) as mirror:\n mirror.update_repo()\n\nNote\n======\n\n- By default, the script will:\n - assume that registry index directory is located at :code:`/srv/git/index`, and crates are saved at :code:`/srv/www/crates`\n - save downloaded crate as :code:`/{name}/{name}-{version}.crate`\n - save the database file at :code:`os.getcwd()/crates.db`\n- If the environment variable :code:`CRATES_DL` or :code:`CRATES_API` is set, its value will be saved at :code:`/config.json` and the changes will be committed automatically.\n- After the first run, all you need to do is to run this script periodically using crontab-like tools or systemd.timers to sync with upstream.\n\n\n.. :changelog:\n\nChangeLog\n---------------\n\n1.1.3\n+++++++++++\n\n**Bugfix**\n\n- Pick up all new crates in the modified index file, not only the latest\n\n1.1.2\n+++++++++++\n\n**Bugfix**\n\n- Replace None with default value in :code:`/config.json`\n\n1.1.1\n+++++++++++\n\n**Miscellaneous**\n\n- Add changelog\n\n1.1.0\n++++++++++++++++++\n\n**Improvement**\n\n- Always download crates using multithreading\n\n1.0.4\n++++++++++++++++++\n\n**Feature**\n\n- Add -c/--checkdb option, enable users to check database for missing crates\n\n1.0.3\n++++++++++++++++++\n\n**Improvement**\n\n- When is empty, download all crates in a multithreaded way\n\n1.0.2\n++++++++++++++++++\n\n- Naive crawler", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ustclug/LUG-crates_mirror", "keywords": null, "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "cratesmirror", "package_url": "https://pypi.org/project/cratesmirror/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/cratesmirror/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/ustclug/LUG-crates_mirror" }, "release_url": "https://pypi.org/project/cratesmirror/1.1.3/", "requires_dist": null, "requires_python": null, "summary": "Download all crates on https://crates.io", "version": "1.1.3" }, "last_serial": 2063357, "releases": { "1.0.2": [ { "comment_text": "", "digests": { "md5": "1c7c03315c5d93819ddb6e5aa4038df9", "sha256": "84ae2fe35f4c0dde5f3f11c4b186931597fc88922e5c9bffd481686d1bf9f42c" }, "downloads": -1, "filename": "cratesmirror-1.0.2.tar.gz", "has_sig": false, "md5_digest": "1c7c03315c5d93819ddb6e5aa4038df9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7652, "upload_time": "2016-01-23T15:17:54", "url": "https://files.pythonhosted.org/packages/27/71/9007b6812276be60d2f083562ad26d800c8dabc430032c697db97de6af3c/cratesmirror-1.0.2.tar.gz" } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "7abb6d42aac1d4c3fafbf5d3063cd427", "sha256": "ceb8f260d55a4d21c70059231c4dba1c2491560effd17fa40e3c81971f4dfa11" }, "downloads": -1, "filename": "cratesmirror-1.0.3.tar.gz", "has_sig": false, "md5_digest": "7abb6d42aac1d4c3fafbf5d3063cd427", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7599, "upload_time": "2016-01-23T15:44:07", "url": "https://files.pythonhosted.org/packages/31/be/1a6497b6b2ee7d511deefd041f1bd4f1ffc0d3335a03d286ef952aaf1590/cratesmirror-1.0.3.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "b4b0a09caf131b484d4665e7277fb5a6", "sha256": "b35ea2512558d6f7a16997b2ec455dfcce1b2330576fffa486c968ea5d6a427f" }, "downloads": -1, "filename": "cratesmirror-1.0.4.tar.gz", "has_sig": false, "md5_digest": "b4b0a09caf131b484d4665e7277fb5a6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8390, "upload_time": "2016-03-28T04:27:44", "url": "https://files.pythonhosted.org/packages/62/fc/9ef8a79f1726194bcfa2483fbe24688f8a91309c5d35789f562b5638bf7c/cratesmirror-1.0.4.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "c3bc0cc23d217ee73510b4abfc07dcef", "sha256": "5e4c525aac0ce2f9214e8f03648b1526ab1eee5bce081b4222a7c837d88353e2" }, "downloads": -1, "filename": "cratesmirror-1.1.0.tar.gz", "has_sig": false, "md5_digest": "c3bc0cc23d217ee73510b4abfc07dcef", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10443, "upload_time": "2016-04-03T01:40:43", "url": "https://files.pythonhosted.org/packages/6c/bc/2b719c888414a513e89081cf675aba8890d31001b72be62ab19b40f8d936/cratesmirror-1.1.0.tar.gz" } ], "1.1.1": [ { "comment_text": "", "digests": { "md5": "f9945d194d38e7ce53dcd4865cb25f6f", "sha256": "6b7c80860b75b1065354c39662d2fd8eb9a8a4379dc756cad09a8339fdc17252" }, "downloads": -1, "filename": "cratesmirror-1.1.1.tar.gz", "has_sig": false, "md5_digest": "f9945d194d38e7ce53dcd4865cb25f6f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11066, "upload_time": "2016-04-03T06:13:14", "url": "https://files.pythonhosted.org/packages/59/22/f55786dcdcc29a8069e3e12abf9865a047587c94d7a9edd0516a74a9f172/cratesmirror-1.1.1.tar.gz" } ], "1.1.2": [ { "comment_text": "", "digests": { "md5": "8b48c901f51782f143fd10c54cb39c6f", "sha256": "b206e2cf3e8bc9797c65304a4b5a2a6c7d8fa133ea62db342a688e2a19c438de" }, "downloads": -1, "filename": "cratesmirror-1.1.2.tar.gz", "has_sig": false, "md5_digest": "8b48c901f51782f143fd10c54cb39c6f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11160, "upload_time": "2016-04-03T10:32:39", "url": "https://files.pythonhosted.org/packages/bf/a2/a7ef9deb528d6e9c41a4118c4e5840214c252a4dee268bb752d95bd91a1f/cratesmirror-1.1.2.tar.gz" } ], "1.1.3": [ { "comment_text": "", "digests": { "md5": "ceb8c11b4021e25ced0c581500642ea2", "sha256": "a7afe8fba0c13d1cc9fb93fa0da9e50d8feda3337709b6337ce640e8ae4d780b" }, "downloads": -1, "filename": "cratesmirror-1.1.3.tar.gz", "has_sig": false, "md5_digest": "ceb8c11b4021e25ced0c581500642ea2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11379, "upload_time": "2016-04-14T08:49:22", "url": "https://files.pythonhosted.org/packages/ca/09/3f196abf4f7fa54bb6e0c3beaa2081cf0dbbb4708f94f54fd3edd361c091/cratesmirror-1.1.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ceb8c11b4021e25ced0c581500642ea2", "sha256": "a7afe8fba0c13d1cc9fb93fa0da9e50d8feda3337709b6337ce640e8ae4d780b" }, "downloads": -1, "filename": "cratesmirror-1.1.3.tar.gz", "has_sig": false, "md5_digest": "ceb8c11b4021e25ced0c581500642ea2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11379, "upload_time": "2016-04-14T08:49:22", "url": "https://files.pythonhosted.org/packages/ca/09/3f196abf4f7fa54bb6e0c3beaa2081cf0dbbb4708f94f54fd3edd361c091/cratesmirror-1.1.3.tar.gz" } ] }