{ "info": { "author": "Software Heritage developers", "author_email": "swh-devel@inria.fr", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# SWH Tarball Loader\n\nThe Software Heritage Tarball Loader is in charge of ingesting the directory\nrepresentation of the tarball into the Software Heritage archive.\n\n## Sample configuration\n\nThe loader's configuration will be taken from the default configuration file:\n`~/.config/swh/loader/tar.yml` (you can choose a different path by setting the\n`SWH_CONFIG_FILENAME` environment variable).\n\nThis file holds information for the loader to work, including celery\nconfiguration:\n\n```YAML\nworking_dir: /home/storage/tmp/\nstorage:\n cls: remote\n args:\n url: http://localhost:5002/\ncelery:\ntask_modules:\n - swh.loader.tar.tasks\ntask_queues:\n - swh.loader.tar.tasks.LoadTarRepository\n```\n\n### Local\n\nLoad local tarball directly from code or python3's toplevel:\n\n``` Python\n# Fill in those\nrepo = '8sync.tar.gz'\ntarpath = '/home/storage/tar/%s' % repo\norigin = {'url': 'file://%s' % repo, 'type': 'tar'}\nvisit_date = 'Tue, 3 May 2017 17:16:32 +0200'\nlast_modified = 'Tue, 10 May 2016 16:16:32 +0200'\nimport logging\nlogging.basicConfig(level=logging.DEBUG)\n\nfrom swh.loader.tar.tasks import load_tar\nload_tar(origin=origin, visit_date=visit_date,\n last_modified=last_modified)\n```\n\n### Remote\n\nLoad remote tarball is the same sample:\n\n```Python\nurl = 'https://ftp.gnu.org/gnu/8sync/8sync-0.1.0.tar.gz'\norigin = {'url': url, 'type': 'tar'}\nvisit_date = 'Tue, 3 May 2017 17:16:32 +0200'\nlast_modified = '2016-04-22 16:35'\nimport logging\nlogging.basicConfig(level=logging.DEBUG)\n\nfrom swh.loader.tar.tasks import load_tar\nload_tar(origin=origin, visit_date=visit_date,\n last_modified=last_modified)\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://forge.softwareheritage.org/diffusion/DLDTAR", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "swh.loader.tar", "package_url": "https://pypi.org/project/swh.loader.tar/", "platform": "", "project_url": "https://pypi.org/project/swh.loader.tar/", "project_urls": { "Bug Reports": "https://forge.softwareheritage.org/maniphest", "Funding": "https://www.softwareheritage.org/donate", "Homepage": "https://forge.softwareheritage.org/diffusion/DLDTAR", "Source": "https://forge.softwareheritage.org/source/swh-loader-tar" }, "release_url": "https://pypi.org/project/swh.loader.tar/0.0.41/", "requires_dist": [ "arrow", "vcversioner", "requests", "click", "python-dateutil", "swh.core (>=0.0.46)", "swh.model (>=0.0.27)", "swh.scheduler (>=0.0.39)", "swh.storage (>=0.0.83)", "swh.loader.core (>=0.0.35)", "swh.loader.dir (>=0.0.33)", "pytest (<4) ; extra == 'testing'", "swh-scheduler[testing] ; extra == 'testing'", "requests-mock ; extra == 'testing'" ], "requires_python": "", "summary": "Software Heritage Tarball Loader", "version": "0.0.41" }, "last_serial": 5953933, "releases": { "0.0.35": [ { "comment_text": "", "digests": { "md5": "2946b49ce4c25515782a3327cfaa68ec", "sha256": "059ba9919bb92a29daa157ae08ae0f4eccc3d375dce70d362ab114403a6c4f1f" }, "downloads": -1, "filename": "swh.loader.tar-0.0.35-py3-none-any.whl", "has_sig": true, "md5_digest": "2946b49ce4c25515782a3327cfaa68ec", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 22426, "upload_time": "2018-10-18T09:08:26", "url": "https://files.pythonhosted.org/packages/48/5b/9626a7eae4a2e038e2b3ef58225c401b6182b7b4db14f7a99e9c156703c5/swh.loader.tar-0.0.35-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "02cace4ba530038cc895fb86e8600cc8", "sha256": "df2959307ef92e22299fc9f8aa06846ee6298f6f69d01b8200ebf991a16a0219" }, "downloads": -1, "filename": "swh.loader.tar-0.0.35.tar.gz", "has_sig": true, "md5_digest": "02cace4ba530038cc895fb86e8600cc8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8102, "upload_time": "2018-10-18T09:08:33", "url": "https://files.pythonhosted.org/packages/ca/5d/93734561c20af8358920e4cef417a3914fa646ad8c118dc5dc22b9146a73/swh.loader.tar-0.0.35.tar.gz" } ], "0.0.36": [ { "comment_text": "", "digests": { "md5": "e8c2a1751f38672b4de1636225042c57", "sha256": "9dd9e42f26261cc0337c85b19d3f8b81e45e8436eb84e2315bf24bf27eb85c27" }, "downloads": -1, "filename": "swh.loader.tar-0.0.36-py3-none-any.whl", "has_sig": false, "md5_digest": "e8c2a1751f38672b4de1636225042c57", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 27710, "upload_time": "2018-12-05T13:58:48", "url": "https://files.pythonhosted.org/packages/52/0b/820106b9e496815607f186170477ed0e32b89b69402f20dcc8fe99c05b1f/swh.loader.tar-0.0.36-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2895223ff2d2954d2c7d37883cd5a91c", "sha256": "d885c758076014d1afe98c5aa627730410d6ca1423ed943fbc3a14f8909ae3c7" }, "downloads": -1, "filename": "swh.loader.tar-0.0.36.tar.gz", "has_sig": false, "md5_digest": "2895223ff2d2954d2c7d37883cd5a91c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11579, "upload_time": "2018-12-05T13:58:50", "url": "https://files.pythonhosted.org/packages/0e/0e/3210d961d61b51c56f12d86bcb261ee661532da889a146b5ce611e0b7c0a/swh.loader.tar-0.0.36.tar.gz" } ], "0.0.37": [ { "comment_text": "", "digests": { "md5": "91c6475b85edeaf921ba35cdd72aed54", "sha256": "ba1bd6904d01fc41175e6a4ef1ea626008a5d25489eae833db8bd2b7f803e254" }, "downloads": -1, "filename": "swh.loader.tar-0.0.37-py3-none-any.whl", "has_sig": false, "md5_digest": "91c6475b85edeaf921ba35cdd72aed54", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26813, "upload_time": "2018-12-20T10:47:09", "url": "https://files.pythonhosted.org/packages/83/d9/b43e61f6e8616e64eed3f5ccfba0ebc9a4efcec16ac07e773d7848036991/swh.loader.tar-0.0.37-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3e2bbdc4412a030b9e8ea713c51f1837", "sha256": "6f64e9b6d40e53582acec566b8c7998754c8b8f7bce557ce9507ca075fa1a515" }, "downloads": -1, "filename": "swh.loader.tar-0.0.37.tar.gz", "has_sig": false, "md5_digest": "3e2bbdc4412a030b9e8ea713c51f1837", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10742, "upload_time": "2018-12-20T10:47:11", "url": "https://files.pythonhosted.org/packages/97/48/36cef6dd8619851e338ac6f29e1e7c89ae9d35b1ca0aedd6d08fb48de1f7/swh.loader.tar-0.0.37.tar.gz" } ], "0.0.38": [ { "comment_text": "", "digests": { "md5": "fc02d4d9e1d90f4135fa88480ee1306a", "sha256": "3ac56a8f129d2821753c4c58193a400f37e08ac72de038879fc386e00507021c" }, "downloads": -1, "filename": "swh.loader.tar-0.0.38-py3-none-any.whl", "has_sig": false, "md5_digest": "fc02d4d9e1d90f4135fa88480ee1306a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 27043, "upload_time": "2019-02-07T16:38:45", "url": "https://files.pythonhosted.org/packages/38/81/01c5f0a2ee2f1cddcaf3837603f238f5e623b98447e15f1df3a40661c5e2/swh.loader.tar-0.0.38-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "205ccfd816f160165ca31bec005e8744", "sha256": "37f81b5eccbb3bd37fec03247fd0fc20b5e38cbb7f64dce1e5e9cd0694862af0" }, "downloads": -1, "filename": "swh.loader.tar-0.0.38.tar.gz", "has_sig": false, "md5_digest": "205ccfd816f160165ca31bec005e8744", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10802, "upload_time": "2019-02-07T16:38:46", "url": "https://files.pythonhosted.org/packages/bb/52/7253810be40f5c99a9ed96ef6c36ea95a2788a3afb17025d42440dd757c9/swh.loader.tar-0.0.38.tar.gz" } ], "0.0.39": [ { "comment_text": "", "digests": { "md5": "2e525d13835824321fc2cd64c24a723e", "sha256": "17a61263e5036b4ded339b2687bb6a6181ba7176df59b10625fc1259919dad22" }, "downloads": -1, "filename": "swh.loader.tar-0.0.39-py3-none-any.whl", "has_sig": false, "md5_digest": "2e525d13835824321fc2cd64c24a723e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 27058, "upload_time": "2019-02-18T10:49:13", "url": "https://files.pythonhosted.org/packages/6c/24/bd9d659c05224c4c4d5af63839a0caa40c524b826a1baad7d2221d54febb/swh.loader.tar-0.0.39-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "65647f0a18be12f09bf438932799add0", "sha256": "e48b0bc1c17b785c0b7f38352ff542179f45ed95fda45f06a652c42325a87d0b" }, "downloads": -1, "filename": "swh.loader.tar-0.0.39.tar.gz", "has_sig": false, "md5_digest": "65647f0a18be12f09bf438932799add0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10813, "upload_time": "2019-02-18T10:49:15", "url": "https://files.pythonhosted.org/packages/34/23/692b05f81e104eb6f7e2cf192b5bc7e57dbe55c3f21346c2d0628de507a6/swh.loader.tar-0.0.39.tar.gz" } ], "0.0.40": [ { "comment_text": "", "digests": { "md5": "2ae1ce81b0f846deac4682daa919ed7a", "sha256": "daf2110e3c0de1322458629dc1c0ff596801bd6d6bf621ddff3163471959638e" }, "downloads": -1, "filename": "swh.loader.tar-0.0.40-py3-none-any.whl", "has_sig": false, "md5_digest": "2ae1ce81b0f846deac4682daa919ed7a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 27557, "upload_time": "2019-04-19T06:58:20", "url": "https://files.pythonhosted.org/packages/5d/71/f9369c4fa89f9112b714e57ba795f6fd7471f4078f29a5a19779cad97f11/swh.loader.tar-0.0.40-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "dcd2d13a19f1b1b0e149233e80dfdd95", "sha256": "4cb69f58ed55b72c334d79a554e825780fda83708689abaf5943ffc4e67c7b00" }, "downloads": -1, "filename": "swh.loader.tar-0.0.40.tar.gz", "has_sig": false, "md5_digest": "dcd2d13a19f1b1b0e149233e80dfdd95", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11257, "upload_time": "2019-04-19T06:58:22", "url": "https://files.pythonhosted.org/packages/b2/c4/78fa20d727cdf2561efe20bd6f0cb0758ec9cc3644a94083be9328072b9a/swh.loader.tar-0.0.40.tar.gz" } ], "0.0.41": [ { "comment_text": "", "digests": { "md5": "68b1a8cf38b2e91310d581bf7934b538", "sha256": "31013770d2388c14c34c276bb0fa47e5167c3c6c86c3ab9b228e8c0d1e8429d6" }, "downloads": -1, "filename": "swh.loader.tar-0.0.41-py3-none-any.whl", "has_sig": false, "md5_digest": "68b1a8cf38b2e91310d581bf7934b538", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 27866, "upload_time": "2019-10-10T09:48:06", "url": "https://files.pythonhosted.org/packages/68/ba/9d50c4c40f156a66ca21fff516930475e51efe7e646a2e1b51c183ae1fac/swh.loader.tar-0.0.41-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "73acbbf8315a85c853560c08f61e0d7e", "sha256": "52472c3e3b6d3ef6130c28c17bfec2c65d956f1e43a63520cdbf2b8308c4a3f2" }, "downloads": -1, "filename": "swh.loader.tar-0.0.41.tar.gz", "has_sig": false, "md5_digest": "73acbbf8315a85c853560c08f61e0d7e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11382, "upload_time": "2019-10-10T09:48:08", "url": "https://files.pythonhosted.org/packages/21/ae/72e84e81bb76154878153fc88ceb36022c2806a4b50a2234479f5ed15f6d/swh.loader.tar-0.0.41.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "68b1a8cf38b2e91310d581bf7934b538", "sha256": "31013770d2388c14c34c276bb0fa47e5167c3c6c86c3ab9b228e8c0d1e8429d6" }, "downloads": -1, "filename": "swh.loader.tar-0.0.41-py3-none-any.whl", "has_sig": false, "md5_digest": "68b1a8cf38b2e91310d581bf7934b538", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 27866, "upload_time": "2019-10-10T09:48:06", "url": "https://files.pythonhosted.org/packages/68/ba/9d50c4c40f156a66ca21fff516930475e51efe7e646a2e1b51c183ae1fac/swh.loader.tar-0.0.41-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "73acbbf8315a85c853560c08f61e0d7e", "sha256": "52472c3e3b6d3ef6130c28c17bfec2c65d956f1e43a63520cdbf2b8308c4a3f2" }, "downloads": -1, "filename": "swh.loader.tar-0.0.41.tar.gz", "has_sig": false, "md5_digest": "73acbbf8315a85c853560c08f61e0d7e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11382, "upload_time": "2019-10-10T09:48:08", "url": "https://files.pythonhosted.org/packages/21/ae/72e84e81bb76154878153fc88ceb36022c2806a4b50a2234479f5ed15f6d/swh.loader.tar-0.0.41.tar.gz" } ] }