{ "info": { "author": "t4skforce", "author_email": "7422037+t4skforce@users.noreply.github.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Environment :: Console", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8" ], "description": "# fdroid-dl\nIs a python based f-droid mirror generation and update utility. Point at one or more existing f-droid repositories and the utility will download the metadata (pictures, descriptions,..) for you and place it in your local system. Simply run ```fdroid-dl && fdroid update``` and you are set.\n\n## Motivation\nThe idea is to have an no internet local copy of one or more repositories, without the need to compile the thousands of apps on your own build server but rather download them like the android client does. So this tool came into existence to simply download a whole repository and import the apps into your own locally installed one. At the time of writing a full offline copy including assets is ~7.5GB of the official repository of f-droid.org.\n\n# Installation\nfdroid-dl is available via pip, simply run ```pip install fdroid-dl``` and you can use ```fdroid-dl``` on your command line. [pypi.org - fdroid-dl](https://pypi.org/project/fdroid-dl/)\n\n# Documentation\nCan be found at [fdroid-dl.readthedocs.io](https://fdroid-dl.readthedocs.io/en/latest/)\n\n# Command Line Options\n```\nUsage: fdroid-dl [OPTIONS] COMMAND [ARGS]...\n\n Is a python based f-droid mirror generation and update utility. Point at\n one or more existing f-droid repositories and the utility will download\n the metadata (pictures, descriptions,..) for you and place it in your\n local system.\n\n Simply run \"fdroid-dl update && fdroid update\" in your folder with repo\n and you are set.\n\nOptions:\n -d, --debug enable debug level logging\n -c, --config FILE location of your fdroid-dl.json configuration file\n [default: fdroid-dl.json]\n -r, --repo DIRECTORY location of your fdroid repository to store the\n apk files [default: ./repo]\n -m, --metadata DIRECTORY location of your fdroid metadata to store the\n asset files [default: ./metadata]\n --cache DIRECTORY location for fdroid-dl to store cached data\n [default: ./.cache]\n --help Show this message and exit.\n\nCommands:\n update starts updating process\n```\n```\nUsage: fdroid-dl update [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n --index / --no-index download repository index files [default: True]\n --metadata / --no-metadata download metadata assset files [default: True]\n --apk / --no-apk download apk files [default: True]\n --apk-versions INTEGER how many versions of apk to download [default:\n 1]\n --src / --no-src download src files [default: True]\n --threads INTEGER configure number of parallel threads used for\n download [default: 10]\n --head-timeout INTEGER maximum time in seconds a HEAD request is\n allowed to take [default: 10]\n --index-timeout INTEGER maximum time in seconds index file download is\n allowed to take [default: 60]\n --download-timeout INTEGER maximum time in seconds file download is allowed\n to take [default: 60]\n --help Show this message and exit.\n```\n\n# Configuration File\n\n# TODO\n- [x] Create backend to crawl existing repos\n- [x] Fetch info directly index.jar and index-v1.jar\n- [x] Compatibility with old and new repo styles\n- [x] Download multi threaded\n- [x] Verify apk checksum\n- [x] Local cache for index files\n- [ ] Source code download not implemented yet\n- [ ] Metadata update to do delta not full download all the time\n- [ ] Cleanup strategy for old apk files (maybe ```fdroid update``` does this already?)\n- [x] Create a CLI [python click](http://click.pocoo.org/5/)\n- [x] pip package [packaging.python.org](https://packaging.python.org/tutorials/packaging-projects/)\n- [x] CI builds for pip package\n- [ ] Documentation ;-)\n- [ ] Writing tests [pytest](https://docs.pytest.org/en/latest/)\n\n# CHANGELOG\n- WIP: Documentation added\n- WIP: Test added\n - requests-mock ?\n\n## Ideas\n- requests-cache ?\n\n# Development\n## Requirements\n* python 2.7.* or 3.5.*\n* pip 3.x\n* virtualenv 3.x\n\n## install locally\n```\n# git clone https://github.com/t4skforce/fdroid-dl.git\n# cd fdroid-dl\n# virtualenv .env\n# source .env/bin/activate\n# python setup.py install\n# fdroid-dl --help\n# python3 -m fdroid_dl --help\n```\n\n### References\nWhile this project was developed the following references where used\n\n#### F-Droid\n* Setup an F-Droid App Repo [f-droid.org](https://f-droid.org/en/docs/Setup_an_F-Droid_App_Repo/)\n* Build Metadata Reference [f-droid.org](https://f-droid.org/en/docs/Build_Metadata_Reference/)\n* All About Descriptions, Graphics, and Screenshots [f-droid.org](https://f-droid.org/en/docs/All_About_Descriptions_Graphics_and_Screenshots/)\n* How to Add a Repo to F-Droid [f-droid.org](https://f-droid.org/en/tutorials/add-repo/)\n* How to Send and Receive Apps Offline [f-droid.org](https://f-droid.org/en/tutorials/swap/)\n\n#### Python\n* Python Documentation [python.org](https://docs.python.org/3/)\n* PyYAML Documentation [pyyaml.org](https://pyyaml.org/wiki/PyYAMLDocumentation)\n* Requests: HTTP for Humans [python-requests.org](http://docs.python-requests.org/en/master/)\n * Suppress InsecureRequestWarning: Unverified HTTPS request is being made in Python2.6 [stackoverflow.com](https://stackoverflow.com/questions/27981545/suppress-insecurerequestwarning-unverified-https-request-is-being-made-in-pytho)\n * How to download large file in python with requests.py? [stackoverflow.com](https://stackoverflow.com/questions/16694907/how-to-download-large-file-in-python-with-requests-py)\n* Asynchronous Python HTTP Requests for Humans using Futures [requests-futures](https://github.com/ross/requests-futures)\n* Testing Your Code [python-guide.org](https://docs.python-guide.org/writing/tests/)\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/t4skforce/fdroid-dl", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "fdroid-dl", "package_url": "https://pypi.org/project/fdroid-dl/", "platform": "", "project_url": "https://pypi.org/project/fdroid-dl/", "project_urls": { "Homepage": "https://github.com/t4skforce/fdroid-dl" }, "release_url": "https://pypi.org/project/fdroid-dl/0.0.5/", "requires_dist": [ "requests-futures (>=0.9.7)", "PyYAML (>=3.13)", "click (>=6.7)" ], "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "summary": "fdroid-dl is a f-droid (offline) mirror generation and update utility", "version": "0.0.5" }, "last_serial": 4889186, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "04e1519287429a614085ff14573eac9e", "sha256": "a3827dcc1c5356a0ef789cc5725bd59378c4d8382cf0088014dc4b880520e66a" }, "downloads": -1, "filename": "fdroid_dl-0.0.1-py3.6.egg", "has_sig": false, "md5_digest": "04e1519287429a614085ff14573eac9e", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": ">=3.5", "size": 66114, "upload_time": "2018-07-20T17:49:57", "url": "https://files.pythonhosted.org/packages/13/0c/99b444ea24ae6fcf8df9610623fde719e0ab5907c07a830db17126af5193/fdroid_dl-0.0.1-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "9f6834f21fd985775976aa4aff482f29", "sha256": "f5c75bd574dbd66acc75271ce3a0387a61d83334ec29717b4a5c9dbbd2e80aeb" }, "downloads": -1, "filename": "fdroid_dl-0.0.1-py3.7.egg", "has_sig": false, "md5_digest": "9f6834f21fd985775976aa4aff482f29", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": ">=3.5", "size": 69092, "upload_time": "2018-12-19T13:08:33", "url": "https://files.pythonhosted.org/packages/53/1f/0adc7be88e701a1febae9350a56a9b6faf45df28df608a5cb11714f689ef/fdroid_dl-0.0.1-py3.7.egg" }, { "comment_text": "", "digests": { "md5": "53defcef0537ad8f1d5bbfbf2c051591", "sha256": "adde59a57baf5cd1bba9b19001f07cb6c794f17dfb665713e0b3930b5e8d0415" }, "downloads": -1, "filename": "fdroid_dl-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "53defcef0537ad8f1d5bbfbf2c051591", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 28252, "upload_time": "2018-07-20T17:49:56", "url": "https://files.pythonhosted.org/packages/f3/79/9c2eb74dec635222b62d7d6e65455703c5baf5727192bdc8a63248466867/fdroid_dl-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a92b1a1d37ad2a1c895db33276daa372", "sha256": "34a3b34ec29caa0bbf82c3c1ecabd951e40694273644f44a01feb297a4882b4f" }, "downloads": -1, "filename": "fdroid-dl-0.0.1.tar.gz", "has_sig": false, "md5_digest": "a92b1a1d37ad2a1c895db33276daa372", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 20290, "upload_time": "2018-07-20T17:49:58", "url": "https://files.pythonhosted.org/packages/d6/1d/b39dc333100b7a294090d5475885c45edcdef976a6e019ffaf55fd6448ed/fdroid-dl-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "f1a5062b48985ede730c6e4054064e31", "sha256": "43fa0841970ca8e22204bdd6852bc25350bfa045f3eb367d984713cd7f1d83e3" }, "downloads": -1, "filename": "fdroid_dl-0.0.2-py3.7.egg", "has_sig": false, "md5_digest": "f1a5062b48985ede730c6e4054064e31", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": ">=3.5", "size": 69413, "upload_time": "2018-12-19T13:08:36", "url": "https://files.pythonhosted.org/packages/0b/16/f599bf562331e04ab7ff3732454d29d18849c8b6630f74b8d271067894a2/fdroid_dl-0.0.2-py3.7.egg" }, { "comment_text": "", "digests": { "md5": "3db2bf9cbd2349992804444c6d46a103", "sha256": "49199539ec38a79bed03ed5cb5e7108be5932a75d3331e4a6834c37a7be75c34" }, "downloads": -1, "filename": "fdroid_dl-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "3db2bf9cbd2349992804444c6d46a103", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 30289, "upload_time": "2018-12-19T13:08:31", "url": "https://files.pythonhosted.org/packages/d8/3a/3b2d0860087525528e757c539a77cfca6fe9898dbba1b3d8cde7d5a29289/fdroid_dl-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "77cd1fe78207fcb825a0312df350b42f", "sha256": "531978fefff31810e27583f36084623aa1ba9ca27dbfa636c8cd79e3bfe2dfc5" }, "downloads": -1, "filename": "fdroid-dl-0.0.2.tar.gz", "has_sig": false, "md5_digest": "77cd1fe78207fcb825a0312df350b42f", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 19914, "upload_time": "2018-12-19T13:08:37", "url": "https://files.pythonhosted.org/packages/8f/d2/c0277b619701128d84713fb722c6ffb95afefd288e252990346fbd197729/fdroid-dl-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "1196064ab0d704179d43fe791670fc50", "sha256": "de010fe25ea9ce3d682a9fc13c3ced4256518589ce72e84b9bf6c2bafd13c8a2" }, "downloads": -1, "filename": "fdroid_dl-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "1196064ab0d704179d43fe791670fc50", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 28124, "upload_time": "2019-02-24T13:02:53", "url": "https://files.pythonhosted.org/packages/1a/8f/da516109f7d63b3c126c1d19917015d6dca628541d78c352e3728283e646/fdroid_dl-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "66e6f6787c3c895d7e4b2e63f7849877", "sha256": "91096bebc7fe297d5b979cc18cbcd6942661dbf58083bc87c41b5e959b99724b" }, "downloads": -1, "filename": "fdroid-dl-0.0.3.tar.gz", "has_sig": false, "md5_digest": "66e6f6787c3c895d7e4b2e63f7849877", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 19783, "upload_time": "2019-02-24T13:02:54", "url": "https://files.pythonhosted.org/packages/66/50/b97b3d4554a5d415d8178c9e7305e397a0868986ddb12ae5b1852384e24f/fdroid-dl-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "277341bf9d0f927cbb4d3bd2f5b8fb10", "sha256": "09436996919aba7276eb84c030ebc3e213a0548a051d724de310a050e20db9cd" }, "downloads": -1, "filename": "fdroid_dl-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "277341bf9d0f927cbb4d3bd2f5b8fb10", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 28283, "upload_time": "2019-03-02T12:22:59", "url": "https://files.pythonhosted.org/packages/13/4c/9ff7ef8fb82324468304565642ac1faff78f40dc2b4f2cb7437995db3592/fdroid_dl-0.0.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "148a430990249d03121859438febe6c3", "sha256": "0003ab3159668339659ddf26d16051a7f595c15701ea789cd73c3b99492e089e" }, "downloads": -1, "filename": "fdroid-dl-0.0.4.tar.gz", "has_sig": false, "md5_digest": "148a430990249d03121859438febe6c3", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 19935, "upload_time": "2019-03-02T12:23:01", "url": "https://files.pythonhosted.org/packages/57/19/e9429163310b5988b208611bf6627b6350ca5075f9274a8b98e616c0b03e/fdroid-dl-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "a6d5bfa8472cdb2aefe7d3515569b692", "sha256": "e7b244300bb80215ca1907c4cfbef3a6725fb36646b5ae48613438762aca747b" }, "downloads": -1, "filename": "fdroid_dl-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "a6d5bfa8472cdb2aefe7d3515569b692", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 29387, "upload_time": "2019-03-02T21:35:15", "url": "https://files.pythonhosted.org/packages/9b/80/1e12a89feb794dd6a899e903baaa05ab8b0e33a61c4aba0506746c14b5a5/fdroid_dl-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "641b91db4eff3a4b10027ccda860a541", "sha256": "224a85c52b77355cc52873eaf36e469e6a74f7a7413ffb6a9e9b0ad84556158a" }, "downloads": -1, "filename": "fdroid-dl-0.0.5.tar.gz", "has_sig": false, "md5_digest": "641b91db4eff3a4b10027ccda860a541", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 21353, "upload_time": "2019-03-02T21:35:16", "url": "https://files.pythonhosted.org/packages/c2/0c/b82b258a0a9bcd46d97423b33c6bd7962e1b8be2a2b176958856608240c8/fdroid-dl-0.0.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a6d5bfa8472cdb2aefe7d3515569b692", "sha256": "e7b244300bb80215ca1907c4cfbef3a6725fb36646b5ae48613438762aca747b" }, "downloads": -1, "filename": "fdroid_dl-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "a6d5bfa8472cdb2aefe7d3515569b692", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 29387, "upload_time": "2019-03-02T21:35:15", "url": "https://files.pythonhosted.org/packages/9b/80/1e12a89feb794dd6a899e903baaa05ab8b0e33a61c4aba0506746c14b5a5/fdroid_dl-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "641b91db4eff3a4b10027ccda860a541", "sha256": "224a85c52b77355cc52873eaf36e469e6a74f7a7413ffb6a9e9b0ad84556158a" }, "downloads": -1, "filename": "fdroid-dl-0.0.5.tar.gz", "has_sig": false, "md5_digest": "641b91db4eff3a4b10027ccda860a541", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 21353, "upload_time": "2019-03-02T21:35:16", "url": "https://files.pythonhosted.org/packages/c2/0c/b82b258a0a9bcd46d97423b33c6bd7962e1b8be2a2b176958856608240c8/fdroid-dl-0.0.5.tar.gz" } ] }