{ "info": { "author": "Marat (@cmu.edu)", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Science/Research", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Scientific/Engineering" ], "description": "# Python interface for code hosting platforms API\n\nIt is intended to facilitate research of Open Source projects.\nAt this point, it is basically functional but is missing:\n\n- tests\n- documentation\n- good architecture\n\nFeel free to contribute any of those.\n\n### Installation\n\n```bash\npip install --user --upgrade strudel.scraper\n``` \n\n\n### Usage\n\n```python\nimport stscraper as scraper\nimport pandas as pd\n\ngh_api = scraper.GitHubAPI()\n# so far only GiHub, Bitbucket and Gitlab are supported\n# bb_api = scraper.BitbucketAPI()\n# gl_api = scraper.GitLabAPI()\n\n# repo_issues is a generator that can be used\n# to instantiate a pandas dataframe\nissues = pd.DataFrame(gh_api.repo_issues('cmustrudel/strudel.scraper'))\n```\n\n\n\n### Settings\n\nGitHub and GitLab APIs limit request rate for unauthenticated requests\n(although GitLab limit is much more generous).\nThere are several ways to set your API keys, listed below in order of priority.\n\n**Important note:** API objects are reused in subsequent calls.\nThe same keys used to instantiate the first API object will be used by\nALL other instances.\n\n#### Class instantiation:\n\n```python\nimport stscraper\n\ngh_api = stscraper.GitHubAPI(tokens=\"comman-separated list of tokens\")\n```\n\n#### At runtime:\n\n```python\nimport stscraper\nimport stutils\n\n# IMPORTANT: do this before creation of the first API object!\nstutils.CONFIG['GITHUB_API_TOKENS'] = 'comma-separated list of tokens'\nstutils.CONFIG['GITLAB_API_TOKENS'] = 'comma-separated list of tokens'\n\n# any api instance created after this, will use the provided tokens\ngh_api = stscraper.GitHubAPI()\n```\n\n#### settings file:\n\n```\nproject root\n \\\n |- my_module\n | \\- my_file.py\n |- settings.py\n```\n\n```python\n# settings.py\n\nGITHUB_API_TOKENS = 'comma-separated list of tokens'\nGITLAB_API_TOKENS = 'comma-separated list of tokens'\n```\n\n```python\n# my_file.py\nimport stscraper\n\n# keys from settings.py will be reused automatically\ngh_api = stscraper.GitHubAPI()\n```\n\n#### Environment variable:\n\n\n```bash\n# somewhere in ~/.bashrc\nexport GITHUB_API_TOKENS='comma-separated list of tokens'\nexport GITLAB_API_TOKENS='comma-separated list of tokens'\n```\n\n```python\n# somewhere in the code\nimport stscraper\n\n# keys from environment variables will be reused automatically\ngh_api = stscraper.GitHubAPI()\n```\n\n\n#### Hub config:\n\nIf you have [hub](https://github.com/github/hub) installed and everything else\nfails, its configuration will be reused for GitHub API.\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/cmustrudel/strudel.scraper", "keywords": "", "license": "GPL v3", "maintainer": "", "maintainer_email": "", "name": "strudel.scraper", "package_url": "https://pypi.org/project/strudel.scraper/", "platform": "Linux", "project_url": "https://pypi.org/project/strudel.scraper/", "project_urls": { "Homepage": "https://github.com/cmustrudel/strudel.scraper" }, "release_url": "https://pypi.org/project/strudel.scraper/0.4.0/", "requires_dist": [ "requests", "strudel.utils" ], "requires_python": ">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "summary": "Python interfaces to Github, Bitbucket and Gitlab APIs", "version": "0.4.0" }, "last_serial": 5497977, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "d6b258aa50e6ba2fa352cf5a809dc6ba", "sha256": "8020d5534093ce9b92ab4b6c5df319d1158a35c7b545efc58e27c6ce0a4f342a" }, "downloads": -1, "filename": "strudel.scraper-0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d6b258aa50e6ba2fa352cf5a809dc6ba", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 27176, "upload_time": "2018-08-20T19:07:19", "url": "https://files.pythonhosted.org/packages/7d/4f/8dfbba52576001ade063ab749d2ab303d955f4e80671aa2e1bac2c515e0e/strudel.scraper-0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3d264a554dc3bfe674331d4eddb8dce8", "sha256": "40773d6e2b00812c0747a1d643325fb7bd76a350c02e52e29f10b90cce3418bb" }, "downloads": -1, "filename": "strudel.scraper-0.1.tar.gz", "has_sig": false, "md5_digest": "3d264a554dc3bfe674331d4eddb8dce8", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 11970, "upload_time": "2018-08-20T19:07:20", "url": "https://files.pythonhosted.org/packages/8a/c2/395d7a789d4afe62875e3c30f4e597f31ad43ae498644bc08489f2bb99ba/strudel.scraper-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "3bd24e51422c362bcadfe37a9d1880a3", "sha256": "63d1f1018c7b9ff0b3590e7c493e237db0539bb1d9007932b3971a8e66ab410f" }, "downloads": -1, "filename": "strudel.scraper-0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3bd24e51422c362bcadfe37a9d1880a3", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 33038, "upload_time": "2018-09-10T03:12:44", "url": "https://files.pythonhosted.org/packages/6d/08/e9e38ea701a6172b2346ab0cb7a0049d94986f0fee55d4de9ed9e6f883de/strudel.scraper-0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "42015bf18c8aa1285e3a56b71110fd81", "sha256": "2549882db413370eebdd2108a339be62d26ca77fcc92118042738a6d427be905" }, "downloads": -1, "filename": "strudel.scraper-0.2.tar.gz", "has_sig": false, "md5_digest": "42015bf18c8aa1285e3a56b71110fd81", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 19214, "upload_time": "2018-09-10T03:12:46", "url": "https://files.pythonhosted.org/packages/07/cd/d27ef657c2097873f915300747c0a580ff55a96404acb6b6f56bfe5ef171/strudel.scraper-0.2.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "b0d9450ee7df0b82c127955accb826d1", "sha256": "646f7ca5fc36e781b9be8b2ad09e63efbfad60a245e172c9985c3f69073a804a" }, "downloads": -1, "filename": "strudel.scraper-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b0d9450ee7df0b82c127955accb826d1", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 33811, "upload_time": "2018-10-22T21:01:43", "url": "https://files.pythonhosted.org/packages/bd/0c/d8efa350e246bde5f52febde831cf01bc93cad684e45f62b8e4a057f32cd/strudel.scraper-0.2.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "64620188e6bb266725859e1f4c083846", "sha256": "7dc50d9367992c2290185c2cefabaf123b37a3f2628dc40349dbb0b8b30eeea4" }, "downloads": -1, "filename": "strudel.scraper-0.2.1.tar.gz", "has_sig": false, "md5_digest": "64620188e6bb266725859e1f4c083846", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 19910, "upload_time": "2018-10-22T21:01:45", "url": "https://files.pythonhosted.org/packages/ba/b4/a41b24f9bf1c17c1624ec8626f77e3e6648e6d4b8e9baa0d368c9da7a6f2/strudel.scraper-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "72a9d5704955a324f8958a42dd9f8d46", "sha256": "57bbb475de176b165212850d2513d0493dfbcac2bb03c981b2c8f45ad296e4ae" }, "downloads": -1, "filename": "strudel.scraper-0.2.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "72a9d5704955a324f8958a42dd9f8d46", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 33885, "upload_time": "2018-10-31T02:17:16", "url": "https://files.pythonhosted.org/packages/8f/af/2b7e512b56c9c149a4b1abb81fc09f8a965b396675030fb5c7ab627b1b04/strudel.scraper-0.2.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "565ccd8c05ad6eab3ef30b4bcf1ad7cb", "sha256": "e7c191ea5337efe445cecec06e2e97ab32db3df3a478b5ff94578596e8b4e328" }, "downloads": -1, "filename": "strudel.scraper-0.2.2.tar.gz", "has_sig": false, "md5_digest": "565ccd8c05ad6eab3ef30b4bcf1ad7cb", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 19986, "upload_time": "2018-10-31T02:17:18", "url": "https://files.pythonhosted.org/packages/b2/05/82bbf0656d95c92631633c52cbf1501659c6ea7bb0eff7ccc54df48eb1ff/strudel.scraper-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "208c0ede497a0638750ae02efdf0b21d", "sha256": "dbac10741a18e9d5fd8c8bf3a3562b4c6fe7eb78f5c48ad8f030ca5ddebdfb69" }, "downloads": -1, "filename": "strudel.scraper-0.2.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "208c0ede497a0638750ae02efdf0b21d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 33992, "upload_time": "2018-11-12T17:26:40", "url": "https://files.pythonhosted.org/packages/80/b0/69324680cfe1949e32cff9dc3394f524be7757ef8c181397ad640fa75867/strudel.scraper-0.2.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1de8e7562b75d4305249dcec0e181e51", "sha256": "0415e556f824bb2cdf5fd78f3373085e155d2de2b17a14d2d88e111d04c9e6c5" }, "downloads": -1, "filename": "strudel.scraper-0.2.3.tar.gz", "has_sig": false, "md5_digest": "1de8e7562b75d4305249dcec0e181e51", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 20111, "upload_time": "2018-11-12T17:26:41", "url": "https://files.pythonhosted.org/packages/fa/58/fac8535479114008bae3bdfb8cffdca8c3a6db5b2e552ea7cb37b31b950e/strudel.scraper-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "eb9c63123b825c039a92a1f5a2752934", "sha256": "477a286941ad14c7d3e4aaa8f31fe634d6a79e81289ea4fac06a4c83b36f4be6" }, "downloads": -1, "filename": "strudel.scraper-0.2.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "eb9c63123b825c039a92a1f5a2752934", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 35117, "upload_time": "2018-11-13T20:22:38", "url": "https://files.pythonhosted.org/packages/39/5d/7f9e144ba3df7978aa99ab407ac969a4b335bf6353275462aba5eec55bb8/strudel.scraper-0.2.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "dae6b7795114f9c453365614d3e970b5", "sha256": "860d12b8b4ec7d3747c3c252e6f0ee897563368d4640a729907c942a1a88f112" }, "downloads": -1, "filename": "strudel.scraper-0.2.4.tar.gz", "has_sig": false, "md5_digest": "dae6b7795114f9c453365614d3e970b5", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 20956, "upload_time": "2018-11-13T20:22:39", "url": "https://files.pythonhosted.org/packages/39/37/d94d7ad5dfba194dadcb8d5c6175e53862aa77e652276905edde1ccba929/strudel.scraper-0.2.4.tar.gz" } ], "0.2.5": [ { "comment_text": "", "digests": { "md5": "fa42b259e1c3838bbe5820358c1ff51d", "sha256": "e6b8520a568b63b1d5a2a55e4b919c3d8b8586821c39a242aca4649e455a6872" }, "downloads": -1, "filename": "strudel.scraper-0.2.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "fa42b259e1c3838bbe5820358c1ff51d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 35740, "upload_time": "2018-11-13T21:43:46", "url": "https://files.pythonhosted.org/packages/b7/f6/1873fead009f94e09d467002ece872c861ea1e6d34165b30fca3285c9e08/strudel.scraper-0.2.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9b55edfe4268cac1337d4a769f034ef9", "sha256": "4eda607526e6814347e977dd71ac9dc98876f7014d48b0fe3c98267eda8779fe" }, "downloads": -1, "filename": "strudel.scraper-0.2.5.tar.gz", "has_sig": false, "md5_digest": "9b55edfe4268cac1337d4a769f034ef9", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 21367, "upload_time": "2018-11-13T21:43:47", "url": "https://files.pythonhosted.org/packages/c0/1c/8b79e03fffed767d84e7ef10a74975fcc7e3c34970b8b659204a5b39c3ec/strudel.scraper-0.2.5.tar.gz" } ], "0.2.6": [ { "comment_text": "", "digests": { "md5": "cbc3893ba0971e56c383ae601d0b89b8", "sha256": "65d57aad97f745f3fb7f7093bb2ead0c0064bffdad3c0b1570101579205061be" }, "downloads": -1, "filename": "strudel.scraper-0.2.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "cbc3893ba0971e56c383ae601d0b89b8", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 35602, "upload_time": "2018-11-17T03:44:35", "url": "https://files.pythonhosted.org/packages/c4/5f/a09af32b95af70bab9f57581ef002a96b6741698d9fe9a260bf7cd71db09/strudel.scraper-0.2.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "337633986f0d35d938554042973cb706", "sha256": "e91e6c8d858b8a8c6afc27beaafa7111f589e1d2fb9d452394b2c78418c0c9ec" }, "downloads": -1, "filename": "strudel.scraper-0.2.6.tar.gz", "has_sig": false, "md5_digest": "337633986f0d35d938554042973cb706", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, <4", "size": 21449, "upload_time": "2018-11-17T03:44:37", "url": "https://files.pythonhosted.org/packages/7e/3f/a3a1fa9dad637fc6a22343f2951297b17a30b8900e3307dedc3e0a3ef242/strudel.scraper-0.2.6.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "615ce13ffd842ba019b7c60ed44c6a48", "sha256": "a78b37b7a8859c9da5daaaedbd3554b1972c707113a602a509da147962b69abb" }, "downloads": -1, "filename": "strudel.scraper-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "615ce13ffd842ba019b7c60ed44c6a48", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 37064, "upload_time": "2019-02-26T20:27:45", "url": "https://files.pythonhosted.org/packages/de/a9/ab1cb008f7f6ebbf117c35ff5b437918079d11fe34addcef7850bbd023ba/strudel.scraper-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1303d7ba2a998ca821ad0a1d09f47f2b", "sha256": "88ee0a972491fa5c58f9bbf090c472796aeb69a99e9ff9807985fd75d896bb17" }, "downloads": -1, "filename": "strudel.scraper-0.3.0.tar.gz", "has_sig": false, "md5_digest": "1303d7ba2a998ca821ad0a1d09f47f2b", "packagetype": "sdist", "python_version": "source", "requires_python": ">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 20848, "upload_time": "2019-02-26T20:27:46", "url": "https://files.pythonhosted.org/packages/70/32/fb0d6bfd962e401bd812877536f0037c338d6f1c7ed84df54e7bcbd49f50/strudel.scraper-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "0557291653e10d5eafe75aa2f02d92cb", "sha256": "cbb71fe1706321f0e6b3f01424651cb51ecb435c15ff757b3790ecf512ffe3e2" }, "downloads": -1, "filename": "strudel.scraper-0.4.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0557291653e10d5eafe75aa2f02d92cb", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 35072, "upload_time": "2019-07-07T20:43:01", "url": "https://files.pythonhosted.org/packages/44/f6/710e907c15d644ae3643bb9f71394048c51fad3523ab39505ea50c1c732c/strudel.scraper-0.4.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9913c9c509ce64ab073eb77075a2e777", "sha256": "4f8440b450d10a5e8bbaeb32f4dc65f083b8f6f840f2d7c5ff48853cb4ab7877" }, "downloads": -1, "filename": "strudel.scraper-0.4.0.tar.gz", "has_sig": false, "md5_digest": "9913c9c509ce64ab073eb77075a2e777", "packagetype": "sdist", "python_version": "source", "requires_python": ">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 30903, "upload_time": "2019-07-07T20:43:03", "url": "https://files.pythonhosted.org/packages/79/62/2dd3cf4afe896d42ba545cd8152a4e3a63e4943711d19f7e35f2e4e0952e/strudel.scraper-0.4.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "0557291653e10d5eafe75aa2f02d92cb", "sha256": "cbb71fe1706321f0e6b3f01424651cb51ecb435c15ff757b3790ecf512ffe3e2" }, "downloads": -1, "filename": "strudel.scraper-0.4.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0557291653e10d5eafe75aa2f02d92cb", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 35072, "upload_time": "2019-07-07T20:43:01", "url": "https://files.pythonhosted.org/packages/44/f6/710e907c15d644ae3643bb9f71394048c51fad3523ab39505ea50c1c732c/strudel.scraper-0.4.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9913c9c509ce64ab073eb77075a2e777", "sha256": "4f8440b450d10a5e8bbaeb32f4dc65f083b8f6f840f2d7c5ff48853cb4ab7877" }, "downloads": -1, "filename": "strudel.scraper-0.4.0.tar.gz", "has_sig": false, "md5_digest": "9913c9c509ce64ab073eb77075a2e777", "packagetype": "sdist", "python_version": "source", "requires_python": ">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4", "size": 30903, "upload_time": "2019-07-07T20:43:03", "url": "https://files.pythonhosted.org/packages/79/62/2dd3cf4afe896d42ba545cd8152a4e3a63e4943711d19f7e35f2e4e0952e/strudel.scraper-0.4.0.tar.gz" } ] }