{ "info": { "author": "Erick Torres and Erik Quintanilla", "author_email": "ericktorresdeveloper@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 3.5" ], "description": "# NewsDataScrapper\n\nPython package that helps you easily retrieve complete web articles.\n\n[![License: MIT](https://img.shields.io/github/license/erikqu/NewsDataScraper)](https://opensource.org/licenses/MIT)\n[![pypi: newsdatascraper](https://img.shields.io/pypi/pyversions/newsdatascraper)](https://pypi.org/project/newsdatascraper/)\n\n## Requirements\n- Python 3.5+\n- [Newspaper3k](https://newspaper.readthedocs.io/en/latest/)\n- API Key from [NewsApi](https://newsapi.org) or API Key from [GNews](https://gnews.io)\n\n## Installation\n```bash\npip3 install newsdatascraper\n```\n\n## Usage\n```python\nfrom newsdatascraper import Scraper\n#To first get a single article on a topic\nnew_scraper = Scraper('mock-api-key')\narticles = new_scraper.fetch_all_articles(query='two sigma', pageSize = 10)\n\n\"\"\"\nWe support two APIs: NewsApi and GNewsApi\nTo control the API being used change the argument of mode to either 'NEWSPAPER' or 'GNEWS'\n\"\"\"\n\nnew_scraper = Scraper('mock-api-key', mode = 'GNEWS')\narticles = new_scraper.fetch_all_articles(query='two sigma', pageSize = 10, \n dateFrom = \"2019-08-04\", dateTo = \"2019-08-10\")\n\n#To access individual articles and their properties\nfirst_article = articles.articles[0]\nprint(first_article.content)\n#We also provide helper functions to serialize the data\narticles.toCsv('test.csv')\narticles.toPickle('test.pickle')\narticles.toJson()\n```\n\n## Important Note\n\nPlease look at rate limits in the APIs to determine your prefered usage\n\n### Working on the Project\nRun format\n```bash\nblack .\n```\nRun Linter\n```bash\npylama -o setup.cfg .\n```\nRun tests\n```bash\npytest\n```\nRun tests + code coverage\n```bash\nsh ./scripts/generate_coverage.sh\n```", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/erikqu/NewsDataScraper", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "newsdatascraper", "package_url": "https://pypi.org/project/newsdatascraper/", "platform": "", "project_url": "https://pypi.org/project/newsdatascraper/", "project_urls": { "Homepage": "https://github.com/erikqu/NewsDataScraper" }, "release_url": "https://pypi.org/project/newsdatascraper/0.0.9.4/", "requires_dist": null, "requires_python": "", "summary": "Easily query articles", "version": "0.0.9.4", "yanked": false, "yanked_reason": null }, "last_serial": 6110740, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "6407893690492c116e43880a197668eb", "sha256": "aaa593e8579b84f80d83bcafd51c20131a9f7c7e07326faef18b3f1e93b8bc0b" }, "downloads": -1, "filename": "newsdatascraper-0.0.1.tar.gz", "has_sig": false, "md5_digest": "6407893690492c116e43880a197668eb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3000, "upload_time": "2019-08-12T18:55:57", "upload_time_iso_8601": "2019-08-12T18:55:57.418503Z", "url": "https://files.pythonhosted.org/packages/c2/81/722033e89dd457bc1ddd88095bd7c250978dddc1bfe4e0e2013b609421ef/newsdatascraper-0.0.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "484b23ed340b10aa39bb0fa4e50c0392", "sha256": "aeeaf5f85e6da83ac657c434477f5c2d2ba1dcb803e537640b93d84392fb67c1" }, "downloads": -1, "filename": "newsdatascraper-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "484b23ed340b10aa39bb0fa4e50c0392", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4609, "upload_time": "2019-08-12T19:14:40", "upload_time_iso_8601": "2019-08-12T19:14:40.788883Z", "url": "https://files.pythonhosted.org/packages/0e/c8/2f09644ca0b643f135f09ea83a4a0940ecb2db303145fb9e10e09b3b28ec/newsdatascraper-0.0.2-py3-none-any.whl", "yanked": false, "yanked_reason": null } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "03ce35c789651e3f5606963124a94853", "sha256": "54fd9e1c07686597707e8f9a5e5592fbd7418662792cd84457678073284bafc7" }, "downloads": -1, "filename": "newsdatascraper-0.0.3.tar.gz", "has_sig": false, "md5_digest": "03ce35c789651e3f5606963124a94853", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3401, "upload_time": "2019-08-13T18:20:24", "upload_time_iso_8601": "2019-08-13T18:20:24.969175Z", "url": "https://files.pythonhosted.org/packages/49/cf/2db4b05d2ece02c79d3d2ec847608e03ad8769e9dc0bea6fbc74d8cda40d/newsdatascraper-0.0.3.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "f2bd50ea4a010f91e28bfe8b01a92cbb", "sha256": "3f00b43840854d792944a43914402b3c68505601dfa3a712aa78889c65df3a86" }, "downloads": -1, "filename": "newsdatascraper-0.0.4.tar.gz", "has_sig": false, "md5_digest": "f2bd50ea4a010f91e28bfe8b01a92cbb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3365, "upload_time": "2019-08-17T04:35:52", "upload_time_iso_8601": "2019-08-17T04:35:52.818430Z", "url": "https://files.pythonhosted.org/packages/8b/45/af5c32fd7f1752570ecb857ce84b8f2f39ea3ccaf0da3e9f959ba8c6882f/newsdatascraper-0.0.4.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "60c8dd41bac71ac651e00820f59a5540", "sha256": "92d128a780ba506029c399415bdc152610260cf6abdd05ea3ca2872a2815495d" }, "downloads": -1, "filename": "newsdatascraper-0.0.5.tar.gz", "has_sig": false, "md5_digest": "60c8dd41bac71ac651e00820f59a5540", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3366, "upload_time": "2019-08-17T04:42:13", "upload_time_iso_8601": "2019-08-17T04:42:13.141635Z", "url": "https://files.pythonhosted.org/packages/e9/cc/546e36a9253da7f10d9698d738ed882b9b7af70424a65a48c8d82d3910de/newsdatascraper-0.0.5.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "13ff496fb28abe4e33d6afc944da1b52", "sha256": "b92aec41d00a4fdfc6a84f1b27ad3130f4b0ddcbf850a410db3b088caaf5b9e8" }, "downloads": -1, "filename": "newsdatascraper-0.0.6.tar.gz", "has_sig": false, "md5_digest": "13ff496fb28abe4e33d6afc944da1b52", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4159, "upload_time": "2019-08-17T04:47:21", "upload_time_iso_8601": "2019-08-17T04:47:21.334869Z", "url": "https://files.pythonhosted.org/packages/6b/7a/b7a18e01f88bfef479d1cb37276eb8d2030bfb068ce6cb67f89ecb7e007d/newsdatascraper-0.0.6.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "12bcfd34ff01edf9196f7a4e0093f6c8", "sha256": "c0f061b076f850f910e4686caf0d88fae452243f56cff629575f210d6f92480c" }, "downloads": -1, "filename": "newsdatascraper-0.0.7.tar.gz", "has_sig": false, "md5_digest": "12bcfd34ff01edf9196f7a4e0093f6c8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4149, "upload_time": "2019-08-17T04:52:11", "upload_time_iso_8601": "2019-08-17T04:52:11.968811Z", "url": "https://files.pythonhosted.org/packages/2c/16/cff950dcc7f7d9ee48bd093aa71d6e2b54f275daf89681d0e4474dd2b277/newsdatascraper-0.0.7.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "b9b0b33108c6a67052a45b5d1e9bc06c", "sha256": "733fbb2a313fd99d5482d1a88202b48477a7b89dfdc95c2a0e056b3b0c488166" }, "downloads": -1, "filename": "newsdatascraper-0.0.8.tar.gz", "has_sig": false, "md5_digest": "b9b0b33108c6a67052a45b5d1e9bc06c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4187, "upload_time": "2019-10-30T01:23:16", "upload_time_iso_8601": "2019-10-30T01:23:16.422212Z", "url": "https://files.pythonhosted.org/packages/ef/e8/4365e689a2270fab209c8c73eb6ec1b7bf7b08ec98a4f5552bd43b56d328/newsdatascraper-0.0.8.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "1332407c1489d664406425dc6b40dce3", "sha256": "6eacf06408139eeb9156c0150a568fcda8196d5f1e070f8381109d36b031ae6a" }, "downloads": -1, "filename": "newsdatascraper-0.0.9.tar.gz", "has_sig": false, "md5_digest": "1332407c1489d664406425dc6b40dce3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4237, "upload_time": "2019-10-30T01:38:55", "upload_time_iso_8601": "2019-10-30T01:38:55.367531Z", "url": "https://files.pythonhosted.org/packages/bc/14/fb51b54ef74125ef861853929fef81a3fdbef6ed4a75d0ddf8bed56829e1/newsdatascraper-0.0.9.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.9.1": [ { "comment_text": "", "digests": { "md5": "38c90a3227b6c2fce945536167bc690c", "sha256": "2b3342b48a06fab2f5177f23aa10010e44e4559212cbe87ef6d564f37334363e" }, "downloads": -1, "filename": "newsdatascraper-0.0.9.1.tar.gz", "has_sig": false, "md5_digest": "38c90a3227b6c2fce945536167bc690c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4236, "upload_time": "2019-10-30T01:51:21", "upload_time_iso_8601": "2019-10-30T01:51:21.745033Z", "url": "https://files.pythonhosted.org/packages/fc/0f/1bb8fb8a362d8cafac52fa92f38fe1b4a93c07de19cc80d1e8a991936c1c/newsdatascraper-0.0.9.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.9.2": [ { "comment_text": "", "digests": { "md5": "01119b58ac2990462fd813ce3736e600", "sha256": "02d7cc51a87d5651eafc8de97e941319d75466f16a29e2092559731747c1995c" }, "downloads": -1, "filename": "newsdatascraper-0.0.9.2.tar.gz", "has_sig": false, "md5_digest": "01119b58ac2990462fd813ce3736e600", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4264, "upload_time": "2019-11-07T14:06:39", "upload_time_iso_8601": "2019-11-07T14:06:39.533466Z", "url": "https://files.pythonhosted.org/packages/cd/80/3637b8b54b8e9f75563cdaa42360f8441a645702a38ab882515db7e97a9c/newsdatascraper-0.0.9.2.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.9.3": [ { "comment_text": "", "digests": { "md5": "c7b8afb04d8fb578c39720074ca799e6", "sha256": "aa2abee7177726479344393d19e083be776993930ea7b144834fe77d4cd73f6b" }, "downloads": -1, "filename": "newsdatascraper-0.0.9.3.tar.gz", "has_sig": false, "md5_digest": "c7b8afb04d8fb578c39720074ca799e6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4732, "upload_time": "2019-11-10T05:00:35", "upload_time_iso_8601": "2019-11-10T05:00:35.676552Z", "url": "https://files.pythonhosted.org/packages/18/19/5a3eed32e47e3e5236f759e46ef4207d5c2d3376d15094ce72d94770f2b7/newsdatascraper-0.0.9.3.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.9.4": [ { "comment_text": "", "digests": { "md5": "6cdeac2a97dd7c5d2eb36bfbd229b2df", "sha256": "93b7103982d2c36382c5e262b99e3a9d97a8ce86c83f12c457acc2122e613b4e" }, "downloads": -1, "filename": "newsdatascraper-0.0.9.4.tar.gz", "has_sig": false, "md5_digest": "6cdeac2a97dd7c5d2eb36bfbd229b2df", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4726, "upload_time": "2019-11-10T05:46:01", "upload_time_iso_8601": "2019-11-10T05:46:01.320620Z", "url": "https://files.pythonhosted.org/packages/01/dc/3b3c06086de0da9d11539c8dc3792db13f5ec8bfdb6f76edda0a87385b39/newsdatascraper-0.0.9.4.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "6cdeac2a97dd7c5d2eb36bfbd229b2df", "sha256": "93b7103982d2c36382c5e262b99e3a9d97a8ce86c83f12c457acc2122e613b4e" }, "downloads": -1, "filename": "newsdatascraper-0.0.9.4.tar.gz", "has_sig": false, "md5_digest": "6cdeac2a97dd7c5d2eb36bfbd229b2df", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4726, "upload_time": "2019-11-10T05:46:01", "upload_time_iso_8601": "2019-11-10T05:46:01.320620Z", "url": "https://files.pythonhosted.org/packages/01/dc/3b3c06086de0da9d11539c8dc3792db13f5ec8bfdb6f76edda0a87385b39/newsdatascraper-0.0.9.4.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }