{ "info": { "author": "michaelb", "author_email": "michaelpb@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: End Users/Desktop", "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)", "Natural Language :: English", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: PyPy" ], "description": "alertscraper\n============\n\n.. figure:: https://badge.fury.io/py/alertscraper.png\n :alt: alertscraper badge\n\n alertscraper badge\n\n.. figure:: https://travis-ci.org/michaelpb/alertscraper.png?branch=master\n :alt: travis badge\n\n travis badge\n\nGeneral purpose flexible tool for scraping a given URL for a certain\ntype of items, and then email if new items are added. Useful for\nmonitoring ad or auction websites. Could also be useful for setting up\nemail alerts on your own site.\n\nWARNING\n=======\n\n- Check the Terms of Service of the site before you use this tool! For\n some sites, using this tool may violate their terms of service, and\n should not be used.\n\nLimitations\n===========\n\n- This code ONLY scrapes based on the initial HTTP request. Websites\n that function as single-page apps will not work. This could be\n supported in the future using JSON, or integrating with something\n heavier weight like Selenium.\n\nUsage\n=====\n\nInstallation\n------------\n\nAssuming Python's ``pip`` is installed (for Debian-based systems, this\ncan be installed with ``sudo apt-get install python-pip``), alertscraper\ncan be installed directly from PyPI:\n\n::\n\n pip install alertscraper\n\nPython versions 3.3+ (and 2.6+) are supported and tested against.\n\nQuick start\n-----------\n\n``alertscraper`` is based on URLs, and maintains a history file for each\nURL that you scrape so it knows when something is new.\n\nStart by navigating in your web-browser to the website you want to\nscrape, and then copying and pasting the URL. Then, inspect the page\nsource of the site and see if you can figure out the DOM path to the\nrelevant element. In this case, it was a ``li`` element with the class\nname ``result`` so the combined thing becomes ``li.result``.\n\n::\n\n alertscraper 'https://some-site.org/?query=guitar&maxprice=550' li.result\n\nThis will download the given URL and list the text content of each item\nspecified. This lets you know your query is correct.\n\nNow we want to save this to a database file, that is, say that \"I've\nseen everything currently posted and am only now interested in new\nstuff\".\n\n::\n\n alertscraper 'https://some-site.org/?query=guitar&maxprice=550' li.result --file=guitars.txt\n\nNotice that it prints out again all the links it found. If we were to\nrun the command again, it would not print them out since it will have\nstored them as \"already seen\".\n\nFinally, lets run the command to email us everything that has not yet\nbeen seen.\n\n::\n\n alertscraper 'https://some-site.org/?query=guitar&maxprice=550' li.result --file=guitars.txt --email=myemail@gmail.com\n\nThis only runs once. If you want it to run continually, I'd recommend\nputting it in a cronjob. Eventually I may add a daemon mode, but this is\ngood for now.\n\nHappy scraping!\n\nContributing\n============\n\n- `CONDUCT.md `__\n\nNew features, tests, and bug fixes are welcome!\n\n\n\n\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/michaelpb/alertscraper", "keywords": "alertscraper", "license": "GPL3", "maintainer": "", "maintainer_email": "", "name": "alertscraper", "package_url": "https://pypi.org/project/alertscraper/", "platform": "", "project_url": "https://pypi.org/project/alertscraper/", "project_urls": { "Homepage": "https://github.com/michaelpb/alertscraper" }, "release_url": "https://pypi.org/project/alertscraper/0.1.7/", "requires_dist": null, "requires_python": "", "summary": "Flexible tool for scraping for certain certain DOM elements, and then emailing if new ones are added.", "version": "0.1.7" }, "last_serial": 3313775, "releases": { "0.1.2": [ { "comment_text": "", "digests": { "md5": "8b881c78c246568ea1980243b39d5c0d", "sha256": "13bc40acd1580a0e7a290b1b904d1135ea13f1b1924eb24eb1663f35d2af20db" }, "downloads": -1, "filename": "alertscraper-0.1.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "8b881c78c246568ea1980243b39d5c0d", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 7901, "upload_time": "2017-11-06T21:05:26", "url": "https://files.pythonhosted.org/packages/27/93/e985e1a8df1b885824558005f08b811e2df62c2a1e7ccdda906520d431c9/alertscraper-0.1.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a50cbe370de7ef6e92963c03174f16f1", "sha256": "18ecb1ee93cffe266bf9ecee600fddb68fac9f1c4f793f77cc972a1abc2db531" }, "downloads": -1, "filename": "alertscraper-0.1.2.tar.gz", "has_sig": false, "md5_digest": "a50cbe370de7ef6e92963c03174f16f1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19512, "upload_time": "2017-11-06T21:05:22", "url": "https://files.pythonhosted.org/packages/3a/6c/3c6e55410f1e3380b47ea63821b6aa84252d011dab0e3b4c0c7f5ada98ca/alertscraper-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "c2d08d180d7f6f4440ffc07769122c2f", "sha256": "079532374bea6f021d7191cd7f96d8eae9ff6f49d5134d1068111a636758ba13" }, "downloads": -1, "filename": "alertscraper-0.1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c2d08d180d7f6f4440ffc07769122c2f", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 7903, "upload_time": "2017-11-06T21:07:58", "url": "https://files.pythonhosted.org/packages/4a/a6/d87d5abe3fb9f493ab298ca57b35cd0b630da340b9fb691cc2348c94ff54/alertscraper-0.1.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "342f85013c6a8ca857ea2e47222ceb2d", "sha256": "afc25afacdbef4bf759de11e0f8086f9eac4d2baa2318881afb521839e557cd5" }, "downloads": -1, "filename": "alertscraper-0.1.3.tar.gz", "has_sig": false, "md5_digest": "342f85013c6a8ca857ea2e47222ceb2d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19506, "upload_time": "2017-11-06T21:07:55", "url": "https://files.pythonhosted.org/packages/6f/1c/05abd6fde657c2370ffefa5b7bb792552872fdd03552b0a56b69384c1242/alertscraper-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "9f1e52d37961dfaf6caa6ed5ce84df06", "sha256": "c54e85e482c3aecde8c17326bce3f1f3877254c0b366e45e95c32c4bd32f6b3a" }, "downloads": -1, "filename": "alertscraper-0.1.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9f1e52d37961dfaf6caa6ed5ce84df06", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 7932, "upload_time": "2017-11-06T21:15:30", "url": "https://files.pythonhosted.org/packages/b4/3b/bf156dace92bb8800ad250a3ec26513568a4ff36cf8f0e292d7487530a84/alertscraper-0.1.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "dbb02f4fca6f10373c8f9e7bf8a47f48", "sha256": "8452836bc663d96884090efdfb3875bb46313f25202b5a4a0de1ea4bf9876927" }, "downloads": -1, "filename": "alertscraper-0.1.4.tar.gz", "has_sig": false, "md5_digest": "dbb02f4fca6f10373c8f9e7bf8a47f48", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19537, "upload_time": "2017-11-06T21:15:26", "url": "https://files.pythonhosted.org/packages/e0/17/d28bce66cbe5e4ae8b1104e3fc83f0cf37a11f61e269d8c48b4a4196dcea/alertscraper-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "3c207ffa551340ca6224b51f4d41e22d", "sha256": "22446ea83a004c8f5ae41c908960a4e6a2762112c42613f6480fa38d600d1d9b" }, "downloads": -1, "filename": "alertscraper-0.1.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3c207ffa551340ca6224b51f4d41e22d", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 7981, "upload_time": "2017-11-06T21:49:42", "url": "https://files.pythonhosted.org/packages/cb/02/e24e048a1dea2ea2c00eb5c03be88083150810e40373e99ff58beb4657e9/alertscraper-0.1.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9c6b6732738d2b58d767e850a2797fc5", "sha256": "fe66f1377f1a281078cbcba54ec508c447666b6d0b9f937db10821c511413d78" }, "downloads": -1, "filename": "alertscraper-0.1.5.tar.gz", "has_sig": false, "md5_digest": "9c6b6732738d2b58d767e850a2797fc5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19589, "upload_time": "2017-11-06T21:49:39", "url": "https://files.pythonhosted.org/packages/0d/66/c31d43126b4d8a87d9b639f122ead908f7a3d53ec79a2d44b8bbe0232824/alertscraper-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "0c913949c2477dce51c8229e49eb31f5", "sha256": "c5ed19f6f3499d772dc2c584a3fdcdfe4a05e0c1d30a94a70950b2a121106fb9" }, "downloads": -1, "filename": "alertscraper-0.1.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0c913949c2477dce51c8229e49eb31f5", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 8054, "upload_time": "2017-11-07T20:05:04", "url": "https://files.pythonhosted.org/packages/cf/43/7ba35f90600c1141dbba4ce291a71ae5a1ded60e30afec622caeac6df860/alertscraper-0.1.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "14c110397d39cfc860f69f79fb6f4e23", "sha256": "d347b753f13d66a3badf32b01823885e7f3e0baf206a3d7f3536d9c9be95bf49" }, "downloads": -1, "filename": "alertscraper-0.1.6.tar.gz", "has_sig": false, "md5_digest": "14c110397d39cfc860f69f79fb6f4e23", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19665, "upload_time": "2017-11-07T20:05:00", "url": "https://files.pythonhosted.org/packages/30/04/d8fb07ee4ffebc498c45151861faf55b3a2c3a3cf3cd28d7bea80ab6e7d4/alertscraper-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "72a34198a74e1a6304fb64119b9b0695", "sha256": "b6d1f21d7e1a6ac906f64ce98b8ea729bc822cbc07d76d1cfc8f43d3d6684c6c" }, "downloads": -1, "filename": "alertscraper-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "72a34198a74e1a6304fb64119b9b0695", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 8339, "upload_time": "2017-11-07T20:36:49", "url": "https://files.pythonhosted.org/packages/57/c7/e9f44b84d727cb05e8f8d51d79da6b67810da32fdf8fcf882170e915e647/alertscraper-0.1.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "92c32f4e4a77f4aa57d3c2bd97d06ca6", "sha256": "704855d44a14b9833c275bb503e85d7e3fd2eeaaaa8edab67d5b6c46dea0599a" }, "downloads": -1, "filename": "alertscraper-0.1.7.tar.gz", "has_sig": false, "md5_digest": "92c32f4e4a77f4aa57d3c2bd97d06ca6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20109, "upload_time": "2017-11-07T20:36:46", "url": "https://files.pythonhosted.org/packages/fe/67/1b56a087ae93225db1c849d7ecd8dbdcbe11ac471cd2531fe353217f052e/alertscraper-0.1.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "72a34198a74e1a6304fb64119b9b0695", "sha256": "b6d1f21d7e1a6ac906f64ce98b8ea729bc822cbc07d76d1cfc8f43d3d6684c6c" }, "downloads": -1, "filename": "alertscraper-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "72a34198a74e1a6304fb64119b9b0695", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 8339, "upload_time": "2017-11-07T20:36:49", "url": "https://files.pythonhosted.org/packages/57/c7/e9f44b84d727cb05e8f8d51d79da6b67810da32fdf8fcf882170e915e647/alertscraper-0.1.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "92c32f4e4a77f4aa57d3c2bd97d06ca6", "sha256": "704855d44a14b9833c275bb503e85d7e3fd2eeaaaa8edab67d5b6c46dea0599a" }, "downloads": -1, "filename": "alertscraper-0.1.7.tar.gz", "has_sig": false, "md5_digest": "92c32f4e4a77f4aa57d3c2bd97d06ca6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20109, "upload_time": "2017-11-07T20:36:46", "url": "https://files.pythonhosted.org/packages/fe/67/1b56a087ae93225db1c849d7ecd8dbdcbe11ac471cd2531fe353217f052e/alertscraper-0.1.7.tar.gz" } ] }