{ "info": { "author": "Grant Williams", "author_email": "grant.williams2986@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3 :: Only" ], "description": "===============================\nWG Gesucht Crawler\n===============================\n\n.. image:: https://img.shields.io/travis/grantwilliams/wg-gesucht-crawler-cli.svg\n :target: https://travis-ci.org/grantwilliams/wg-gesucht-crawler-cli\n\n.. image:: https://img.shields.io/pypi/v/wg-gesucht-crawler-cli.svg\n :target: https://pypi.python.org/pypi/wg-gesucht-crawler-cli\n\n.. image:: https://readthedocs.org/projects/wg-gesucht-crawler-cli/badge/?version=latest\n :target: https://wg-gesucht-crawler-cli.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\nPython web crawler / scraper for WG-Gesucht. Crawls the WG-Gesucht site for new apartment listings and send a message to the poster, based off your saved filters and saved text template.\n\nInstallation\n------------\n::\n\n $ pip install wg-gesucht-crawler-cli\n\nOr, if you have virtualenvwrapper installed::\n\n $ mkvirtualenv wg-gesucht-crawler-cli\n $ pip install wg-gesucht-crawler-cli\n\nUse\n---\nCan be run directly from the command line with::\n\n $ wg-gesucht-crawler-cli --help\n\nOr if you want to use it in your own project:\n\n.. code-block:: python\n\n from wg_gesucht.crawler import WgGesuchtCrawler\n\nJust make sure to save at least one search filter as well as a template text on your wg-gesucht account.\n\n* Free software: MIT license\n* Documentation: https://wg-gesucht-crawler-cli.readthedocs.org.\n\nFeatures\n--------\n\n* Searches https://wg-gesucht.de for new WG ads based off your saved filters\n* Sends your saved template message and applies to all matching listings\n* Reruns every ~5 minutes\n* Run on a RPi or free EC2 micro instance 24/7 to always be one of the first to apply for new listings\n\n\n\n**Getting Caught with reCAPTCHA**\n\nI've made the crawler sleep for 5-8 seconds between each request to try and avoid their reCAPTCHA, but if the crawler does get caught, you can sign into your wg-gesucht account manually through the browser and solve the reCAPTCHA, then start the crawler again.\nIf it continues to happen, you can also increase the sleep time in the :code:`get_page()` function in :code:`wg_gesucht.py`\n\n\n\n\nHistory\n-------\n\nPre-release\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/grantwilliams/wg-gesucht-crawler-cli", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "wg-gesucht-crawler-cli", "package_url": "https://pypi.org/project/wg-gesucht-crawler-cli/", "platform": "", "project_url": "https://pypi.org/project/wg-gesucht-crawler-cli/", "project_urls": { "Homepage": "https://github.com/grantwilliams/wg-gesucht-crawler-cli" }, "release_url": "https://pypi.org/project/wg-gesucht-crawler-cli/0.1.9/", "requires_dist": [ "beautifulsoup4 (==4.6.0)", "certifi (==2018.4.16)", "chardet (==3.0.4)", "click (==6.7)", "idna (==2.7)", "requests (==2.21.0)", "urllib3 (==1.23)" ], "requires_python": "", "summary": "Python web crawler / scraper for WG-Gesucht. Crawls the WG-Gesucht site for new apartment listings and send a message to the poster, based off your saved filters and saved text template.", "version": "0.1.9" }, "last_serial": 4936144, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "9d9b6f9b749fd9473991e23a9c9076a5", "sha256": "7fcac73cac2d56323c1067a622f126d1b8e8699f414312442df2d927d990c9c0" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9d9b6f9b749fd9473991e23a9c9076a5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10148, "upload_time": "2018-07-07T00:47:22", "url": "https://files.pythonhosted.org/packages/85/04/ef04b46c58bdb2b699d76af9879bd54c576becdd9636c99f086447cb3a80/wg_gesucht_crawler_cli-0.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e616653e85edf670ffaaddbc06dc2362", "sha256": "5010f524517e1033e84ad3c54148ec96dffb252eae4183c55c1ebc6cd16e91ab" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.0.tar.gz", "has_sig": false, "md5_digest": "e616653e85edf670ffaaddbc06dc2362", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32316, "upload_time": "2018-07-07T00:47:24", "url": "https://files.pythonhosted.org/packages/00/28/6434f419bd3d3cc5e75f64bb4921019c3ae46b578dc3b2687db856a7b056/wg-gesucht-crawler-cli-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "81e21a252d6b2e1e84cf10be8df5a8c8", "sha256": "b314174f9491c5521f3734f89097968d7d1b6ba7760f6445f25c6353095bc12a" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "81e21a252d6b2e1e84cf10be8df5a8c8", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10178, "upload_time": "2018-07-07T18:34:46", "url": "https://files.pythonhosted.org/packages/7f/c8/2fe4ea76cdb5ba51ed7e0768c8524043dfb68ad2c7127428d8ca0906a9c8/wg_gesucht_crawler_cli-0.1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "38fb8fdcd2e2d6c4d1f461bbacadce7f", "sha256": "c13082dc97cce48dc80294e409747542993611357396256c93264adeabfb427e" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.1.tar.gz", "has_sig": false, "md5_digest": "38fb8fdcd2e2d6c4d1f461bbacadce7f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32308, "upload_time": "2018-07-07T18:34:48", "url": "https://files.pythonhosted.org/packages/e5/44/6c4bc5e10a85e83400dcc6cf4654e8310920c1e9c2902fd6da6c30c61333/wg-gesucht-crawler-cli-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "cf0578f15a96156994b64b04d981d629", "sha256": "cc21fe07a307eff122f96b16fe565c53b7e76ea98df9e9b364246e68c72a0ed2" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "cf0578f15a96156994b64b04d981d629", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12215, "upload_time": "2018-07-07T23:54:48", "url": "https://files.pythonhosted.org/packages/37/9b/e5d8efe131748f7bdecb5dcb07efc6af053c19ef91645d96eb2ad7a8b7f8/wg_gesucht_crawler_cli-0.1.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e189711eb07273a5ef14a2bec9291a11", "sha256": "54ed6dd8fbea28fa54e38bf0e5f71eb61c0e3503ba348a6a2a8fd76823db26a7" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.2.tar.gz", "has_sig": false, "md5_digest": "e189711eb07273a5ef14a2bec9291a11", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32277, "upload_time": "2018-07-07T23:54:50", "url": "https://files.pythonhosted.org/packages/d0/d6/7b53e5b0275f49fd9e0d6a70ddf011f6b611df829cc31e6f04c8d970f6ec/wg-gesucht-crawler-cli-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "0cc6876889576f17c2a01fe515923fcb", "sha256": "c26b06207e15ee9470476a57944061db09357711524aaa786ee2d92c70447029" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0cc6876889576f17c2a01fe515923fcb", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10129, "upload_time": "2018-07-30T23:49:25", "url": "https://files.pythonhosted.org/packages/c4/31/1a8612f54971e8177c13d6c3b854add06b12c928010d2af36aeedaae9fa5/wg_gesucht_crawler_cli-0.1.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3d58a88612d33b6f547fe894d09b957b", "sha256": "6c45880678db3ed5d6695112e46dc3118396f2e0b35a05496be06b4d09587c5d" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.3.tar.gz", "has_sig": false, "md5_digest": "3d58a88612d33b6f547fe894d09b957b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32270, "upload_time": "2018-07-30T23:49:28", "url": "https://files.pythonhosted.org/packages/2b/0a/347911bda0d0560a70d305938dc0c30f0048f428fa8e1a3aa3af8ac21260/wg-gesucht-crawler-cli-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "e556cef6bf0c3e69519c5d47cab40a25", "sha256": "a2b8fd51370530ecbd427a9eb8a007d521edd1968e56d9130ef39ecef82c2b3c" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e556cef6bf0c3e69519c5d47cab40a25", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10146, "upload_time": "2018-08-22T19:18:18", "url": "https://files.pythonhosted.org/packages/ed/3e/c50603fd2adc498f9122a2be9eba3eefb0fc4421d71621bd2aaf5b3ad89b/wg_gesucht_crawler_cli-0.1.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ad3518ea72f471ae173b26ffc01db999", "sha256": "a27fc0ca176de59ee755add02b9badf7dd049c2c6ad1ce66462ce8dc916aca1f" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.4.tar.gz", "has_sig": false, "md5_digest": "ad3518ea72f471ae173b26ffc01db999", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32284, "upload_time": "2018-08-22T19:18:20", "url": "https://files.pythonhosted.org/packages/f9/b5/deda683389894d602ffbf068cc601e01eed67009d7c9205c30974554bd89/wg-gesucht-crawler-cli-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "83f3cfa1bc2c997f107b1c89de0cf4b2", "sha256": "189f0b4b5ef16e9e7efb410e97312dad0727f2e3e3e9c092d4aecfd70b0a7716" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "83f3cfa1bc2c997f107b1c89de0cf4b2", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11770, "upload_time": "2019-01-27T17:12:51", "url": "https://files.pythonhosted.org/packages/87/de/f44489e4c732fc2eae374a0813726c67346e2b83a6d330d9b8eb96c3ad89/wg_gesucht_crawler_cli-0.1.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c1a89b2ac6e924e848944630d9489346", "sha256": "e2b21bfac9a23c0ddb1b5688e07b138dbaa487af27f739883c7368cc4e84fe48" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.5.tar.gz", "has_sig": false, "md5_digest": "c1a89b2ac6e924e848944630d9489346", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33458, "upload_time": "2019-01-27T17:12:53", "url": "https://files.pythonhosted.org/packages/4d/cd/03b0ffe0d2f198ab28129333db5acfc6c0ae2a54e6de74076110c68648d8/wg-gesucht-crawler-cli-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "bcdd6ef10ab5b324531222e36c146d78", "sha256": "ad68ec6fe04447872f53a338bfef87b2b5bf958fa9fb89b968096afcae8396ef" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "bcdd6ef10ab5b324531222e36c146d78", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11935, "upload_time": "2019-03-06T19:25:52", "url": "https://files.pythonhosted.org/packages/4a/93/046333f020fbf26310114ccef17c4dda85baea445907c605c92c97ef2155/wg_gesucht_crawler_cli-0.1.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0b499ab4ce87662f4cbcf0211c690641", "sha256": "a279a9cb47cf091e6b49b17a17e09c4c12b45847a2e6cca72c6514e00e8806fe" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.6.tar.gz", "has_sig": false, "md5_digest": "0b499ab4ce87662f4cbcf0211c690641", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33588, "upload_time": "2019-03-06T19:25:55", "url": "https://files.pythonhosted.org/packages/ee/d4/3e6251aacbd241c0036aaa50fb9c842ede4cb0bd557c9d6bd681ca3ef519/wg-gesucht-crawler-cli-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "4698972da5d9ddee7fc1c31ed4ef7a12", "sha256": "f657cc9ebee4b45440c9b63fb827f9d4adbb261d5e3fe4770307b4655e4e4b9f" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "4698972da5d9ddee7fc1c31ed4ef7a12", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11983, "upload_time": "2019-03-06T21:25:33", "url": "https://files.pythonhosted.org/packages/14/49/51c2de892356617cfe05ec52cedd805ef5fb8ef817249073adb67cf0acbb/wg_gesucht_crawler_cli-0.1.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0c0d1768cf9d833243bd40c1e8a297d5", "sha256": "924dbb4f5f188d8300e29a11c84435a67eb7f02b6568a33ac8fc06b7d174a9e3" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.7.tar.gz", "has_sig": false, "md5_digest": "0c0d1768cf9d833243bd40c1e8a297d5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33642, "upload_time": "2019-03-06T21:25:37", "url": "https://files.pythonhosted.org/packages/8b/39/1db2416f849899374e98319f0ee83d47d43007d4ccdc1418521cf1811872/wg-gesucht-crawler-cli-0.1.7.tar.gz" } ], "0.1.8": [ { "comment_text": "", "digests": { "md5": "74a17ba76becab3f02c6dfc70e034caf", "sha256": "f4ec5df520426791000d94213b6f98a606208f4a37fcefcb69b4b61bd4f4ce20" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "74a17ba76becab3f02c6dfc70e034caf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12921, "upload_time": "2019-03-12T23:32:06", "url": "https://files.pythonhosted.org/packages/24/46/035803546d9c8027945f20aa1b49e0c0f7756587022a81f78e389049db9f/wg_gesucht_crawler_cli-0.1.8-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b3388e6c3e314153a59dbaa3cc29edce", "sha256": "1336f4ebe1ae9ef8634a3c19c6659b339bdfcfe6f06f6f9b2ba084418d7f4781" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.8.tar.gz", "has_sig": false, "md5_digest": "b3388e6c3e314153a59dbaa3cc29edce", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33703, "upload_time": "2019-03-12T23:32:12", "url": "https://files.pythonhosted.org/packages/05/e7/a0547846c82a87713ccf43e4608ab5d87a8836bc38d483f8e7da18a6ed15/wg-gesucht-crawler-cli-0.1.8.tar.gz" } ], "0.1.9": [ { "comment_text": "", "digests": { "md5": "87287935badac2c42d5ee60711925880", "sha256": "b24bda1dc884130d49f3260e0e76ce65c1c731809e2d5ba8e7d66efe6b381fc9" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "87287935badac2c42d5ee60711925880", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12139, "upload_time": "2019-03-13T19:30:41", "url": "https://files.pythonhosted.org/packages/8c/70/876e125e3f8849afd7c0e3b7d53e7a423c578d5cf6e7879ca2dfc57d73ab/wg_gesucht_crawler_cli-0.1.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "09d2ce048f1e298f550a76b608fd4ba6", "sha256": "5b0c53b148e294306a147abc8c2879d2b6de3b0da4d8efe6fea8ddd6ee3e6c4e" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.9.tar.gz", "has_sig": false, "md5_digest": "09d2ce048f1e298f550a76b608fd4ba6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33787, "upload_time": "2019-03-13T19:30:47", "url": "https://files.pythonhosted.org/packages/f3/1c/ada0e0cba91676ffe4bb70941f098a1598e4620b9728a35a2bf553d9c6a8/wg-gesucht-crawler-cli-0.1.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "87287935badac2c42d5ee60711925880", "sha256": "b24bda1dc884130d49f3260e0e76ce65c1c731809e2d5ba8e7d66efe6b381fc9" }, "downloads": -1, "filename": "wg_gesucht_crawler_cli-0.1.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "87287935badac2c42d5ee60711925880", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12139, "upload_time": "2019-03-13T19:30:41", "url": "https://files.pythonhosted.org/packages/8c/70/876e125e3f8849afd7c0e3b7d53e7a423c578d5cf6e7879ca2dfc57d73ab/wg_gesucht_crawler_cli-0.1.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "09d2ce048f1e298f550a76b608fd4ba6", "sha256": "5b0c53b148e294306a147abc8c2879d2b6de3b0da4d8efe6fea8ddd6ee3e6c4e" }, "downloads": -1, "filename": "wg-gesucht-crawler-cli-0.1.9.tar.gz", "has_sig": false, "md5_digest": "09d2ce048f1e298f550a76b608fd4ba6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33787, "upload_time": "2019-03-13T19:30:47", "url": "https://files.pythonhosted.org/packages/f3/1c/ada0e0cba91676ffe4bb70941f098a1598e4620b9728a35a2bf553d9c6a8/wg-gesucht-crawler-cli-0.1.9.tar.gz" } ] }