{ "info": { "author": "Filip Pobo\u0159il", "author_email": "tsuki@fpob.eu", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3 :: Only", "Topic :: Internet :: WWW/HTTP" ], "description": "Logicoma\n========\n\nLogicoma is Python package for creating simple web crawlers with as little code\nas necessary. It aims to make simple web crawlers as quick as possible.\n\nIf some complex crawling is needed, this package may not be the right choice,\n`Scrapy `_ or something else could be better choice.\n\n\nExample\n-------\n\nDownload images or even entire galleries from `Imgur `_:\n\n.. code:: python\n\n import click\n import logicoma\n\n @logicoma.crawler()\n def crawler(links):\n return links\n\n @crawler.handler(r'//imgur\\.com/')\n def imgur_gallery(client, url):\n response, page = client.get_page(url)\n images = page.find(class_='post-images').find_all('a', class_='zoom')\n for img in images:\n img_url = logicoma.url_join(url, img.get('href'))\n yield logicoma.Download(img_url)\n\n @crawler.handler(r'//i\\.imgur\\.com/')\n def imgur_image(url):\n yield logicoma.Download(url)\n\n @click.command()\n @click.option('-c', '--count', type=int, default=1, help='Number of threads')\n @click.argument('links', nargs=-1)\n def cli(**kwargs):\n crawler(**kwargs)\n\n if __name__ == '__main__':\n cli()\n\nExmples with comments are in the `examples` directory.\n\n\nInstallation\n------------\n\nInstall it using pip ::\n\n pip3 install logicoma\n\nor clone repository ::\n\n git clone https://github.com/fpob/logicoma\n cd logicoma\n\nand install Python package including dependencies ::\n\n python3 setup.py install\n\nRecommended packages\n^^^^^^^^^^^^^^^^^^^^\n\n* `click `_ - package for creating command line interfaces\n* `browsercookie `_ - loads cookies used by web browser", "description_content_type": "", "docs_url": null, "download_url": "https://github.com/fpob/logicoma/archive/v0.4.tar.gz", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/fpob/logicoma", "keywords": "Web Crawling", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "logicoma", "package_url": "https://pypi.org/project/logicoma/", "platform": "", "project_url": "https://pypi.org/project/logicoma/", "project_urls": { "Download": "https://github.com/fpob/logicoma/archive/v0.4.tar.gz", "Homepage": "https://github.com/fpob/logicoma" }, "release_url": "https://pypi.org/project/logicoma/0.4/", "requires_dist": null, "requires_python": "", "summary": "Package for creating simple web crawlers.", "version": "0.4" }, "last_serial": 4101330, "releases": { "0.3": [ { "comment_text": "", "digests": { "md5": "23ec88a4f6d8c3b785ed3f982e13d3a5", "sha256": "32e4307dede401d1913e140c9662d62107ead1f3d47df3da6391a6760397e59d" }, "downloads": -1, "filename": "logicoma-0.3.tar.gz", "has_sig": false, "md5_digest": "23ec88a4f6d8c3b785ed3f982e13d3a5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7965, "upload_time": "2018-04-18T18:10:22", "url": "https://files.pythonhosted.org/packages/eb/99/30c836609b13b13a67188ced6688926a8fc7d5b89acb28c0d6826436cbdb/logicoma-0.3.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "70b6a53409d74e80e0fcf91e5eae7506", "sha256": "ef6380fd1fb9f34268beab5c85da4920216cf6a98097c4f5cc81beb9f9ee5bbc" }, "downloads": -1, "filename": "logicoma-0.3.1.tar.gz", "has_sig": false, "md5_digest": "70b6a53409d74e80e0fcf91e5eae7506", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7975, "upload_time": "2018-04-18T18:11:54", "url": "https://files.pythonhosted.org/packages/65/3c/96fc6609ca230c1f42b48eb952ae0b714a791806295d26f6c248ebd235aa/logicoma-0.3.1.tar.gz" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "3101b6362a40766960e12101f4def11b", "sha256": "5d68b0e4faebc40cc22a4d904fc03076eb9506e77ceb061eadd62eefca483165" }, "downloads": -1, "filename": "logicoma-0.4.tar.gz", "has_sig": false, "md5_digest": "3101b6362a40766960e12101f4def11b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7918, "upload_time": "2018-07-25T17:00:37", "url": "https://files.pythonhosted.org/packages/ec/a6/8fe047952f77183af770c4544e0b0c57f8cae8aa1feaf23a81a719009288/logicoma-0.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3101b6362a40766960e12101f4def11b", "sha256": "5d68b0e4faebc40cc22a4d904fc03076eb9506e77ceb061eadd62eefca483165" }, "downloads": -1, "filename": "logicoma-0.4.tar.gz", "has_sig": false, "md5_digest": "3101b6362a40766960e12101f4def11b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7918, "upload_time": "2018-07-25T17:00:37", "url": "https://files.pythonhosted.org/packages/ec/a6/8fe047952f77183af770c4544e0b0c57f8cae8aa1feaf23a81a719009288/logicoma-0.4.tar.gz" } ] }