{ "info": { "author": "ChenXinqun", "author_email": "chenxinqun163@163.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Framework :: AsyncIO", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Internet :: WWW/HTTP", "Topic :: Software Development :: Libraries :: Application Frameworks", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "CrawlFrame: A Web Crawling framework\n==========================\n[![PyPI](https://img.shields.io/pypi/v/crawlframe.svg)](https://pypi.org/project/crawlframe/)\n\n##### Behold, the power of CrawlFrame:\n\n``` \ncrawlf createapp -url http:///\ncrawlf createpro \ncrawlf start app:\ncrawlf start project\ncrawlf stop app:\ncrawlf stop project\ncrawlf stop \n```\n\n[CrawlFrame](https://github.com/chenxinqun/crawlframe) officially supports python3.5+ .\n\nInstallation\n------------\n\nTo install CrawlFrame, simply use [pip](https://pypi.org/project/crawlframe/):\n\n``` \n$ pip install crawlframe\n```\n\n\nDocumentation\n-------------\nIn the construction...\n\n\nChanged\n-------------\n[CHANGES.md](https://github.com/chenxinqun/crawlframe/blob/prepack/CHANGES.md)\n\ncrawlframe-1.0.3\n==========================\n\u4e0b\u8f7d\u5668\u5e76\u53d1\u65b9\u5f0f\u53d8\u66f4\n\n\n\ncrawlframe-1.0.2\n==========================\n```\nCRAWLFRAME_SURVIVE_SWITCH = True\nCRAWLFRAME_SURVIVE_MAX = int\nreload spider\n```\nCRAWLFRAME_SURVIVE_SWITCH = True\nCRAWLFRAME_SURVIVE_MAX = int\n\u8bbe\u7f6e\u4e24\u4e2a\u53c2\u6570\n\u53d1\u51fa\u4e00\u5b9a\u8bf7\u6c42\u6570\u4e4b\u540e, \u81ea\u52a8\u91cd\u542f\u722c\u866b. \u907f\u514d\u5185\u5b58\u6ea2\u51fa.\n\u4fee\u590dmiddle\u4e0d\u80fd\u52a0\u8f7d\u4ee5\u53calogger\u4e0d\u80fd\u521b\u5efa\u7684bug.\nBaseLogger\u6539\u4e3a\u5355\u4f8b\u6a21\u5f0f.\n\n\ncrawlframe-1.0.1\n==========================\n``` \ncrawlf createapp -url http:///\ncrawlf createpro \ncrawlf start app:\ncrawlf start project\ncrawlf stop app:\ncrawlf stop project\ncrawlf stop \n``` \n\n\u5b9e\u9645\u4e0a\u4ee5\u4e0a\u547d\u4ee4\u53ea\u5b9e\u73b0\u4e86start \u4e0e stop. \n\u4f46\u662f\u5df2\u7ecf\u80fd\u591f\u5728\u9879\u76ee\u4e2d\u4f7f\u7528\u4e86.\n\u4e3a\u4e86\u65b9\u4fbf\u81ea\u5df1\u5b89\u88c5\u4f7f\u7528, \u5c31\u5148\u4e0a\u4f20\u5230 pypi \u4e0e github \u4e86.\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/chenxinqun/crawlframe", "keywords": "", "license": "Apache 2.0", "maintainer": "ChenXinqun ", "maintainer_email": "chenxinqun163@163.com", "name": "crawlframe", "package_url": "https://pypi.org/project/crawlframe/", "platform": "", "project_url": "https://pypi.org/project/crawlframe/", "project_urls": { "Homepage": "https://github.com/chenxinqun/crawlframe" }, "release_url": "https://pypi.org/project/crawlframe/1.0.3/", "requires_dist": [ "bs4", "pytz", "lxml", "redis", "psutil", "psycopg2", "requests" ], "requires_python": ">=3.5.3", "summary": "A Web Crawling and Web Scraping framework (crawlframe) for Python.", "version": "1.0.3" }, "last_serial": 5512368, "releases": { "1.0.3": [ { "comment_text": "", "digests": { "md5": "d1a2806498636c3864729ed681365b84", "sha256": "3f3c5feb3b20cb97ed81d4b36eba48a4300fb0b2dc30c4523e370e89e57c2c18" }, "downloads": -1, "filename": "crawlframe-1.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d1a2806498636c3864729ed681365b84", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5.3", "size": 46147, "upload_time": "2019-04-09T14:10:56", "url": "https://files.pythonhosted.org/packages/a3/e5/c15adce5b73fb9c4b38bd09c07e30e4b1e200946d619acc65bbbe760f2b4/crawlframe-1.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8fa65e08bf86c6689373534ad4fb2184", "sha256": "619b1431411c9ea76d1b3787ee03fdfd606802b8918f8e374b02378646baa453" }, "downloads": -1, "filename": "crawlframe-1.0.3.tar.gz", "has_sig": false, "md5_digest": "8fa65e08bf86c6689373534ad4fb2184", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5.3", "size": 27384, "upload_time": "2019-04-09T14:10:57", "url": "https://files.pythonhosted.org/packages/24/31/886f70a99116dcd9d3376583286574222d97c4e431791c2f826d8fed7e3a/crawlframe-1.0.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d1a2806498636c3864729ed681365b84", "sha256": "3f3c5feb3b20cb97ed81d4b36eba48a4300fb0b2dc30c4523e370e89e57c2c18" }, "downloads": -1, "filename": "crawlframe-1.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d1a2806498636c3864729ed681365b84", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5.3", "size": 46147, "upload_time": "2019-04-09T14:10:56", "url": "https://files.pythonhosted.org/packages/a3/e5/c15adce5b73fb9c4b38bd09c07e30e4b1e200946d619acc65bbbe760f2b4/crawlframe-1.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8fa65e08bf86c6689373534ad4fb2184", "sha256": "619b1431411c9ea76d1b3787ee03fdfd606802b8918f8e374b02378646baa453" }, "downloads": -1, "filename": "crawlframe-1.0.3.tar.gz", "has_sig": false, "md5_digest": "8fa65e08bf86c6689373534ad4fb2184", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5.3", "size": 27384, "upload_time": "2019-04-09T14:10:57", "url": "https://files.pythonhosted.org/packages/24/31/886f70a99116dcd9d3376583286574222d97c4e431791c2f826d8fed7e3a/crawlframe-1.0.3.tar.gz" } ] }