{ "info": { "author": "Scrapedia", "author_email": "Scrapedia@outlook.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Environment :: Plugins", "Framework :: Scrapy", "Intended Audience :: Developers", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Internet :: WWW/HTTP", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "================\nScrapy-Pipelines\n================\n\nOverview\n========\n\n.. image:: https://bestpractices.coreinfrastructure.org/projects/2828/badge\n :alt: CII Best Practices\n :target: https://bestpractices.coreinfrastructure.org/projects/2828\n\n.. image:: https://mperlet.github.io/pybadge/badges/9.43.svg\n :alt: pylint Score\n\n.. image:: https://img.shields.io/travis/scrapedia/scrapy-pipelines/master.svg\n :target: http://travis-ci.org/scrapedia/scrapy-pipelines\n :alt: Travis branch\n\n.. image:: https://codecov.io/gh/scrapedia/scrapy-pipelines/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/scrapedia/scrapy-pipelines\n :alt: Coverage Report\n\n.. image:: https://codebeat.co/badges/fabc61ba-6a20-4bd1-bf73-a2f091a9ad80\n :target: https://codebeat.co/projects/github-com-scrapedia-scrapy-pipelines-master\n :alt: codebeat badge\n\n.. image:: https://api.codacy.com/project/badge/Grade/aeda92e058434a9eb2e8b0512a02235f\n :target: https://www.codacy.com/app/grammy-jiang/scrapy-pipelines?utm_source=github.com&utm_medium=referral&utm_content=scrapedia/scrapy-pipelines&utm_campaign=Badge_Grade\n\n.. image:: https://pyup.io/repos/github/scrapedia/scrapy-pipelines/shield.svg\n :target: https://pyup.io/repos/github/scrapedia/scrapy-pipelines/\n :alt: Updates\n\n.. image:: https://snyk.io/test/github/scrapedia/scrapy-pipelines/badge.svg\n :target: https://snyk.io/test/github/scrapedia/scrapy-pipelines\n :alt: Known Vulnerabilities\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n :target: https://github.com/python/black\n :alt: Code style: black\n\n.. image:: https://img.shields.io/badge/License-GPLv3-blue.svg\n :target: https://www.gnu.org/licenses/gpl-3.0\n :alt: License: AGPL v3\n\nThese pipelines enable Scrapy to save items into various backends, including:\n\n* MongoDB\n\nAnd also these pipelines provide multiple ways to save or update the items, and\nreturn id created by backends\n\nRequirements\n=============\n\n.. image:: https://pyup.io/repos/github/scrapedia/r18/python-3-shield.svg\n :target: https://pyup.io/repos/github/scrapedia/r18/\n :alt: Python 3\n\n* Python 3.6+\n* Works on Linux, Windows, Mac OSX\n\nInstallation\n============\n\n.. image:: https://img.shields.io/pypi/v/scrapy-pipelines.svg\n :target: https://pypi.python.org/pypi/scrapy-pipelines\n :alt: PyPI\n.. image:: https://img.shields.io/pypi/pyversions/scrapy-pipelines.svg\n :target: https://pypi.python.org/pypi/scrapy-pipelines\n :alt: PyPI - Python Version\n.. image:: https://img.shields.io/pypi/wheel/scrapy-pipelines.svg\n :target: https://pypi.python.org/pypi/scrapy-pipelines\n :alt: PyPI - Wheel\n\nThe quick way:\n\n pip install scrapy-pipelines\n\nFor more details see the installation section in the documentation:\nhttps://scrapy-pipelines.readthedocs.io/en/latest/intro/installation.html\n\nDocumentation\n=============\n\nDocumentation is available online at\nhttps://scrapy-pipelines.readthedocs.io/en/latest/ and in the docs directory.\n\nCommunity (blog, twitter, mail list, IRC)\n=========================================\n\n*Keeping this section same as Scrapy is intending to benefit back to Scrapy.*\n\nSee https://scrapy.org/community/\n\nContributing\n============\n\n*Keeping this section same as Scrapy is intending to be easier when this repo\nmerge back to Scrapy.*\n\nSee https://doc.scrapy.org/en/master/contributing.html\n\nCode of Conduct\n---------------\n\nPlease note that this project is released with a Contributor Code of Conduct\n(see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).\n\nBy participating in this project you agree to abide by its terms.\nPlease report unacceptable behavior to opensource@scrapinghub.com.\n\n\nCompanies using Scrapy\n======================\n\n*Keeping this section same as Scrapy is intending to benefit back to Scrapy.*\n\nSee https://scrapy.org/companies/\n\nCommercial Support\n==================\n\n*Keeping this section same as Scrapy is intending to benefit back to Scrapy.*\n\nSee https://scrapy.org/support/\n\nTODO\n====\n\n* [X] Add indexes creation in open_spider()\n* [X] Add item_completed method\n* [X] Add signals for MongoDB document's id return\n* [ ] Add MongoDB document update\n* [ ] Add Percona Server for MongoDB docker support\n* [ ] Add Redis support\n* [ ] Add InfluxDB support\n* [ ] Add LevelDB support\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/scrapedia/scrapy-pipelines", "keywords": "", "license": "GPLv3", "maintainer": "Scrapedia", "maintainer_email": "Scrapedia@outlook.com", "name": "Scrapy-Pipelines", "package_url": "https://pypi.org/project/Scrapy-Pipelines/", "platform": "", "project_url": "https://pypi.org/project/Scrapy-Pipelines/", "project_urls": { "Homepage": "https://github.com/scrapedia/scrapy-pipelines" }, "release_url": "https://pypi.org/project/Scrapy-Pipelines/0.2/", "requires_dist": [ "scrapy", "txmongo" ], "requires_python": "", "summary": "A collection of scrapy item pipelines", "version": "0.2" }, "last_serial": 5389392, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "0d7d6da52ddbbe70b929805596516d91", "sha256": "da53a9a8e1a14f237df2b1e8dedca966d192ba159a978c426a300c30d3c1795f" }, "downloads": -1, "filename": "Scrapy_Pipelines-0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "0d7d6da52ddbbe70b929805596516d91", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14979, "upload_time": "2019-05-22T13:50:07", "url": "https://files.pythonhosted.org/packages/56/2d/38ec783e50b9dce565e92a3b8dd67b635f3255db0d534e96c63618a1e851/Scrapy_Pipelines-0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f3476b1423ab35b997071beb644b4f38", "sha256": "04d2e630b9a2bdb1276010561cbcb572be22980c97b6cd0a641d7f7c96051c7f" }, "downloads": -1, "filename": "Scrapy-Pipelines-0.1.tar.gz", "has_sig": false, "md5_digest": "f3476b1423ab35b997071beb644b4f38", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19218, "upload_time": "2019-05-22T13:50:10", "url": "https://files.pythonhosted.org/packages/73/cb/3a778b82e190fa43f28b3786ec88837256af972ff5d1800c1448d9d4edda/Scrapy-Pipelines-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "c67e197b3eba77f03d0ebfa581134572", "sha256": "e5f336e5b69b79e6653dcc5d49a34184580107c7a84941f973ff5e69b64710fa" }, "downloads": -1, "filename": "Scrapy_Pipelines-0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "c67e197b3eba77f03d0ebfa581134572", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 25017, "upload_time": "2019-06-12T04:24:13", "url": "https://files.pythonhosted.org/packages/74/10/4fa4c50099386727714960d96822d51908d30ab2ce582da134032586ab01/Scrapy_Pipelines-0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "709d033b87674d040029ae36952a0d69", "sha256": "6812b3438a6c1cdfbb024f3448101637fc40db03731a16b5533f93a09617b392" }, "downloads": -1, "filename": "Scrapy-Pipelines-0.2.tar.gz", "has_sig": false, "md5_digest": "709d033b87674d040029ae36952a0d69", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22128, "upload_time": "2019-06-12T04:24:15", "url": "https://files.pythonhosted.org/packages/13/d6/6ca908c0584ad4f61546d407efaad855094f7003d4a67a6a1b40692f1c29/Scrapy-Pipelines-0.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "c67e197b3eba77f03d0ebfa581134572", "sha256": "e5f336e5b69b79e6653dcc5d49a34184580107c7a84941f973ff5e69b64710fa" }, "downloads": -1, "filename": "Scrapy_Pipelines-0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "c67e197b3eba77f03d0ebfa581134572", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 25017, "upload_time": "2019-06-12T04:24:13", "url": "https://files.pythonhosted.org/packages/74/10/4fa4c50099386727714960d96822d51908d30ab2ce582da134032586ab01/Scrapy_Pipelines-0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "709d033b87674d040029ae36952a0d69", "sha256": "6812b3438a6c1cdfbb024f3448101637fc40db03731a16b5533f93a09617b392" }, "downloads": -1, "filename": "Scrapy-Pipelines-0.2.tar.gz", "has_sig": false, "md5_digest": "709d033b87674d040029ae36952a0d69", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22128, "upload_time": "2019-06-12T04:24:15", "url": "https://files.pythonhosted.org/packages/13/d6/6ca908c0584ad4f61546d407efaad855094f7003d4a67a6a1b40692f1c29/Scrapy-Pipelines-0.2.tar.gz" } ] }