{ "info": { "author": "", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Framework :: Scrapy", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Internet :: WWW/HTTP", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "==========\nscrapy_rss\n==========\n\n.. image:: https://img.shields.io/pypi/v/scrapy_rss.svg\n :target: https://pypi.python.org/pypi/scrapy_rss\n :alt: PyPI Version\n\n.. image:: https://img.shields.io/travis/woxcab/scrapy_rss/master.svg\n :target: http://travis-ci.org/woxcab/scrapy_rss\n :alt: Build Status\n\n.. image:: https://img.shields.io/badge/wheel-yes-brightgreen.svg\n :target: https://pypi.python.org/pypi/scrapy_rss\n :alt: Wheel Status\n\n.. image:: https://img.shields.io/codecov/c/github/woxcab/scrapy_rss/master.svg\n :target: http://codecov.io/github/woxcab/scrapy_rss?branch=master\n :alt: Coverage report\n\nTools for easy `RSS feed `_ generating that contains each scraped item using `Scrapy framework `_.\n\nPackage works with Python 2.7, 3.3, 3.4, 3.5, 3.6 and 3.7.\n\nIf you use Python 3.3 then you have to use Scrapy<1.5.0.\n\n\nTable of Contents\n=================\n* `Installation <#installation>`__\n* `How To Use <#how-to-use>`__\n\n * `Configuration <#configuration>`__\n * `Optional Additional Customization <#feed-channel-elements-customization-optionally>`__\n * `Usage <#usage>`__\n\n* `Scrapy Project Examples <#scrapy-project-examples>`__\n\n\n`Installation `_\n==========================================================\n* Install :code:`scrapy_rss` using pip\n\n .. code:: bash\n\n pip install scrapy_rss\n\n or using pip for the specific interpreter, e.g.:\n\n .. code:: bash\n\n pip3 install scrapy_rss\n\n* or using setuptools directly:\n\n .. code:: bash\n\n cd path/to/root/of/scrapy_rss\n python setup.py install\n\n or using setuptools for specific interpreter, e.g.:\n\n .. code:: bash\n\n cd path/to/root/of/scrapy_rss\n python3 setup.py install\n\n\nHow To Use\n==========\n\nConfiguration\n-------------\n\nAdd parameters to the Scrapy project settings (`settings.py` file)\nor to the :code:`custom_settings` attribute of the spider:\n\n1. Add item pipeline that export items to rss feed:\n\n .. code:: python\n\n ITEM_PIPELINES = {\n # ...\n 'scrapy_rss.pipelines.RssExportPipeline': 900, # or another priority\n # ...\n }\n\n\n2. Add required feed parameters:\n\n FEED_FILE\n the absolute or relative file path where the result RSS feed will be saved.\n For example, :code:`feed.rss` or :code:`output/feed.rss`.\n FEED_TITLE\n the name of the channel (feed),\n FEED_DESCRIPTION\n the phrase or sentence that describes the channel (feed),\n FEED_LINK\n the URL to the HTML website corresponding to the channel (feed)\n\n .. code:: python\n\n FEED_FILE = 'path/to/feed.rss'\n FEED_TITLE = 'Some title of the channel'\n FEED_LINK = 'http://example.com/rss'\n FEED_DESCRIPTION = 'About channel'\n\n\nFeed (Channel) Elements Customization [optionally]\n--------------------------------------------------\n\nIf you want to change another channel parameters (such as language, copyright, managing_editor,\nwebmaster, pubdate, last_build_date, category, generator, docs, ttl)\nthen declare your own exporter that's inherited from :code:`RssItemExporter` class, for example:\n\n.. code:: python\n\n from scrapy_rss.exporters import RssItemExporter\n\n class MyRssItemExporter(RssItemExporter):\n def __init__(self, *args, **kwargs):\n kwargs['generator'] = kwargs.get('generator', 'Special generator')\n kwargs['language'] = kwargs.get('language', 'en-us')\n super(CustomRssItemExporter, self).__init__(*args, **kwargs)\n\nAnd add :code:`FEED_EXPORTER` parameter to the Scrapy project settings\nor to the :code:`custom_settings` attribute of the spider:\n\n.. code:: python\n\n FEED_EXPORTER = 'myproject.exporters.MyRssItemExporter'\n\n\nUsage\n-----\n\nDeclare your item directly as RssItem():\n\n.. code:: python\n\n import scrapy_rss\n\n item1 = scrapy_rss.RssItem()\n\nOr use predefined item class :code:`RssedItem` with RSS field named as :code:`rss`\nthat's instance of :code:`RssItem`:\n\n.. code:: python\n\n import scrapy_rss\n\n class MyItem(scrapy_rss.RssedItem):\n field1 = scrapy.Field()\n field2 = scrapy.Field()\n # ...\n\n item2 = MyItem()\n\n\nSet/get item fields. Case sensitive attributes of :code:`RssItem()` are appropriate to RSS elements,\nAttributes of RSS elements are case sensitive too.\nIf editor allows autocomplete then it suggests attributes for instances of :code:`RssedItem` and :code:`RssItem`.\nIt's allowed to set **any** subset of RSS elements (e.g. only title). For example:\n\n.. code:: python\n\n from datetime import datetime\n\n item1.title = 'RSS item title' # set value of element\n title = item1.title.title # get value of <title> element\n item1.description = 'description'\n\n item1.guid = 'item identifier'\n item1.guid.isPermaLink = True # set value of attribute isPermalink of <guid> element,\n # isPermaLink is False by default\n is_permalink = item1.guid.isPermaLink # get value of attribute isPermalink of <guid> element\n guid = item1.guid.guid # get value of element <guid>\n\n item1.category = 'single category'\n category = item1.category\n item1.category = ['first category', 'second category']\n first_category = item1.category[0].category # get value of the element <category> with multiple values\n all_categories = [cat.category for cat in item1.category]\n\n # direct attributes setting\n item1.enclosure.url = 'http://example.com/file'\n item1.enclosure.length = 0\n item1.enclosure.type = 'text/plain'\n\n # or dict based attributes setting\n item1.enclosure = {'url': 'http://example.com/file', 'length': 0, 'type': 'text/plain'}\n item1.guid = {'guid': 'item identifier', 'isPermaLink': True}\n\n item1.pubDate = datetime.now() # correctly works with Python' datetimes\n\n\n item2.rss.title = 'Item title'\n item2.rss.guid = 'identifier'\n item2.rss.enclosure = {'url': 'http://example.com/file', 'length': 0, 'type': 'text/plain'}\n\n\nAll allowed elements are listed in the `scrapy_rss/items.py <https://github.com/woxcab/scrapy_rss/blob/master/scrapy_rss/items.py>`_.\nAll allowed attributes of each element with constraints and default values\nare listed in the `scrapy_rss/elements.py <https://github.com/woxcab/scrapy_rss/blob/master/scrapy_rss/elements.py>`_.\nYou also can read `RSS specification <http://www.rssboard.org/rss-specification>`_ for more details.\n\nScrapy Project Examples\n=======================\n\n`Examples directory <https://github.com/woxcab/scrapy_rss/blob/master/examples>`_ contains\nseveral Scrapy projects with the scrapy_rss usage demonstration. It crawls\n`this website <https://woxcab.github.io/scrapy_rss/>`_ whose source code is\n`here <https://github.com/woxcab/scrapy_rss/blob/master/examples/website>`_.\n\nJust go to the Scrapy project directory and run commands\n\n.. code:: bash\n\n scrapy crawl first_spider\n scrapy crawl second_spider\n\nThereafter `feed.rss` and `feed2.rss` files will be created in the same directory.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/woxcab/scrapy_rss", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "scrapy-rss", "package_url": "https://pypi.org/project/scrapy-rss/", "platform": "", "project_url": "https://pypi.org/project/scrapy-rss/", "project_urls": { "Homepage": "https://github.com/woxcab/scrapy_rss" }, "release_url": "https://pypi.org/project/scrapy-rss/0.1.7/", "requires_dist": [ "cryptography (<2.0)", "pyOpenSSL (<17.3.0)", "python-dateutil", "scrapy (<1.5.0)", "six" ], "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*", "summary": "RSS Tools for Scrapy Framework", "version": "0.1.7" }, "last_serial": 5564788, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "0b90d9344898dcad48fca61785d31e06", "sha256": "75dec407fd68a6cc472522e17d3a5e4498b8b0b0041778ff2fc52fe4fe34d3a3" }, "downloads": -1, "filename": "scrapy_rss-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0b90d9344898dcad48fca61785d31e06", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12671, "upload_time": "2017-02-02T08:18:30", "url": "https://files.pythonhosted.org/packages/65/e2/f89d780ab496c95ded1edcccec5561d78ff1d54eeeffba17bbae6a4e5e7f/scrapy_rss-0.1.0-py2.py3-none-any.whl" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "90d01588be0fcbfa6a94c0cae7e86b90", "sha256": "9fa39f687481ed74f141f493ca371a3d28f9531df652746dace3cba7b8c7ae5f" }, "downloads": -1, "filename": "scrapy_rss-0.1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "90d01588be0fcbfa6a94c0cae7e86b90", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12851, "upload_time": "2017-02-02T10:16:45", "url": "https://files.pythonhosted.org/packages/45/66/0d9898555b12030992e82c543031c3239f8bca88105558c6322587eec1b2/scrapy_rss-0.1.1-py2.py3-none-any.whl" } ], "0.1.2": [], "0.1.3": [ { "comment_text": "", "digests": { "md5": "b41cf6e7c0c30ad8f7eade64f67cb6ea", "sha256": "79d8d44696d813f653b62ba8634af0036a0a723404570e9406dfe63b902c8869" }, "downloads": -1, "filename": "scrapy_rss-0.1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b41cf6e7c0c30ad8f7eade64f67cb6ea", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 13093, "upload_time": "2017-06-19T19:30:46", "url": "https://files.pythonhosted.org/packages/e3/5c/62007ae82d01a47b118cc1fb663ef0003d7be694d44e907ec4219624f01e/scrapy_rss-0.1.3-py2.py3-none-any.whl" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "076f873ce7a66b71901cc3ed86eeb3b4", "sha256": "7bbdc84183f3648b989f2e7b2a8f9b2c36d339415aa31252e03ad02cb2704f72" }, "downloads": -1, "filename": "scrapy_rss-0.1.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "076f873ce7a66b71901cc3ed86eeb3b4", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 14476, "upload_time": "2017-06-26T10:19:50", "url": "https://files.pythonhosted.org/packages/5c/f9/acdffadf08b29fac36f4b56edae487e92a5433bc373dfa2fe640681012e9/scrapy_rss-0.1.4-py2.py3-none-any.whl" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "fdc666eb50ec3c0d77415a89fb3676c6", "sha256": "98bba4ee1a7d51bec662345bbbf7e4901a420318fe03807ae1e5683c4b14408b" }, "downloads": -1, "filename": "scrapy_rss-0.1.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "fdc666eb50ec3c0d77415a89fb3676c6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*", "size": 14621, "upload_time": "2017-12-30T06:10:26", "url": "https://files.pythonhosted.org/packages/b2/d6/91a3e13994f73557a3e355377a3112a0006bda23e2c727339364e585b7e5/scrapy_rss-0.1.5-py2.py3-none-any.whl" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "0acb5c941249de8246d60e4db19036f2", "sha256": "f3110c467f0ef219d6e103696f311ee35c70926d7e20fa0f3ad3fbf4f44aa347" }, "downloads": -1, "filename": "scrapy_rss-0.1.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0acb5c941249de8246d60e4db19036f2", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*", "size": 14581, "upload_time": "2018-04-03T17:56:18", "url": "https://files.pythonhosted.org/packages/85/95/aab571b9663a53a85ebfaa7b467c7412bb4d793d5c0fb003328729a308af/scrapy_rss-0.1.6-py2.py3-none-any.whl" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "729efc6d88c09791af776647933d1824", "sha256": "a29e8f749b07a519971d67245ecf0f71bf72fde55249bba330eb4f105157d2ac" }, "downloads": -1, "filename": "scrapy_rss-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "729efc6d88c09791af776647933d1824", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*", "size": 14648, "upload_time": "2019-07-21T21:41:56", "url": "https://files.pythonhosted.org/packages/0a/6a/266b6f50760672149c49720ec8b40fa04229a01dbe8bf88a252af0577766/scrapy_rss-0.1.7-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "729efc6d88c09791af776647933d1824", "sha256": "a29e8f749b07a519971d67245ecf0f71bf72fde55249bba330eb4f105157d2ac" }, "downloads": -1, "filename": "scrapy_rss-0.1.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "729efc6d88c09791af776647933d1824", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*", "size": 14648, "upload_time": "2019-07-21T21:41:56", "url": "https://files.pythonhosted.org/packages/0a/6a/266b6f50760672149c49720ec8b40fa04229a01dbe8bf88a252af0577766/scrapy_rss-0.1.7-py2.py3-none-any.whl" } ] }