{ "info": { "author": "Bruno Cezar Rocha", "author_email": "rochacbruno@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Framework :: Opps", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Topic :: Internet :: WWW/HTTP :: Dynamic Content", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "opps-feedcrawler\n================\n\nFeedCrawler takes a **feed** of any type, executes its customized processor in order to create CMS Entries.\n\n\nFeed\n====\n\nFeed is commonly a url with some configurations, **url**, **credentials**, **processor** and **actions**\n\nThe most simple example is an RSS feed\n\n- url = 'http://site.com/feed.rss'\n- processor = 'opps.feedcrawler.processors.rss.RSSProcessor'\n- actions = ['opps.feedcrawler.actions.rss.RSSActions'\n\nIn the above example we have an **url** to read feed entries, and feedcrawler comes with a builtin processor for RSS feeds\n**RSSProcessor** wil take the feed url and do all the job fetching, reading and creating **entries** on database.\n\n> You can replace RSSProcessor with your own processor class, following the processor API. \n\n> Example: 'yourproject.yourmodule.processors.MyProcessor'\n\n> The processor API is documented in the item **Processor API**\n\n\nAlso, your **feed** takes **actions** which is a path to a callable returning a list of Django admin actions in the form of functions.\nan example of action is \"Create posts\" which takes the selected entries and convert it in to Opps Posts.\n\nProcessor API\n=============\n\nfeedcrawler provides a **BaseProcessor** class for you to extend and you have to override some methods.\n\n\n\n from opps.feedcrawler.processors.base import BaseProcessor\n \n class MyProcessor(BaseProcessor):\n \"\"\"\n BaseProcessor.__init__ receives the **feed** object as parameter\n \n def __init__(feed, entry_model, *args, **kwargs):\n self.feed = feed\n self.entry_model = entry_model\n \n You override if you need, but be careful.\n \"\"\"\n \n def process(self):\n url = self.feed.source_url\n max_entries = self.feed.max_entries\n ...\n \n # here you have access to the **feed** object in **self.feed**\n entries = read_and_parse_rss_feed(url) # example function which fetch and parse XML feed \n \n # Now you have access to **self.entry_model** which you will use to create CMS entries.\n for entry in entries:\n # remember to implement your own logic to avoid duplications\n self.entry_model.objects.get_or_create(\n title=entry['title']\n ...\n ...\n )\n \n # this method should return the count of entries read and created or 0 \n return len(entries)\n \n \n\nThe processor above will be executed by management command **manage.py process_feeds -f feed_slug** also you can put this command to run on **cron** or **celery**", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/opps/opps-feedcrawler/tarball/master", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://oppsproject.org", "keywords": "rss parser opps cms django apps magazines websites", "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "opps-feedcrawler", "package_url": "https://pypi.org/project/opps-feedcrawler/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/opps-feedcrawler/", "project_urls": { "Download": "https://github.com/opps/opps-feedcrawler/tarball/master", "Homepage": "http://oppsproject.org" }, "release_url": "https://pypi.org/project/opps-feedcrawler/0.2.1/", "requires_dist": null, "requires_python": null, "summary": "Feed reader App for Opps CMS", "version": "0.2.1" }, "last_serial": 1229419, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "0fd3b4694304d1bd5a8bc92cd86a5692", "sha256": "a3b2bb2136b309b9aee64d6213dc81dd73c205c693f2a6abf7b7af94880630f3" }, "downloads": -1, "filename": "opps-feedcrawler-0.1.0.tar.gz", "has_sig": false, "md5_digest": "0fd3b4694304d1bd5a8bc92cd86a5692", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9321, "upload_time": "2013-05-29T02:18:19", "url": "https://files.pythonhosted.org/packages/7b/50/cba665215332c1510e93503de2410c1870578810e4d546cd508fc92633ec/opps-feedcrawler-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "ccb94add79ff2c2b99e66a8b7b0adbb0", "sha256": "42d7d90e473ed062f4adb7e9252cf7750cc606b0b2d8c149fa40e4b4dc01f07d" }, "downloads": -1, "filename": "opps-feedcrawler-0.1.1.tar.gz", "has_sig": false, "md5_digest": "ccb94add79ff2c2b99e66a8b7b0adbb0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9367, "upload_time": "2013-07-31T13:38:49", "url": "https://files.pythonhosted.org/packages/7a/c1/3f9f442c10915da790bf2617c772ec925523a43b51158c7ee056b3444262/opps-feedcrawler-0.1.1.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "18ee38ae43b7a67ce044e83e5cefa706", "sha256": "dd0df66c7d5b6d80ac82f0e4709017b140f3a549644197214034ca808c17b991" }, "downloads": -1, "filename": "opps_feedcrawler-0.2.1-py2.7.egg", "has_sig": false, "md5_digest": "18ee38ae43b7a67ce044e83e5cefa706", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 106571, "upload_time": "2014-09-18T15:35:09", "url": "https://files.pythonhosted.org/packages/70/1c/35c9fea007e37e5ddb9c59770b575a2afc31eac44585f77cbd04d48c1b63/opps_feedcrawler-0.2.1-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "bcf26d6e169000b4ff114f47ed83e89d", "sha256": "fe93d09f3acb896969ab95d0e97e799d1d50dfb8f559fe0dacd7cc76dfff7f7e" }, "downloads": -1, "filename": "opps-feedcrawler-0.2.1.tar.gz", "has_sig": false, "md5_digest": "bcf26d6e169000b4ff114f47ed83e89d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 31408, "upload_time": "2014-09-18T15:35:05", "url": "https://files.pythonhosted.org/packages/6e/a7/c58103253646c67fd999720dde61a80d177342d4ecf52e11ac001426c661/opps-feedcrawler-0.2.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "18ee38ae43b7a67ce044e83e5cefa706", "sha256": "dd0df66c7d5b6d80ac82f0e4709017b140f3a549644197214034ca808c17b991" }, "downloads": -1, "filename": "opps_feedcrawler-0.2.1-py2.7.egg", "has_sig": false, "md5_digest": "18ee38ae43b7a67ce044e83e5cefa706", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 106571, "upload_time": "2014-09-18T15:35:09", "url": "https://files.pythonhosted.org/packages/70/1c/35c9fea007e37e5ddb9c59770b575a2afc31eac44585f77cbd04d48c1b63/opps_feedcrawler-0.2.1-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "bcf26d6e169000b4ff114f47ed83e89d", "sha256": "fe93d09f3acb896969ab95d0e97e799d1d50dfb8f559fe0dacd7cc76dfff7f7e" }, "downloads": -1, "filename": "opps-feedcrawler-0.2.1.tar.gz", "has_sig": false, "md5_digest": "bcf26d6e169000b4ff114f47ed83e89d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 31408, "upload_time": "2014-09-18T15:35:05", "url": "https://files.pythonhosted.org/packages/6e/a7/c58103253646c67fd999720dde61a80d177342d4ecf52e11ac001426c661/opps-feedcrawler-0.2.1.tar.gz" } ] }