{ "info": { "author": "Mathijs de Bruin", "author_email": "mathijs@visualspace.nl", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Web Environment", "Framework :: Django", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Utilities" ], "description": "newspeak: Standalone Django based feed aggregator\n==================================================\n\n.. image:: https://secure.travis-ci.org/bitsoffreedom/newspeak.png?branch=master\n :target: http://travis-ci.org/bitsoffreedom/newspeak\n\nWhat it does\n------------\nNewspeak is a feed aggregator with advanced features for keyword filtering\nand link content extraction, implemented as a standaloone Django application.\n\nArchitecture\n------------\nNewspeak performs the following tasks (in order):\n\n#. Fetch specified RSS/Atom feeds as per the `Feed `_ model (in parallel).\n#. Parses the feeds using `feedparser `_.\n#. (Optionally) applies per-feed inclusive/exclusive keyword filters on the title and/or summary, based on the `KeywordFilter `_ model.\n#. (Optionally) extract summary data using an XPath expression from feed entry's link URL, using `lxml `_.\n#. (Optionally) extract enclosure information using XPath expressions from the feed entry's link URL, using `lxml `_.\n#. Store the resulting feed information locally in a database.\n#. Serve the aggregate of all the feed entries in a single RSS/Atom feed.\n\nThe flow of feed data through the application is roughly as follows (given some example feeds and keyword filters)::\n\n [Feed 1]-[Keyword filter 1]-[Keyword filter 2]-[XPath content extraction]-----------------------------`\\\n [Feed 2]--------------------[Keyword filter 3]-[XPath summary extraction]-[XPath content extraction ] -+--[Aggregate output feed]\n [Feed 3]-[Keyword filter 3]-[Keyword filter 4]---------------------------------------------------------/\n\nInstalling\n----------------\nGetting started with newspeak is really easy thanks to David Cramer's awesome\n`logan `_ for making standalone Django apps.\nSimply perform the following steps:\n\n#. Install such that you can easily code along::\n\n pip install -e \\\n git+https://github.com/bitsoffreedom/newspeak.git#egg=newspeak\n\n If you're smart and like to keep your Python environment clean, do this\n in a `VirtualEnv `_.\n\n#. Initialize configuration in `~/.newspeak/newspeak.conf.py`::\n\n newspeak init\n\n#. Perform (optional) configuration by editing the settings file. Because\n Newspeak is based on Django, all\n `available Django settings `_\n can be used. Furthermore, there are some Newspeak-specific settings:\n\n * `NEWSPEAK_THREADS`: The number of (lightweight) threads used for crawling\n feed data.\n * `NEWSPEAK_METADATA`: Metadata used in the generated output feed.\n\n For a more thorough description and an example of these settings, please\n have a look at the initial settings file generated in the previous step.\n\n#. (Optionally) Run the tests::\n\n newspeak test newspeak\n\n This might take a while, so go fetch a cup of coffee. If something fails,\n please supply the output of the command `newspeak test newspeak --traceback`\n in an `issue `_ on GitHub.\n\n#. Create admin user and SQLite database (proper database is optional)::\n\n newspeak syncdb --migrate\n\n#. Start the local webserver::\n\n newspeak run_gunicorn\n\n#. Open `http://127.0.0.1:8000/admin/` in your browser, add some feed. Only\n the URL is required, the description and title will be fetched\n automatically, as well as the first set of entries.\n\n#. (Optionally) Configure one or more keyword-based filters for your feed(s).\n\n#. Make sure the following command gets executed to update the feeds::\n\n newspeak update_feeds\n\n (Optionally, add `-v <1|2|3>` to get more feedback on the process.)\n\n#. Look at the pretty feeds: open `http://127.0.0.1:8000/all/rss/` or\n `http://127.0.0.1:8000/all/atom/` in your favorite feed reader. All input\n feeds will be aggregated there.\n\n Alternatively, the original feeds, keywords and XPath expressions as used\n by Bits of Freedom are contained in a fixture called `feeds_bof.json`. This\n fixture can be loaded using::\n\n newspeak loaddata feeds_bof\n\n#. Setup a `Cronjob `_ to automatically\n update the feed data using the `newspeak update_feeds` command. For\n example, a cron job updating the feeds every hour could look as follows::\n\n 0 * * * * /newspeak update_feeds\n\nUpgrading\n----------\n#. Run the PIP installation command again::\n\n pip install -e \\\n git+https://github.com/bitsoffreedom/newspeak.git#egg=newspeak\n\n#. (Optionally) Run the tests::\n\n newspeak test newspeak\n\n#. Apply any database migrations::\n\n newspeak migrate", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/bitsoffreedom/newspeak/", "keywords": null, "license": "UNKNOWN", "maintainer": null, "maintainer_email": null, "name": "newspeak", "package_url": "https://pypi.org/project/newspeak/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/newspeak/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/bitsoffreedom/newspeak/" }, "release_url": "https://pypi.org/project/newspeak/0.1/", "requires_dist": null, "requires_python": null, "summary": "Standalone Django based feed aggregator.", "version": "0.1" }, "last_serial": 795340, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "064413de8c9b8ccb876406776c676d09", "sha256": "f61105d96e8a92d227b7fe7fe49dcb61c047948338c67204c2b2053d7a4500da" }, "downloads": -1, "filename": "newspeak-0.1.tar.gz", "has_sig": false, "md5_digest": "064413de8c9b8ccb876406776c676d09", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 29908, "upload_time": "2013-04-02T16:08:51", "url": "https://files.pythonhosted.org/packages/08/2a/c84583264cb6dc5f6f0c4a0f4a51b67052d14d9f924e5b045b4956877e46/newspeak-0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "064413de8c9b8ccb876406776c676d09", "sha256": "f61105d96e8a92d227b7fe7fe49dcb61c047948338c67204c2b2053d7a4500da" }, "downloads": -1, "filename": "newspeak-0.1.tar.gz", "has_sig": false, "md5_digest": "064413de8c9b8ccb876406776c676d09", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 29908, "upload_time": "2013-04-02T16:08:51", "url": "https://files.pythonhosted.org/packages/08/2a/c84583264cb6dc5f6f0c4a0f4a51b67052d14d9f924e5b045b4956877e46/newspeak-0.1.tar.gz" } ] }