{ "info": { "author": "aodj", "author_email": "alexodonovanjones@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "|Build Status| |Coveralls Status| |Requirements Status|\n\nscrapy-sqs-exporter\n===================\n\nThis is an extension to Scrapy_ to allow exporting of scraped items to an Amazon SQS instance.\n\nSetup\n=====\n\nAfter installing the package, the two classes defined in the library need to be added to the relevant\nsections of the settings file::\n\n FEED_EXPORTERS = {\n 'sqs': 'sqsfeedexport.SQSExporter'\n }\n\n FEED_STORAGES = {\n 'sqs': 'sqsfeedexport.SQSFeedStorage'\n }\n\nThe ``FEED_STORAGES`` section uses a URL prefixed with ``sqs`` to differentiate it from other URI based storage\noptions.\n\nIn the environment we also need to define some keys::\n\n AWS_DEFAULT_REGION=eu-central-1\n AWS_ACCESS_KEY_ID=...\n AWS_SECRET_ACCESS_KEY=...\n FEED_URI=sqs://foo\n FEED_FORMAT=sqs\n\nThe ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY`` are the AWS credentials to be used, and ``AWS_DEFAULT_REGION``\nis the region to default to for the SQS instance. ``FEED_URI`` is the name of the AWS SQS instance in the\n``AWS_DEFAULT_REGION`` region for example::\n\n AWS_DEFAULT_REGION=us-east-1\n FEED_URI=sqs://bar\n FEED_FORMAT=sqs\n\nwould refer to a queue name ``bar`` in the ``us-east-1`` region.\n\nFinally, the ``FEED_FORMAT`` option makes the Scrapy spiders use the SQSExporter class.\n\n.. _Scrapy: https://github.com/scrapy/scrapy/\n.. |Build Status| image:: https://travis-ci.org/multiplechoice/scrapy-sqs-exporter.svg?branch=master\n :target: https://travis-ci.org/multiplechoice/scrapy-sqs-exporter\n.. |Coveralls Status| image:: https://coveralls.io/repos/github/multiplechoice/scrapy-sqs-exporter/badge.svg?branch=master\n :target: https://coveralls.io/github/multiplechoice/scrapy-sqs-exporter?branch=master\n.. |Requirements Status| image:: https://requires.io/github/multiplechoice/scrapy-sqs-exporter/requirements.svg?branch=master\n :target: https://requires.io/github/multiplechoice/scrapy-sqs-exporter/requirements/?branch=master\n :alt: Requirements Status\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/multiplechoice/scrapy-sqs-exporter", "keywords": "scrapy sqs", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "scrapy-sqs-exporter", "package_url": "https://pypi.org/project/scrapy-sqs-exporter/", "platform": "", "project_url": "https://pypi.org/project/scrapy-sqs-exporter/", "project_urls": { "Homepage": "https://github.com/multiplechoice/scrapy-sqs-exporter" }, "release_url": "https://pypi.org/project/scrapy-sqs-exporter/1.1.0/", "requires_dist": [ "boto3", "Scrapy", "six", "pytest", "pytest-cov", "moto", "pytest-env" ], "requires_python": "", "summary": "Scrapy extension for outputting scraped items to an Amazon SQS instance", "version": "1.1.0" }, "last_serial": 3950965, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "93a3394ce0690f19fb66058e00f707a8", "sha256": "a7fb7a6d6dd979c68e7fbde5b7a1365c0c5e6cbf33448fd303998d7d25312880" }, "downloads": -1, "filename": "scrapy-sqs-exporter-1.0.0.tar.gz", "has_sig": false, "md5_digest": "93a3394ce0690f19fb66058e00f707a8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3265, "upload_time": "2017-06-07T18:43:23", "url": "https://files.pythonhosted.org/packages/6f/9a/bf3281fcd68c417d8b3b7bc4ef97b7fb448e67a4370b8e59c905b8d795d6/scrapy-sqs-exporter-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "19b9c15345eb192f466bf68c81f09d11", "sha256": "5e62655f9f18ac63ed91793ccff8e4bb92e925cae164fc34b70d524c47925274" }, "downloads": -1, "filename": "scrapy-sqs-exporter-1.0.1.tar.gz", "has_sig": false, "md5_digest": "19b9c15345eb192f466bf68c81f09d11", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3267, "upload_time": "2017-06-07T18:49:19", "url": "https://files.pythonhosted.org/packages/6e/58/d68849553af9362b4c162d6b975e27b3b16f6d1a2590f7c867c53898a78f/scrapy-sqs-exporter-1.0.1.tar.gz" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "938936865819a8223a77ed763ff871ae", "sha256": "87d225ed29884fc2a31996d68386f7e02c7a133b3c6ca15fe9c86cb44a385937" }, "downloads": -1, "filename": "scrapy-sqs-exporter-1.0.2.tar.gz", "has_sig": false, "md5_digest": "938936865819a8223a77ed763ff871ae", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3270, "upload_time": "2017-06-07T18:57:26", "url": "https://files.pythonhosted.org/packages/2a/b7/6cef54f56169d43e73ee53b790fa4a44e5a14f63b3c1bd06aaca5b0f52f6/scrapy-sqs-exporter-1.0.2.tar.gz" } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "dbfac71d641cdbf3192da783ece4ac80", "sha256": "4916db88ec28c7e6fcd3a14392bb99ee86e8ea64f2a20798980263df41f14dff" }, "downloads": -1, "filename": "scrapy_sqs_exporter-1.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "dbfac71d641cdbf3192da783ece4ac80", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 5066, "upload_time": "2017-06-07T19:07:41", "url": "https://files.pythonhosted.org/packages/fa/7a/d2cb6e53f885a35daf4f81edf31a88dae3f5dd0d7f785602f6d8878579b9/scrapy_sqs_exporter-1.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "73f124b84bd975819d261766e8ace751", "sha256": "1c35863c9c70a48e130dd8d049f21689a2085b6f7fda372bfc823156c2357364" }, "downloads": -1, "filename": "scrapy-sqs-exporter-1.0.3.tar.gz", "has_sig": false, "md5_digest": "73f124b84bd975819d261766e8ace751", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3268, "upload_time": "2017-06-07T19:07:43", "url": "https://files.pythonhosted.org/packages/6e/a2/49f90bc269066c0cc1844099178cf6939b50b0b996484bd03683a3790b84/scrapy-sqs-exporter-1.0.3.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "4d30ff80a7c298a55667023676c8622a", "sha256": "849b56ff21624d25d7b4f7d7ab5d398247829886fb7c24b356138e4e81af70d6" }, "downloads": -1, "filename": "scrapy_sqs_exporter-1.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "4d30ff80a7c298a55667023676c8622a", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 5085, "upload_time": "2017-06-16T12:32:26", "url": "https://files.pythonhosted.org/packages/3a/f1/2547dd671531631b1c48bdbd309f8980d51c588a18775e7f6de0d4d88a76/scrapy_sqs_exporter-1.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b3ded3abca218a8d716c1509d22aac13", "sha256": "b2e240ff4ab92c6c62d1f003460b42b2cff09ac560b3539de5a746fc73fa105b" }, "downloads": -1, "filename": "scrapy-sqs-exporter-1.0.4.tar.gz", "has_sig": false, "md5_digest": "b3ded3abca218a8d716c1509d22aac13", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3278, "upload_time": "2017-06-16T12:32:28", "url": "https://files.pythonhosted.org/packages/a8/7e/7bfd947bce4680786d94a19b6dc1accffa31f451e44244033221dc7dd087/scrapy-sqs-exporter-1.0.4.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "915e8a399e2f6b36ce962ccd829a944f", "sha256": "57a729fc1852743bf166b3390ca195ef26e29e7cd09fcccfee33449d207112e8" }, "downloads": -1, "filename": "scrapy_sqs_exporter-1.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "915e8a399e2f6b36ce962ccd829a944f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 3441, "upload_time": "2018-06-11T16:29:40", "url": "https://files.pythonhosted.org/packages/1d/a3/969b94347c5e9dd16c9f18a9e6d1c4a81f5df220f38c5cb9ba31a3855f31/scrapy_sqs_exporter-1.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3f0677175ec3591bf8e85740bba13b2e", "sha256": "bb4c6a13ef0e1507c7c26618787fd644a5b756ed4d58352c8d5c4678c6ede0ea" }, "downloads": -1, "filename": "scrapy-sqs-exporter-1.1.0.tar.gz", "has_sig": false, "md5_digest": "3f0677175ec3591bf8e85740bba13b2e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3256, "upload_time": "2018-06-11T16:29:41", "url": "https://files.pythonhosted.org/packages/8a/bf/42b7412b945a3f7f27bec08ec686a8fd80c6e41f0427804ab528275cf877/scrapy-sqs-exporter-1.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "915e8a399e2f6b36ce962ccd829a944f", "sha256": "57a729fc1852743bf166b3390ca195ef26e29e7cd09fcccfee33449d207112e8" }, "downloads": -1, "filename": "scrapy_sqs_exporter-1.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "915e8a399e2f6b36ce962ccd829a944f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 3441, "upload_time": "2018-06-11T16:29:40", "url": "https://files.pythonhosted.org/packages/1d/a3/969b94347c5e9dd16c9f18a9e6d1c4a81f5df220f38c5cb9ba31a3855f31/scrapy_sqs_exporter-1.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3f0677175ec3591bf8e85740bba13b2e", "sha256": "bb4c6a13ef0e1507c7c26618787fd644a5b756ed4d58352c8d5c4678c6ede0ea" }, "downloads": -1, "filename": "scrapy-sqs-exporter-1.1.0.tar.gz", "has_sig": false, "md5_digest": "3f0677175ec3591bf8e85740bba13b2e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3256, "upload_time": "2018-06-11T16:29:41", "url": "https://files.pythonhosted.org/packages/8a/bf/42b7412b945a3f7f27bec08ec686a8fd80c6e41f0427804ab528275cf877/scrapy-sqs-exporter-1.1.0.tar.gz" } ] }