{ "info": { "author": "Artur Barseghyan", "author_email": "artur.barseghyan@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Framework :: Scrapy", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v2 (GPLv2)", "License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Utilities" ], "description": "=======================\nscrapy-mongoengine-item\n=======================\nBringing ``Scrapy`` and ``MongoEngine`` together.\n\n.. image:: https://img.shields.io/pypi/v/scrapy-mongoengine-item.svg\n :target: https://pypi.python.org/pypi/scrapy-mongoengine-item\n :alt: PyPI Version\n\n.. image:: https://img.shields.io/travis/barseghyanartur/scrapy-mongoengine-item/master.svg\n :target: http://travis-ci.org/barseghyanartur/scrapy-mongoengine-item\n :alt: Build Status\n\n.. image:: https://img.shields.io/badge/license-GPL--2.0--only%20OR%20LGPL--2.1--or--later-blue.svg\n :target: https://github.com/barseghyanartur/scrapy-mongoengine-item/#License\n :alt: GPL-2.0-only OR LGPL-2.1-or-later\n\n``scrapy-mongoengine-item`` is an extension that allows you to define `Scrapy items\n`_ using existing\n`MongoEngine documents `_.\n\nDocumentation is available on `Read the Docs\n`_.\n\nPrerequisites\n=============\nBoth ``Python 2.7`` and ``Python 3.5/3.6`` are\nsupported. For ``Python 3`` you need ``Scrapy v1.1`` or above.\n\nLatest tested MongoEngine version is ``MongoEngine 0.17.0``.\n\nInstallation\n============\n(1) Install latest stable version from PyPI:\n\n .. code-block:: sh\n\n pip install scrapy-mongoengine-item\n\n or latest stable version from GitHub:\n\n .. code-block:: sh\n\n pip install https://github.com/barseghyanartur/scrapy-mongoengine-item/archive/stable.tar.gz\n\n or latest stable version from BitBucket:\n\n .. code-block:: sh\n\n pip install https://bitbucket.org/barseghyanartur/scrapy-mongoengine-item/get/stable.tar.gz\n\nIntroduction\n============\n``MongoEngineItem`` is a class of item that gets its fields definition from a\nMongoEngine document, you simply create a ``MongoEngineItem`` and specify what\nMongoEngine document it relates to.\n\nBesides of getting the document fields defined on your item, ``MongoEngineItem``\nprovides a method to create and populate a MongoEngine document instance with\nthe item data.\n\nUsage\n=====\n``MongoEngineItem`` works as follows: you create a subclass and define its\n``mongoengine_document`` attribute to be a valid MongoEngine document. With\nthis you will get an item with a field for each MongoEngine document field.\n\nIn addition, you can define fields that aren't present in the document and even\noverride fields that are present in the model defining them in the item.\n\nLet's see some examples:\n\nCreating a MongoEngine document for the examples:\n\n.. code-block:: python\n\n from mongoengine import fields, document\n\n class Person(document.Document):\n\n name = fields.StringField(max_length=255)\n age = fields.IntField()\n\nDefining a basic ``MongoEngineItem``:\n\n.. code-block:: python\n\n from scrapy_mongoengine_item import MongoEngineItem\n\n class PersonItem(MongoEngineItem):\n\n mongoengine_document = Person\n\n``MongoEngineItem`` works just like Scrapy items:\n\n.. code-block:: python\n\n p = PersonItem()\n p['name'] = 'John'\n p['age'] = 22\n\nTo obtain the MongoEngine document from the item, we call the extra method\n``MongoEngineItem.save()`` of the ``MongoEngineItem``:\n\n.. code-block:: python\n\n person = p.save()\n person.name\n # 'John'\n person.age\n # 22\n person.id\n # 1\n\nThe document is already saved when we call ``MongoEngineItem.save()``, we\ncan prevent this by calling it with ``commit=False``. We can use\n``commit=False`` in ``MongoEngineItem.save()`` method to obtain an unsaved\ndocument:\n\n.. code-block:: python\n\n person = p.save(commit=False)\n person.name\n # 'John'\n person.age\n # 22\n person.id\n # None\n\nAs said before, we can add other fields to the item:\n\n.. code-block:: python\n\n import scrapy\n from scrapy_mongoengine_item import MongoEngineItem\n\n class PersonItem(MongoEngineItem):\n\n mongoengine_document = Person\n sex = scrapy.Field()\n\n.. code-block:: python\n\n p = PersonItem()\n p['name'] = 'John'\n p['age'] = 22\n p['sex'] = 'M'\n\nAnd we can override the fields of the document with your own:\n\n.. code-block:: python\n\n class PersonItem(MongoEngineItem):\n\n mongoengine_document = Person\n name = scrapy.Field(default='No Name')\n\nThis is useful to provide properties to the field, like a default or any other\nproperty that your project uses. Those additional fields won't be taken into\naccount when doing a ``MongoEngineItem.save()``.\n\nDevelopment\n===========\nTesting\n-------\nTo run tests in your working environment type:\n\n.. code-block:: sh\n\n ./runtests.py\n\nTo test with all supported Python versions type:\n\n.. code-block:: sh\n\n tox\n\nRunning MongoDB\n---------------\nThe easiest way is to run it via Docker:\n\n.. code-block:: sh\n\n docker pull mongo:latest\n docker run -p 27017:27017 mongo:latest\n\nWriting documentation\n---------------------\nKeep the following hierarchy.\n\n.. code-block:: text\n\n =====\n title\n =====\n\n header\n ======\n\n sub-header\n ----------\n\n sub-sub-header\n ~~~~~~~~~~~~~~\n\n sub-sub-sub-header\n ^^^^^^^^^^^^^^^^^^\n\n sub-sub-sub-sub-header\n ++++++++++++++++++++++\n\n sub-sub-sub-sub-sub-header\n **************************\n\nLicense\n=======\nGPL-2.0-only OR LGPL-2.1-or-later\n\nSupport\n=======\nFor any issues contact me at the e-mail given in the `Author`_ section.\n\nAuthor\n======\nArtur Barseghyan \n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/barseghyanartur/scrapy-mongoitem", "keywords": "", "license": "GPL-2.0-only OR LGPL-2.1-or-later", "maintainer": "", "maintainer_email": "", "name": "scrapy-mongoengine-item", "package_url": "https://pypi.org/project/scrapy-mongoengine-item/", "platform": "", "project_url": "https://pypi.org/project/scrapy-mongoengine-item/", "project_urls": { "Homepage": "https://github.com/barseghyanartur/scrapy-mongoitem" }, "release_url": "https://pypi.org/project/scrapy-mongoengine-item/0.1.5/", "requires_dist": null, "requires_python": "", "summary": "Scrapy extension to write scraped items using MongoEngine documents", "version": "0.1.5" }, "last_serial": 5331046, "releases": { "0.1.2": [ { "comment_text": "", "digests": { "md5": "f2242ba369fb458eb6a9b38f38577baa", "sha256": "defbb01ba02eecd6037fca995b0168d36460675ca93fb08387ec56a4169f3753" }, "downloads": -1, "filename": "scrapy_mongoengine_item-0.1.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f2242ba369fb458eb6a9b38f38577baa", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 4911, "upload_time": "2019-03-14T14:03:51", "url": "https://files.pythonhosted.org/packages/f4/eb/905f4b3f28c1b9c82f28e7b56322be33abee010559208af0a4184558617d/scrapy_mongoengine_item-0.1.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b4307a788abe1105df6ca9473a3e4b25", "sha256": "a56f5044224b38ec3ecb35aa631ad23159b5ffa403635e56e34a24ea2dce8fc7" }, "downloads": -1, "filename": "scrapy-mongoengine-item-0.1.2.tar.gz", "has_sig": false, "md5_digest": "b4307a788abe1105df6ca9473a3e4b25", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4033, "upload_time": "2019-03-14T14:03:49", "url": "https://files.pythonhosted.org/packages/ba/96/ea9879bab69c5631cbbbab4cd40793f46ee6962985ac4a83647c94b40e1f/scrapy-mongoengine-item-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "e7bdf34418a8efa6ad811a6b3147c298", "sha256": "cd3f3b249d305227d00541ec875f48f9dd4ce601b40de86f7034a5de1d68ace8" }, "downloads": -1, "filename": "scrapy_mongoengine_item-0.1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e7bdf34418a8efa6ad811a6b3147c298", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 20925, "upload_time": "2019-03-14T22:31:58", "url": "https://files.pythonhosted.org/packages/2d/ac/799210153423ea3e44943fdd6dc28ac1b5f7863206c8c7da252176a5f1b7/scrapy_mongoengine_item-0.1.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "77264602025c8f33e4aadc635065bc3d", "sha256": "d99d248423e3a861a44b61c5407a7080dff071b6d414b5c26d17b04c903a4be7" }, "downloads": -1, "filename": "scrapy-mongoengine-item-0.1.3.tar.gz", "has_sig": false, "md5_digest": "77264602025c8f33e4aadc635065bc3d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4504, "upload_time": "2019-03-14T22:31:56", "url": "https://files.pythonhosted.org/packages/0d/f9/3fd874c1989957a3fd849f7a5e25e5d035d20ac1c74db63f27bac301e5c8/scrapy-mongoengine-item-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "a741c213d486961392d04dd3ac0df33c", "sha256": "987f86b962f5ad7962d9cc0699450aafd1174eee7c3418f3d56266308d7f38c2" }, "downloads": -1, "filename": "scrapy_mongoengine_item-0.1.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a741c213d486961392d04dd3ac0df33c", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 21021, "upload_time": "2019-03-16T21:49:57", "url": "https://files.pythonhosted.org/packages/64/05/38868d88a908993a51bab59223a40261edb17e021bd23854ab5b4832c539/scrapy_mongoengine_item-0.1.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "122b8b1befd1fb3f861f708f7d889d17", "sha256": "1f721a1361019df9b70e223f46e5f83231f34cd08a51daf98c10604a101039c5" }, "downloads": -1, "filename": "scrapy-mongoengine-item-0.1.4.tar.gz", "has_sig": false, "md5_digest": "122b8b1befd1fb3f861f708f7d889d17", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4517, "upload_time": "2019-03-16T21:49:55", "url": "https://files.pythonhosted.org/packages/4d/a0/4fcfe8c8218c6f6e368b60132ac9701bb968fa4ebb6d18cfb70345407c15/scrapy-mongoengine-item-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "07203cf9e874a9f17df3c75c7b529d7d", "sha256": "3368410304d7dd9628c02d85433b078ff396cc257e291e812f09e5f4425c6ccc" }, "downloads": -1, "filename": "scrapy_mongoengine_item-0.1.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "07203cf9e874a9f17df3c75c7b529d7d", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 21083, "upload_time": "2019-05-29T08:13:10", "url": "https://files.pythonhosted.org/packages/36/21/154db3ed6cef4b3e9c7c2347033e223b4c336e118cfa0ab94b72a39026a9/scrapy_mongoengine_item-0.1.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2aeccd15c065a96beaa9407e9b2cc443", "sha256": "2e331e25c920eddc282399054c2ae8a121e432a74e75c3528b935984b7c242ed" }, "downloads": -1, "filename": "scrapy-mongoengine-item-0.1.5.tar.gz", "has_sig": false, "md5_digest": "2aeccd15c065a96beaa9407e9b2cc443", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4609, "upload_time": "2019-05-29T08:13:07", "url": "https://files.pythonhosted.org/packages/e3/13/16adb922920a15a6ee5acd06201920f407a95e299cb928e5d5d9f1aebae1/scrapy-mongoengine-item-0.1.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "07203cf9e874a9f17df3c75c7b529d7d", "sha256": "3368410304d7dd9628c02d85433b078ff396cc257e291e812f09e5f4425c6ccc" }, "downloads": -1, "filename": "scrapy_mongoengine_item-0.1.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "07203cf9e874a9f17df3c75c7b529d7d", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 21083, "upload_time": "2019-05-29T08:13:10", "url": "https://files.pythonhosted.org/packages/36/21/154db3ed6cef4b3e9c7c2347033e223b4c336e118cfa0ab94b72a39026a9/scrapy_mongoengine_item-0.1.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2aeccd15c065a96beaa9407e9b2cc443", "sha256": "2e331e25c920eddc282399054c2ae8a121e432a74e75c3528b935984b7c242ed" }, "downloads": -1, "filename": "scrapy-mongoengine-item-0.1.5.tar.gz", "has_sig": false, "md5_digest": "2aeccd15c065a96beaa9407e9b2cc443", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4609, "upload_time": "2019-05-29T08:13:07", "url": "https://files.pythonhosted.org/packages/e3/13/16adb922920a15a6ee5acd06201920f407a95e299cb928e5d5d9f1aebae1/scrapy-mongoengine-item-0.1.5.tar.gz" } ] }