{
    "info": {
        "author": "Bobby",
        "author_email": "bobby@robot.studio",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 2 - Pre-Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
            "Natural Language :: English",
            "Programming Language :: Python :: 3.6"
        ],
        "description": ".. image:: https://travis-ci.org/RobotStudio/bors.svg?branch=master\n    :target: https://travis-ci.org/RobotStudio/bors\n\n.. image:: https://readthedocs.org/projects/bors/badge/?version=latest\n    :target: https://bors.readthedocs.io/en/latest/?badge=latest\n    :alt: Documentation Status\n\n.. image:: https://pyup.io/repos/github/RobotStudio/bors/shield.svg\n    :target: https://pyup.io/repos/github/RobotStudio/bors/\n    :alt: Updates\n\n.. image:: https://pyup.io/repos/github/RobotStudio/bors/python-3-shield.svg\n    :target: https://pyup.io/repos/github/RobotStudio/bors/\n    :alt: Python 3\n\n\nBors\n====\n\nA highly flexible and extensible service integration framework for\nscraping the web or consuming APIs.\n\nUsage\n=====\n\n1. Create your model based on the data you expect to incorporate.\n2. Decide on what you want to do with your data, and add it.\n3. Create or use an existing API integration library.\n4. Create your root application to tie it all together.\n\nObject Model\n------------\n\nWe use `marshmallow <https://marshmallow.readthedocs.io/en/latest/>`__\nfor the underlying object schema definitions. Here's an example model:\n\n.. code:: python\n\n    from marshmallow import Schema, fields\n\n    class NewsItemSchema(Schema):\n        \"\"\"News item\"\"\"\n        id = f.Str(required=True)\n        url = f.Str(required=True)\n        title = f.Str(required=True)\n        pubDate = f.Str(required=True)\n        timestamp = f.Str(required=True)\n        feed_id = f.Int(required=True)\n        published_date = f.Str(required=True)\n        feed_name = f.Str(required=True)\n        feed_url = f.Str(required=True)\n        feed_enabled = f.Int(required=True)\n        feed_description = f.Str(required=True)\n        url_field = f.Str(required=True)\n        title_field = f.Str(required=True)\n        date_field = f.Str(required=True)\n        feed_image = f.Str(required=True)\n\nSee the ``marshmallow`` docs for more information.\n\nMiddleware Strategies\n---------------------\n\nMiddleware API is implemented in the form of strategies and follows this\nbasic layout:\n\n.. code:: python\n\n    \"\"\"\n    Simple context display strategy\n    \"\"\"\n\n    from bors.app.strategy import IStrategy\n\n\n    class Print(IStrategy):\n        \"\"\"Print Strategy implementation\"\"\"\n        def bind(self, context):\n            \"\"\"\n            Bind the strategy to the middleware pipeline,\n            returning the context\n            \"\"\"\n            print(f\"\"\"PrintStrategy: {context}\"\"\")\n\n            # just a pass-through\n            return context\n\nThe important things to note here: \\* We're inheriting from\n``IStrategy``. \\* We're implementing a ``bind`` method. \\* The bind\nmethod receives, potentially augments, and then returns the ``context``.\n\nAPI Integration\n---------------\n\nRequest Schema\n~~~~~~~~~~~~~~\n\nBecause our API is simple, we're going to use this as-is.\n\n.. code:: python\n\n    from bors.generics.request import RequestSchema\n\nResponse Schema\n~~~~~~~~~~~~~~~\n\nOur API sends us data in the following format:\n\n.. code:: json\n\n    {\n        \"data\": ...,\n        \"status\": \"OK\"\n    }\n\nFor this, we'll need to supplement a bit, removing the root fields and\nreturning the ``data`` value:\n\n.. code:: python\n\n    from marshmallow import fields\n    from bors.generics.request import ResponseSchema\n\n\n    class MyAPIResponseSchema(ResponseSchema):\n        \"\"\"Schema defining how the API will respond\"\"\"\n        status = fields.Str()\n        def get_result(self, data):\n            \"\"\"Return the actual result data\"\"\"\n            return data.get(\"data\", \"\")\n            \n        class Meta:\n            \"\"\"Add 'data' field\"\"\"\n            strict = True\n            additional = (\"data\",)\n\nAPI Class\n~~~~~~~~~\n\n.. code:: python\n\n    from bors.api.requestor import Req\n\n\n    class MyAPI(LoggerMixin):\n        name = \"my_api\"\n        def __init__(self, context):\n            self.create_logger()\n            \n            self.request_schema = RequestSchema\n            self.result_schema = MyAPIResponseSchema\n            self.context = context\n            \n            self.req = Req(\"http://some.api.endpoint/v1\", payload, self.log)\n            \n            # We don't need to deal directly with requests, so we pass them through\n            self.call = self.req.call\n        \n        def shutdown(self):\n            \"\"\"Perform last-minute stuff\"\"\"\n            pass\n\nHere we use the built-in ``Req`` class to issue requests to the API, we\nassign the ``request_schema`` and ``result_schema`` to classes in our\nobject, and we set the ``name``, ``context``, and ``call`` attributes.\nThe results passed through on the API are referencable from within the\nmiddleware context under the key ``my_api``.\n\nPulling it all together\n~~~~~~~~~~~~~~~~~~~~~~~\n\n.. code:: python\n\n    from bors.app.builder import AppBuilder\n    from bors.app.strategy import Strategy\n\n\n    def main():\n        strat = Strategy(Print())\n        app = AppBuilder([MyAPI], strat)\n        app.run()\n        \n    if __name__ == \"__main__\":\n        main()\n\nHere, we set as many strategies and API's as we want, then create and\nrun the ``app``.\n\nArchitecture\n============\n\n::\n\n      +------------+\n    +-+ MIDDLEWARE +------> out\n    | +------------+\n    |                       API/WEB\n    | +------------+\n    +-+ PREPROCESS +<------ in\n      +------------+\n\nAt its most basic level, a ``bors`` integrator engages with an\nintegration library (API) passing incoming data through a prepocessor to\ngenerate and validate incoming objects, then passes that data through\nmiddlewares. Outgoing interactions are initiated from within a\nmiddleware and passed directly to an API, allowing easily for\nrequest/response type behavior in addition to observe and react.\n\nIngesting Data\n--------------\n\n::\n\n          ^\n          |\n    +-----+------+\n    | MIDDLEWARE |\n    +-----+------+\n          ^\n    +-----+------+\n    | PREPROCESS |\n    +-----+------+\n          ^\n          |\n          +\n         API/\n         WEB\n\nIngested data provokes calls along the pipeline.\n\nOutgoing Data\n-------------\n\n::\n\n         API/\n         WEB\n          ^\n          |\n    +-----+------+\n    | MIDDLEWARE |\n    +------------+\n\nEnacted events stimulate API or web actions.\n\nPreprocessing\n-------------\n\nPreprocessing is nothing more than an object-ization of the incoming\ndata. This provides two benefits: 1. Data can be generalized across API\ninterfaces. 2. Data structure can be validated and enforced.\n\nMiddlewares\n-----------\n\nMiddlewares allow for a data processing pipeline to pass data through.\n\n::\n\n      +-+  +-+  +-+\n      |M|  |M|  |M|\n      |I|  |I|  |I|\n      |D|  |D|  |D|\n      |D|  |D|  |D|\n    ->+L+->+L+->+L+->\n      |E|  |E|  |E|\n      |W|  |W|  |W|\n      |A|  |A|  |A|\n      |R|  |R|  |R|\n      |E|  |E|  |E|\n      +-+  +-+  +-+\n\nWith this model, we gain a lot of flexibility in the behavior of our\nintegration. Middleware is up to the developer to create, and can be any\nof the following:\n\n-  Data post-processing, filtering, aggregation, or augmentation\n-  External integrations and interfaces\n-  Stimulate an API/web transaction from external actors or time-based\n   criteria\n-  Hooks and callbacks\n\n\n\n\nHistory\n-------\n\n0.3.5 (2018-06-26)\n---------------------\n\n* Added badges.\n* Cleaned up deps.\n\n0.3.4 (2018-06-26)\n---------------------\n\n* Pruned and upgraded all dependencies.\n\n0.3.3 (2018-06-26)\n---------------------\n\n* Cleaned up tox.ini\n\n0.3.2 (2018-06-26)\n---------------------\n\n* Setup example.py script for others to use.\n\n0.3.1 (2018-06-26)\n---------------------\n\n* Added several unit tests, building out some of the framework.\n\n0.3.0 (2018-06-25)\n---------------------\n\n* Reboot packaging using cookiecutter-pypackage.\n\n0.2.0 (2018-05-17)\n---------------------\n\n* Initialize the repository, breaking it out from nombot.\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/RobotStudio/bors",
        "keywords": "web-scraper api-integrator scraping scraper data-integration data-ingestion bors",
        "license": "GNU General Public License v3",
        "maintainer": "",
        "maintainer_email": "",
        "name": "bors",
        "package_url": "https://pypi.org/project/bors/",
        "platform": "",
        "project_url": "https://pypi.org/project/bors/",
        "project_urls": {
            "Homepage": "https://github.com/RobotStudio/bors"
        },
        "release_url": "https://pypi.org/project/bors/0.3.6/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "A highly flexible and extensible service integration framework for scraping the web or consuming APIs",
        "version": "0.3.6"
    },
    "last_serial": 4365151,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "aa38473f71358bb273bf6b1a63035d41",
                    "sha256": "52280214f33eb9e2fc6eee428f337f0f372a717fd2216170bd379b278ac4f7a8"
                },
                "downloads": -1,
                "filename": "bors-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "aa38473f71358bb273bf6b1a63035d41",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14844,
                "upload_time": "2018-05-18T05:42:51",
                "url": "https://files.pythonhosted.org/packages/18/97/dfe3fe12ef1184da49309dceee4dd3557cfe61cdfd4952907161216b83d7/bors-0.0.1.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c7c00253fde899c79c113df5db5b5c41",
                    "sha256": "953e7816271ce5a47249163e7ff3d2065119768c63f9b1e1c23d2fe7b5d45427"
                },
                "downloads": -1,
                "filename": "bors-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "c7c00253fde899c79c113df5db5b5c41",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14398,
                "upload_time": "2018-06-19T01:12:01",
                "url": "https://files.pythonhosted.org/packages/01/0b/1cb6eff7c4cd6d0975a8fb62b7b004ecd3d196c500e5aad7ac3cd709ac32/bors-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "077d7af2e4d1edf5e4badedf0b7a105f",
                    "sha256": "697677f39c73202c726a760bc9cf565edb5125ba5fc4cf25e317c46759f1e439"
                },
                "downloads": -1,
                "filename": "bors-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "077d7af2e4d1edf5e4badedf0b7a105f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14398,
                "upload_time": "2018-06-22T18:34:23",
                "url": "https://files.pythonhosted.org/packages/4b/fc/d56698d9f6c9d632be19f889ea5882dd84371986fdb8f6ef4043459d87f1/bors-0.2.1.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5bf5c5eeb8369c256db8a551c258bbfb",
                    "sha256": "79afc7538d774aafc1e6699b8851782710127f81e96c31d9448ff20537873655"
                },
                "downloads": -1,
                "filename": "bors-0.3.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "5bf5c5eeb8369c256db8a551c258bbfb",
                "packagetype": "bdist_wheel",
                "python_version": "3.6",
                "requires_python": null,
                "size": 19433,
                "upload_time": "2018-06-25T22:55:44",
                "url": "https://files.pythonhosted.org/packages/fb/2d/18d7b24f8e06160451cf98162d516e69ec4b7604dd65e567a0ccb54d5df8/bors-0.3.0-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "62bbe68c03c9046b346384c5cc32195c",
                    "sha256": "8680a5e55003c69b21ee90ff8defb5acad52046e9abc6da29ef0ce97ad9a31a1"
                },
                "downloads": -1,
                "filename": "bors-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "62bbe68c03c9046b346384c5cc32195c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 34877,
                "upload_time": "2018-06-25T22:55:33",
                "url": "https://files.pythonhosted.org/packages/59/6d/5393831b11e95eca4cc657e9481fdc9d4676ef0d02fcba6b88d346012e7d/bors-0.3.0.tar.gz"
            }
        ],
        "0.3.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1380fbbc00ae5f870648584cfb27ba99",
                    "sha256": "6cfd7d77adf33ee2284677e9e661c7c31463cfdcdc50daca550aaf8cae687e4b"
                },
                "downloads": -1,
                "filename": "bors-0.3.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1380fbbc00ae5f870648584cfb27ba99",
                "packagetype": "bdist_wheel",
                "python_version": "3.6",
                "requires_python": null,
                "size": 19499,
                "upload_time": "2018-06-26T20:27:55",
                "url": "https://files.pythonhosted.org/packages/df/a5/8abef5fe968c074c01ce0cee20aaf290d8da627af2e2b50b8679ab7cece6/bors-0.3.3-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "9f4fc7c482a4b0ef638c88dd68bd6d01",
                    "sha256": "63e573b8485402875d9f1fc5d986bdd78cdd05347d3632bc617385f68e325d3a"
                },
                "downloads": -1,
                "filename": "bors-0.3.3.tar.gz",
                "has_sig": false,
                "md5_digest": "9f4fc7c482a4b0ef638c88dd68bd6d01",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 35703,
                "upload_time": "2018-06-26T20:27:49",
                "url": "https://files.pythonhosted.org/packages/a9/34/b5cf5215540b83352f54483db6161e5e0a4d8b89ad066679ef5d729cf940/bors-0.3.3.tar.gz"
            }
        ],
        "0.3.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7a99888c101e142005488eb22c17c8e0",
                    "sha256": "758a14fff7985d5f4e2528e26263221e8111c9845242727f3af4101a1918fad9"
                },
                "downloads": -1,
                "filename": "bors-0.3.4-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7a99888c101e142005488eb22c17c8e0",
                "packagetype": "bdist_wheel",
                "python_version": "3.6",
                "requires_python": null,
                "size": 19529,
                "upload_time": "2018-06-26T21:03:12",
                "url": "https://files.pythonhosted.org/packages/d1/38/38a06106e80b4c6163e07e6ec14cbf849babd8f6f489e171c352e591ffcf/bors-0.3.4-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e0c34fb126e927ca4b28d11d29e9d132",
                    "sha256": "305fc5267cc2a8d4cfc4747bebbf0cf29a240152ed5bf0c24117ab7e72b704b3"
                },
                "downloads": -1,
                "filename": "bors-0.3.4.tar.gz",
                "has_sig": false,
                "md5_digest": "e0c34fb126e927ca4b28d11d29e9d132",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 35731,
                "upload_time": "2018-06-26T21:03:07",
                "url": "https://files.pythonhosted.org/packages/06/48/37c3e660c4f0c185b5d30ce538c47ee1ffa49c3631915e837ef4b5ce3acf/bors-0.3.4.tar.gz"
            }
        ],
        "0.3.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8f4d390097a9cbf846ad5a54f78e129d",
                    "sha256": "1b4dd4777801e5c26a3b226c8754e502907a630a56279dcd5fe8d73570ee5f7e"
                },
                "downloads": -1,
                "filename": "bors-0.3.5-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "8f4d390097a9cbf846ad5a54f78e129d",
                "packagetype": "bdist_wheel",
                "python_version": "3.6",
                "requires_python": null,
                "size": 19688,
                "upload_time": "2018-06-26T21:32:25",
                "url": "https://files.pythonhosted.org/packages/a6/7d/5c853ae714b9e942e1600e0ac777748e740eef33cb2ac8d2a28d20ccc5c6/bors-0.3.5-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "301f6335ea381d033260d495fe2bed01",
                    "sha256": "9fd32fb081c8ccf2ae30a31bc07b8d542cfc0f3daba846e3157af23bd56796a6"
                },
                "downloads": -1,
                "filename": "bors-0.3.5.tar.gz",
                "has_sig": false,
                "md5_digest": "301f6335ea381d033260d495fe2bed01",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 36049,
                "upload_time": "2018-06-26T21:32:20",
                "url": "https://files.pythonhosted.org/packages/eb/c1/9cf04ec522915212b515b3a3565e7d7b9ca3a0a3efd20c3d1b0a120f8984/bors-0.3.5.tar.gz"
            }
        ],
        "0.3.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9d920be42257af232f73043592be2b51",
                    "sha256": "53d306204f3d720e8e676e9e3fd9ccda3f9e79fd0861a263a9da00928fc524bc"
                },
                "downloads": -1,
                "filename": "bors-0.3.6.tar.gz",
                "has_sig": false,
                "md5_digest": "9d920be42257af232f73043592be2b51",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 36054,
                "upload_time": "2018-10-11T16:50:17",
                "url": "https://files.pythonhosted.org/packages/82/1e/c169201c97a0b4404e7a47ac5cb1094f2184980fdb5836a0f5c2f43172a1/bors-0.3.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "9d920be42257af232f73043592be2b51",
                "sha256": "53d306204f3d720e8e676e9e3fd9ccda3f9e79fd0861a263a9da00928fc524bc"
            },
            "downloads": -1,
            "filename": "bors-0.3.6.tar.gz",
            "has_sig": false,
            "md5_digest": "9d920be42257af232f73043592be2b51",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 36054,
            "upload_time": "2018-10-11T16:50:17",
            "url": "https://files.pythonhosted.org/packages/82/1e/c169201c97a0b4404e7a47ac5cb1094f2184980fdb5836a0f5c2f43172a1/bors-0.3.6.tar.gz"
        }
    ]
}