{ "info": { "author": "Dustin Oprea", "author_email": "myselfasunder@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "----------------------\nArchitectural Overview\n----------------------\n\nA bidirectional pipe is established from N-clients to a central server. The \npipes will aggressively try to remain connected (even after prolonged failure) \nand will allow all nodes to transmit events (queries) to each other via a \nlocal web-server on each. The pipe is SSL-encrypted and -authenticated.\n\n\n--------------\nUsage Overview\n--------------\n\nA request is made from a script, application, etc.. on the server or a client\nnode to the local RestPipe webserver. The webserver converts the REST request\nto a wire-level message ('event') containing the noun, verb, post-data, and \ncontent-type. If the event is sent from the server, then a hostname for the\nclient must be included at the front of the URL path. This hostname will be\nused to lookup the connection.\n\nThe message exchange of the receiving node will receive this message, and \npassed to the message-loop. The message-loop will derive a method on the\nconfigured event-handler class to be used to handle the event. The method will\neither return a 3-tuple of mime-type, return-code, and data, or just straight \ndata (with a default mime-type of \"application/json\" and code of (0)). If the \nmime-type is JSON, the data will be encoded automatically.\n\nThe mime-type, return-code, and data will then be returned as an event back \nover the pipe. The message-exchange will receive this, see that it's a reply,\nstore it as such, and signal the original web-request that a reply has been\nreceived. The reply is collected by the web-request, and then returned with\nthe content-type and data from the reply.\n\n\nEvent Handling\n==============\n\nThe method on the event-handler is derived from the noun and verb from the web-\nrequest. For example, a *GET* request is placed to the noun \"time\". This will\ncall a method named \"get_time\". If you were to call a noun named \"time/utc\",\nthe handler would be named \"get_time_utc\". The method will receive at least two\narguments: connection context information, and post-data. The \"post-data\" will \nbe a 2-tuple of mime-type and actual data. They will both be empty strings if \nnot relevant.\n\nThere is also support for arguments to be passed by URL (like with normal REST\nrequests, passed via URL or query parameters). The arguments are a single-\nslash-delimited list of strings separated from the noun using a double-slash. \nFor example, if \"/cat//hello%20/world\" is the path and *GET* is the verb, then\nthe handler will be \"get_cat\" and the parameters will be context, post-data, \n\"hello \", and \"world\".\n\n\n------------\nRequirements\n------------\n\nClient and server nodes are functionally equivalent. The only difference are \nthe following:\n\n1. Clients establish and maintain connections\n2. Clients always talk to one specific server, but in order for a server to \n send events to a client, it has to have a hostname or IP address of that \n client.\n\nThe general idea is that the client nodes may need access to data that's only \naccessible to the server node, and the server node might need access to \nsomething that's only accessible to one of the client nodes. Therefore, the \nwebsite, backend application, or other miscellaneous processes that you have \nrunning on a particular node can use the local RestPipe interface to send an \nevent to get what it needs. \n\nWe recommend Nginx as a webserver to receive and forward the local requests to\nRP. If a machine need not originate any requests, that instance of RP does not \nneed a webserver.\n\n\n----------------\nExample Use-Case\n----------------\n\nA public-facing webserver needs access to the content of a file-server that \ndoes not allow any inbound connections. You will run RestPipe (RP) as a client \non the file-server and RP as a server on the webserver. The RP client on the \nfile-server will maintain a connection with the RP server on the webserver.\n\nThe application on the webserver will make web-requests to the RP server on\nthe local machine, these requests will be translated to events and sent to the\nRP-client on the fileserver via the secure pipe, the event will be dispatched \nto the event-handler, the event-handler will find/read the requested file and \nreturn the data, the RP-server will receive the response, the web-request will \nreturn the response.\n\n\n------------\nTechnologies\n------------\n\n- web.py\n- gevent\n- Protocol Buffers\n- SSL authentication\n\n\n----------------\nDesign Decisions\n----------------\n\nIt is expected that events and their responses are reasonably sized. As the \nHTTP requests and the event responses are being translated into contiguous, \ndiscrete messages and sent over a socket, there are no elegant ways to handle \nlarge events. If you need to, than use RestPipe only as a signaling solution,\nand have the handlers stage the data into a secondary location (like S3 for \nlarge files, if you're working with AWS).\n\n\n---------------\nGetting Started\n---------------\n\nThis is a walkthrough of how to get an RP server and client running on a \ndevelopment machine. When it comes to moving to production, the following \nthings will probably change (aside from the client and server being on separate \nmachines):\n\n- The webservers' virtualhost hostnames.\n- The RP server and client webserver ports/bindings.\n- The RP server socket-server port/binding.\n- Use a corporate certificate authority to generate official server and client \n identities.\n- Running Gunicorn in production mode (it's started in development mode, \n below).\n- Customized event-handlers.\n\n\nEstablishing SSL Identities\n===========================\n\nWe're going to use `CaKit `_ to establish \nkeys and certificates. You may use any method that you prefer.\n\n1. Extract the CaKit project in order to easily generate keys::\n\n $ sudo git clone https://github.com/dsoprea/CaKit.git ca_kit\n Cloning into 'ca_kit'...\n remote: Counting objects: 38, done.\n remote: Compressing objects: 100% (23/23), done.\n remote: Total 38 (delta 19), reused 31 (delta 12)\n Unpacking objects: 100% (38/38), done.\n Checking connectivity... done.\n\n2. Build identities::\n\n $ cd ca_kit/\n $ sudo ./create_ca.py\n $ sudo ./create.py -n server\n $ sudo ./sign.py -n server\n $ sudo ./create.py -n client\n $ sudo ./sign.py -n client\n $ ls -1 output/\n ca.crt.pem\n ca.csr.pem\n ca.key.pem\n ca.public.pem\n client.crt.pem\n client.csr.pem\n client.key.pem\n client.public.pem\n server.crt.pem\n server.csr.pem\n server.key.pem\n server.public.pem\n\n\nConfigure Nginx\n===============\n\n1. Define *rpclient.local* and *rpserver.local* in your */etc/hosts* file as *127.0.0.1*.\n2. Added example Nginx configs::\n\n upstream rp_client {\n server unix:/tmp/rpclient.gunicorn.sock fail_timeout=0;\n }\n\n server {\n server_name rpclient.local;\n keepalive_timeout 5;\n\n location /favicon.ico {\n return 404;\n }\n\n location / {\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n proxy_set_header Host $http_host;\n proxy_redirect off;\n\n proxy_pass http://rp_client;\n }\n }\n\n upstream rp_server {\n server unix:/tmp/rpserver.gunicorn.sock fail_timeout=0;\n }\n\n server {\n server_name rpserver.local;\n keepalive_timeout 5;\n\n location /favicon.ico {\n return 404;\n }\n\n location / {\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n proxy_set_header Host $http_host;\n proxy_redirect off;\n\n proxy_pass http://rp_server;\n }\n }\n\n\nInstalling RestPipe\n===================\n\n1. Install RestPipe::\n\n $ sudo pip install restpipe\n\n2. Load identities::\n\n $ cd /usr/local/ca_kit/output/\n $ sudo rp_server_set_identity server.key.pem server.crt.pem ca.crt.pem \n $ sudo rp_client_set_identity client.key.pem client.crt.pem\n\n3. Start the RestPipe server::\n\n $ rp_server_start_gunicorn_dev \n\n4. Start the RestPipe client (in another window)::\n\n $ rp_client_start_gunicorn_dev \n\nThe server and the client can actually be started in any order. Also, just as\nthe scripts above are meant to development (notice the \"dev\" suffix), there are\nproduction versions as well.\n\nAt this point, you have a pipe between a single server and a single client. \nThere's not a whole lot of verbosity by default, but you can see the \nunderlying mechanics if the environment variable \"DEBUG\" is set to \"1\".\n\n\nExample Events\n==============\n\nObviously, you're responsible for implementing any event-handlers that you \nmight need. However, there are two event handlers defined by default, as an\nexample, on both the server side and client side. The commands and responses\nbelow correlate to the example Nginx configs, above.\n\n- *time* (*GET*)\n\n From client::\n\n $ curl http://rpclient.local/server/time && echo\n {\"time_from_server\": 1402897823.882672}\n\n From server::\n\n $ curl http://rpserver.local/client/localhost/time && echo\n {\"time_from_client\": 1402897843.879908}\n\n- *cat* (*GET*)\n\n From client:: \n\n $ curl http://rpclient.local/server/cat//hello%20/world && echo\n {\"result_from_server\": \"hello world\"}\n\n From server::\n\n $ curl http://rpserver.local/client/localhost/cat//hello%20/world && echo\n {\"result_from_client\": \"hello world\"}\n\n\n-------------\nCustomization\n-------------\n\nTo set the server hostname and port for the client, set the \n`RP_CLIENT_TARGET_HOSTNAME` and `RP_CLIENT_TARGET_PORT` environment variables.\n\nThe set the interface binding on the server, set the *BIND_IP* and *BIND_PORT*\nenvironment variables.\n\nWhen you're ready to implement your own event-handler, start your own project, \nwrite your module, make sure it inherits properly, and set the right \nenvironment variable with the fully-qualified name of your module.\n\nIf you're writing a server event-handler, make sure it inherits from \n*rpipe.server.connection.ServerEventHandler*, and set the fully-qualified module \nname as the `RP_EVENT_HANDLER_FQ_CLASS` environment variable. If you're writing a \nclient event-handler, use the *ClientEventHandle* class from the same package \nand the `RP_EVENT_HANDLER_FQ_CLASS` environment variable.\n\nMany of the configurables can be overriden via environment variables. If you \nneed to override more than a handful of values, you might prefer to set any \nnumber of values in your own module, and then set the fully-qualified name of \nthe module into the `RP_CLIENT_USER_CONFIG_MODULE` or \n`RP_SERVER_USER_CONFIG_MODULE` environment variable(s). All of the values from \nyour module will overwrite the defaults.\n\nYou may also inherit from `rpipe.connection_state_events.ConnectionStateEvents`\nand override the `connection_success` and `connection_fail` methods.\n\n\n--------------\nError Handling\n--------------\n\nWhen an uncaught exception occurs on the side of the pipe that is handling an \nevent, it will be captured and forwarded via the HTTP body with a non-zero \nreturn-code (which is set into the `X-Event-Return-Code` response header)::\n\n {\n \"exception\": {\n \"message\": \"\",\n \"traceback\": \"\",\n \"class\": \"\",\n }\n }\n\nIf you're using the `requests `_ \nclient, you can call `rpipe.event_response.raise_for_exception` with the \nresponse, and, if there was an error, it'll build a PipeFailError exception \nwith the information from the response and raise it.\n\n\n----------\nStatistics\n----------\n\nRestPipe will emit `statsd `_ events to \n*localhost:8125* by default. To override this, set the `RP_STATSD_HOST` and\n`RP_STATSD_PORT` environment variables. To disable this, set them to empty.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/dsoprea/RestPipe", "keywords": "gevent ssl socket rest event", "license": "GPL 2", "maintainer": null, "maintainer_email": null, "name": "restpipe", "package_url": "https://pypi.org/project/restpipe/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/restpipe/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/dsoprea/RestPipe" }, "release_url": "https://pypi.org/project/restpipe/0.2.20/", "requires_dist": null, "requires_python": null, "summary": "An SSL-authenticated, durable, bidirectional, RESTful, client-server pipe that transports custom events.", "version": "0.2.20" }, "last_serial": 1694888, "releases": { "0.2.10": [ { "comment_text": "", "digests": { "md5": "c2dd327bbf858453bc26b410df187659", "sha256": "ae80b08b4a572ad24cd37b5eaf0f68e6bd44f57e3c61af30176c11d4ae30a3a7" }, "downloads": -1, "filename": "restpipe-0.2.10-py2-none-any.whl", "has_sig": false, "md5_digest": "c2dd327bbf858453bc26b410df187659", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 55343, "upload_time": "2014-07-21T17:44:59", "url": "https://files.pythonhosted.org/packages/06/e0/671f63b17600cd63568d228a3142c2760acd6132edfea93a76011591ad6f/restpipe-0.2.10-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c03356a303a70851808d8fc4c7843877", "sha256": "20981b2f9ad5e0fdcf7b958978b208a1f3fa88e139a24c85d90bbd00cb5b1296" }, "downloads": -1, "filename": "restpipe-0.2.10.tar.gz", "has_sig": false, "md5_digest": "c03356a303a70851808d8fc4c7843877", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1136671, "upload_time": "2014-07-21T17:44:57", "url": "https://files.pythonhosted.org/packages/5a/8a/cb1d5b46c7a6295e993ff71473179a10a4ed5a27e69046c6568f559b753e/restpipe-0.2.10.tar.gz" } ], "0.2.11": [ { "comment_text": "", "digests": { "md5": "039e101552441dede7cc36d7a3fdb62e", "sha256": "60a2cbae2e9a3cb7817ad25cc606ba460fa4c92ae209c05f2df125db1d18dd0c" }, "downloads": -1, "filename": "restpipe-0.2.11-py2-none-any.whl", "has_sig": false, "md5_digest": "039e101552441dede7cc36d7a3fdb62e", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 56102, "upload_time": "2014-08-03T05:54:20", "url": "https://files.pythonhosted.org/packages/95/a3/5634c46346424ed8acd32b579be6fef2742a868af571eeb6f3e10f82aa19/restpipe-0.2.11-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "13bbbf051a901113fb53d4087c52c867", "sha256": "9e5a1ef7a410f8b6f5fb85f29e1cc8800ab572867007be78b341e59921a093a2" }, "downloads": -1, "filename": "restpipe-0.2.11.tar.gz", "has_sig": false, "md5_digest": "13bbbf051a901113fb53d4087c52c867", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1137420, "upload_time": "2014-08-03T05:54:17", "url": "https://files.pythonhosted.org/packages/ca/ec/6bddd5460b5c810687c15578887939d9ab5dd42a6143ac99200cf662e255/restpipe-0.2.11.tar.gz" } ], "0.2.12": [ { "comment_text": "", "digests": { "md5": "9b0f00d6d04b2f3b66ca77f5b76ceec1", "sha256": "a54f65a6f3cdffb4c0f4c945bf058cc7ae6012c9918c4c5a263a911a026dd0ff" }, "downloads": -1, "filename": "restpipe-0.2.12.tar.gz", "has_sig": false, "md5_digest": "9b0f00d6d04b2f3b66ca77f5b76ceec1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1137862, "upload_time": "2015-03-29T01:48:27", "url": "https://files.pythonhosted.org/packages/42/50/d0023b525bb2a2cedd651c4a92baf174b5d49d2cb0f0057cc0910e1d62e6/restpipe-0.2.12.tar.gz" } ], "0.2.13": [ { "comment_text": "", "digests": { "md5": "5d8312489a2150294fa3867c765eb3ba", "sha256": "3e3a70a933e11c4a3583abd5b0e45a09529be758c1813b095612e267c1405e05" }, "downloads": -1, "filename": "restpipe-0.2.13.tar.gz", "has_sig": false, "md5_digest": "5d8312489a2150294fa3867c765eb3ba", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1137853, "upload_time": "2015-03-29T04:48:31", "url": "https://files.pythonhosted.org/packages/07/2d/ab07399f2b268c5a9c6d1b61f5d530e721f0bac8bf0acfc2c34188c29d97/restpipe-0.2.13.tar.gz" } ], "0.2.14": [ { "comment_text": "", "digests": { "md5": "94b808e66c26479923a4fd21180e5acf", "sha256": "6704dfda74ebd9866ab1fb12b5b9c7e42fe3737005ce0a0416f84e1a7ea86021" }, "downloads": -1, "filename": "restpipe-0.2.14.tar.gz", "has_sig": false, "md5_digest": "94b808e66c26479923a4fd21180e5acf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1138544, "upload_time": "2015-03-29T22:58:47", "url": "https://files.pythonhosted.org/packages/2f/03/5c6372489267a5f7ff090ea616a287034902d799d55bdae354fb0e3b1784/restpipe-0.2.14.tar.gz" } ], "0.2.15": [ { "comment_text": "", "digests": { "md5": "27523c3eb316a62bb67d55828c464a07", "sha256": "8aac60282594db42d5e16edd9bfcbbf4d99a0563c050e479ee5d633706afdfdf" }, "downloads": -1, "filename": "restpipe-0.2.15.tar.gz", "has_sig": false, "md5_digest": "27523c3eb316a62bb67d55828c464a07", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1138556, "upload_time": "2015-03-29T23:14:41", "url": "https://files.pythonhosted.org/packages/3c/32/f82194ec18398b596ae90d849b3fabed8389afd0df62722442e1f848f38a/restpipe-0.2.15.tar.gz" } ], "0.2.16": [ { "comment_text": "", "digests": { "md5": "244dbfb9ff723e20afd41ff55b3a139c", "sha256": "ec37d3a4ecade312e9e9f046dd2b8c212697b404e71a55cc6fd519f0a26f8f7a" }, "downloads": -1, "filename": "restpipe-0.2.16.tar.gz", "has_sig": false, "md5_digest": "244dbfb9ff723e20afd41ff55b3a139c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1138673, "upload_time": "2015-03-30T17:11:46", "url": "https://files.pythonhosted.org/packages/f8/5c/2369ae956ab214889660a386d4aa20b5e0317e3941a82b771d2b7caba879/restpipe-0.2.16.tar.gz" } ], "0.2.17": [ { "comment_text": "", "digests": { "md5": "e23b04c3814aba27dccd9c8346913637", "sha256": "8376556b5b96d832b8db79e65546e98f27f9b1910c3922f1e2bd48627ab4b896" }, "downloads": -1, "filename": "restpipe-0.2.17.tar.gz", "has_sig": false, "md5_digest": "e23b04c3814aba27dccd9c8346913637", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1138696, "upload_time": "2015-04-07T03:50:13", "url": "https://files.pythonhosted.org/packages/b3/d1/5a82184a43890ccf8b479f49fbf07019e40f6c6ff63e385b14a6b11e46be/restpipe-0.2.17.tar.gz" } ], "0.2.18": [ { "comment_text": "", "digests": { "md5": "23aea100b3afe9c05895d5026c88bd27", "sha256": "5f0023a1a55ac00cea97f6b11e03e7a6eeb4151c599c501986a56a9f7ec97a0e" }, "downloads": -1, "filename": "restpipe-0.2.18.tar.gz", "has_sig": false, "md5_digest": "23aea100b3afe9c05895d5026c88bd27", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1161077, "upload_time": "2015-06-30T05:58:08", "url": "https://files.pythonhosted.org/packages/ff/4c/3edc3e09c0c13a1b56088b66b03f98d8fcb3f203bd4988051f9a215e872c/restpipe-0.2.18.tar.gz" } ], "0.2.19": [], "0.2.20": [ { "comment_text": "", "digests": { "md5": "28bd708a351aa70e4caef356c1769174", "sha256": "58392d49461a7333ab7d23d528c180dcfc362cb50f350f642fde7b9df86087b3" }, "downloads": -1, "filename": "restpipe-0.2.20.tar.gz", "has_sig": false, "md5_digest": "28bd708a351aa70e4caef356c1769174", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1161597, "upload_time": "2015-08-26T16:43:10", "url": "https://files.pythonhosted.org/packages/d2/40/1d9e951f369f5feead47596b2467666c23bed65884321ee73e4b916ca6ab/restpipe-0.2.20.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "28bd708a351aa70e4caef356c1769174", "sha256": "58392d49461a7333ab7d23d528c180dcfc362cb50f350f642fde7b9df86087b3" }, "downloads": -1, "filename": "restpipe-0.2.20.tar.gz", "has_sig": false, "md5_digest": "28bd708a351aa70e4caef356c1769174", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1161597, "upload_time": "2015-08-26T16:43:10", "url": "https://files.pythonhosted.org/packages/d2/40/1d9e951f369f5feead47596b2467666c23bed65884321ee73e4b916ca6ab/restpipe-0.2.20.tar.gz" } ] }