{ "info": { "author": "Matt Yule-Bennett", "author_email": "", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "Operating System :: MacOS :: MacOS X", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Internet", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# nameko-grpc\n\nThis is a prototype implementation of a GRPC server and client for use in [nameko](https://nameko.io) microservices.\n\nAll four of the request-response patterns are implemented and tested:\n\n1. unary-unary\n2. unary-stream\n3. stream-unary\n4. stream-stream\n\nAsynchronous calls are also supported for every pattern.\n\nPython 3.4+ is supported.\n\n## Installation\n\n```\n$ pip install nameko-grpc\n```\n\n## Example\n\n### Server\n\nExample Nameko service that can respond to GRPC requests:\n\n``` python\nfrom example_pb2 import ExampleReply\nfrom example_pb2_grpc import exampleStub\n\nfrom nameko_grpc.entrypoint import Grpc\n\ngrpc = Grpc.implementing(exampleStub)\n\n\nclass ExampleService:\n name = \"example\"\n\n @grpc\n def unary_unary(self, request, context):\n message = request.value * (request.multiplier or 1)\n return ExampleReply(message=message)\n\n @grpc\n def unary_stream(self, request, context):\n message = request.value * (request.multiplier or 1)\n yield ExampleReply(message=message, seqno=1)\n yield ExampleReply(message=message, seqno=2)\n\n @grpc\n def stream_unary(self, request, context):\n messages = []\n for req in request:\n message = req.value * (req.multiplier or 1)\n messages.append(message)\n\n return ExampleReply(message=\",\".join(messages))\n\n @grpc\n def stream_stream(self, request, context):\n for index, req in enumerate(request):\n message = req.value * (req.multiplier or 1)\n yield ExampleReply(message=message, seqno=index + 1)\n\n```\n\n### Client\n\nExample Nameko service that can make GRPC requests:\n\n``` python\nfrom example_pb2 import ExampleReply\nfrom example_pb2_grpc import exampleStub\n\nfrom nameko.rpc import rpc\n\nfrom nameko_grpc.dependency_provider import GrpcProxy\n\n\nclass ClientService:\n name = \"client\"\n\n example_grpc = GrpcProxy(\"//127.0.0.1\", exampleStub)\n\n @rpc\n def method(self):\n responses = self.example_grpc.unary_stream(ExampleRequest(value=\"A\"))\n for response in responses:\n print(response.message)\n\n```\n\nExample standalone client, can be used with or without Eventlet:\n\n``` python\nfrom example_pb2 import ExampleReply\nfrom example_pb2_grpc import exampleStub\n\nfrom nameko_grpc.client import Client\n\nwith Client(\"//127.0.0.1\", exampleStub) as client:\n responses = client.unary_stream(ExampleRequest(value=\"A\"))\n for response in responses:\n print(response.message)\n\n```\n\n### Protobuf\n\nThe protobuf for the above examples is:\n\n```\nsyntax = \"proto3\";\n\npackage nameko;\n\nservice example {\n rpc unary_unary (ExampleRequest) returns (ExampleReply) {}\n rpc unary_stream (ExampleRequest) returns (stream ExampleReply) {}\n rpc stream_unary (stream ExampleRequest) returns (ExampleReply) {}\n rpc stream_stream (stream ExampleRequest) returns (stream ExampleReply) {}\n}\n\nmessage ExampleRequest {\n string value = 1;\n int32 multiplier = 2;\n}\n\n\nmessage ExampleReply {\n string message = 1;\n int32 seqno = 2;\n}\n\n```\n\n## Style\n\nThe example protobufs in this repo use `snake_case` for method names as per the Nameko conventions rather than `CamelCase` as per GRPC. This is not mandatory -- decorated method names simply match to the methods defined in the protobufs; similarly for service names.\n\n## Context and Metadata\n\nInsofar as it is implemented, the `context` argument to service methods has the same API as the standard Python implementation:\n\n* `context.invocation_metadata()` returns any metadata provided by the calling client.\n* `context.send_initial_metadata()` can be used to add metadata to the response headers.\n* `context.set_trailing_metadata()` can be used to add metadata to the response trailers.\n\nThe standalone Client and DependencyProvider both allow metadata to be provided using the `metadata` keyword argument. They accept a list of `(name, value)` tuples, just as the standard Python client does. Binary values must be base64 encoded and use a header name postfixed with \"-bin\", as in the standard Python client.\n\nGRPC request metadata is added to the \"context data\" of the Nameko worker context, so is availble to other Nameko extensions.\n\nThe DependencyProvider client adds Nameko worker context data as metadata to all GRPC requests. This allows the Nameko call id stack to be populated and propagate, along with any other context data.\n\n## Compression\n\nCompression is supported in both the server and the client. The `deflate` and `gzip` algorithms are available by default and will be included in the `grpc-accept-encoding` headers on requests from the client and responses from the server.\n\nThe server honours any acceptable compression algorithm that it is able to, preferring to encode the response with the same algorithm as the request.\n\nA default compression algorithm is specified when creating the client, and/or can specified per-call using the `compression` keyword argument:\n\n``` python\nclient = Client(default_compression=\"deflate\", ...)\nclient.unary_unary(ExampleRequest(value=\"foo\"), compression=\"gzip\") # use gzip instead\n```\n\nCompression levels are not supported.\n\nThe GRPC spec allows for the server to respond using a different algorithm from the request, or not compressing at all. This is not currently supported in the standard Python GRPC implementation nor nameko-grpc.\n\n\n## Errors\n\nGRPC errors are raised by the client as instances of the `GrpcError` exception class. Similar to the `grpc.RpcError` class defined in the standard Python GRPC client, a `GrpcError` encapsulates the [status code](https://github.com/grpc/grpc/blob/master/doc/statuscodes.md) and a `details` string describing the error.\n\n\n## Timeouts\n\nThe client and server both support timeouts, and will raise `DEADLINE_EXCEEDED` if an RPC has not completed within the requested time. The clock starts ticking on the client when the request is initiated, and on the server when it is received.\n\nThe deadline is calculated as the current time plus the timeout value.\n\nOn the client, the timeout value is specified in seconds by using the `timeout` keyword argument when invoking a method:\n\n``` python\nclient = Client(...)\nclient.unary_unary(ExampleRequest(value=\"foo\"), timeout=0.1) # 100 ms timeout\n```\n\nThere is no default because there's no sensible value applicable to all use-cases, but it is [recommended](https://grpc.io/blog/deadlines) to always set a deadline.\n\n## Tests\n\nMost tests are run against every permutation of GRPC server/client to Nameko server/client. This roughly demonstrates equivalence between the two implementations. These tests are marked with the \"equivalence\" pytest marker.\n\nAdditionally, we run the interop tests from the official GRPC repo, which are used to verify compatibility between language implementations. The Nameko GRPC implementation supports every feature that the official Python GRPC implementation does. These tests are marked with the \"interop\" pytest marker.\n\nThe `test/spec` directory contains the protobufs and server implementations used in the various tests.\n\n### Running the tests\n\nClone or download the repository, and ensure the development dependencies are installed:\n\n```\n$ pip install nameko-grpc[dev]\n```\n\nThen run the tests:\n\n```\n$ pytest test\n```\n\nThe interop tests require docker. They use the image at https://hub.docker.com/r/nameko/nameko-grpc-interop which contains the pre-built C++ interop client. To run all tests excluding the interop tests:\n\n```\n$ pytest test -m \"not interop\"\n```\n\n\n## Implementation Notes\n\nGRPC is built on HTTP2, so nameko-grpc relies heavily on the [hyper-h2](https://python-hyper.org/projects/h2/en/stable/) library. H2 is a finite state-machine implementation of the HTTP2 protocol, and its documentation is very good. The code in nameko-grpc is much more understandable when you're familiar with h2.\n\nMuch of the heavy-lifting in nameko-grpc is done by either the server or client subclasses of `ConnectionManager`. A `ConnectionManager` handles a single HTTP2 connection, and implements the handlers for each HTTP2 event on that connection (e.g. `request_received` or `stream_ended`). See:\n\n* `nameko_grpc/client.py::ClientConnectionManager`\n* `nameko_grpc/entrypoint.py::ServerConnectionManager`\n* `nameko_grpc/connection.py::ConnectionManager`\n\nThe next most significant module is `nameko_grpc/streams.py`. This module contains the `SendStream` and `ReceiveStream` classes, which represent an HTTP2 stream that is being sent or received, respectively. A `ReceiveStream` receives data as bytes from a `ConnectionManager`, and parses them into a stream of GRPC messages. A `SendStream` does the opposite, encoding GRPC messages into bytes that can be sent across an HTTP2 connection.\n\nThe `@grpc` Entrypoint is a normal Nameko entrypoint that executes a service method when an appropriate request is made. The entrypoint deals with a `ReceiveStream` object encapsulating the request, and a `SendStream` object that accepts the response. The streams are managed by a shared `GrpcServer`, which accepts incoming connections and wraps each in a `ServerConnectionManager`.\n\nThe standalone Client is a small wrapper around a `ClientConnectionManager`. The Client simply creates a socket connection and then hands it to the connection manager. When a method is invoked on the client, the connection manager initiates an appropriate request. The headers for that request describe the method being invoked, encodings, message types etc. This logic is all encapsulated into the `Method` class.\n\nThe GRPC DependencyProvider is a normal Nameko DependencyProvider, which is also just a small wrapper around a `ClientConnectionManager`. It functions in exactly the same manner as the standalone Client.\n\n\n## Equivalence tests notes\n\nTo demonstrate equivalence between the nameko-grpc implementations and the standard GRPC implementations, all tests marked with the `equivalence` marker run against every permutation of:\n\n* GRPC standard server (Python implementation) or\n* Nameko server\n\nand\n\n* GRPC standard client (Python implementation) or\n* Nameko standalone client or\n* Nameko DependencyProvider client\n\nNameko uses Eventlet for concurrency, which is incompatible with the standard GRPC server and client. Consequently, these must be run in a separate process and somehow communicated with in order to make assertions about the behaviour of the standard implementation.\n\nThe scripts which run the out-of-process client and server can be found in `test/grpc_indirect_client.py` and `test/grpc_indirect_server.py`\n\nThe communication is done with ZeroMQ. The logic for this is contained within the `RemoteClientTransport` and `Command` classes within `test/helpers.py`, and the `start_grpc_client` and `start_grpc_server` fixtures in `test/conftest.py`.\n\nIn the future this arrangement would allow us to run equivalence tests against a different (more feature-complete) standard GRPC implementation.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/nameko/nameko-grpc", "keywords": "", "license": "Apache License, Version 2.0", "maintainer": "", "maintainer_email": "", "name": "nameko-grpc", "package_url": "https://pypi.org/project/nameko-grpc/", "platform": "", "project_url": "https://pypi.org/project/nameko-grpc/", "project_urls": { "Homepage": "http://github.com/nameko/nameko-grpc" }, "release_url": "https://pypi.org/project/nameko-grpc/1.0.1/", "requires_dist": [ "nameko", "h2 (>=3)", "grpcio", "googleapis-common-protos", "pytest ; extra == 'dev'", "grpcio-tools ; extra == 'dev'", "pre-commit ; extra == 'dev'", "wrapt ; extra == 'dev'", "zmq ; extra == 'dev'" ], "requires_python": "", "summary": "Nameko GRPC extensions", "version": "1.0.1" }, "last_serial": 4866601, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "46b740583b737e7e1e94285ee87d4476", "sha256": "09a91ee4d66dfb99a27acf93d8dc72088c8c31aa453ea87d01265519fd0a56fd" }, "downloads": -1, "filename": "nameko_grpc-1.0.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "46b740583b737e7e1e94285ee87d4476", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 25299, "upload_time": "2019-02-12T16:00:17", "url": "https://files.pythonhosted.org/packages/b7/73/cd67071d28b79af0693600c2091d5e4275202e77120301f28b838fed264b/nameko_grpc-1.0.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b18edc309bb22a42a8e481be8098db80", "sha256": "0761e451e68f9d78dea9ae8be393c548cfce1388bbc2da6c5effa3c63e69fb21" }, "downloads": -1, "filename": "nameko-grpc-1.0.0.tar.gz", "has_sig": false, "md5_digest": "b18edc309bb22a42a8e481be8098db80", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 49496, "upload_time": "2019-02-12T16:00:19", "url": "https://files.pythonhosted.org/packages/e0/34/4414596ceb3173ce3460b01dd5ea2b36dbc4b624d44ae7b97cf18a0e7d44/nameko-grpc-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "dfe4e58c605731ad18ba3838aaee0c16", "sha256": "c948291ef096346b00c041a40f40566e9b2bdc2df93839d6469b48a6bd48a5fe" }, "downloads": -1, "filename": "nameko_grpc-1.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "dfe4e58c605731ad18ba3838aaee0c16", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 25300, "upload_time": "2019-02-25T20:53:12", "url": "https://files.pythonhosted.org/packages/75/75/81b8dec8b05144480fbda0975aa5df2f8350bd6aa0257b48c9c166640dd5/nameko_grpc-1.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "50a6ac05c9799f03c2641ca8b308303e", "sha256": "e2eb861c40f3c80ba2fa33ffd7ba4d0dda3c01edaa3b032abaa25ac261f8475b" }, "downloads": -1, "filename": "nameko-grpc-1.0.1.tar.gz", "has_sig": false, "md5_digest": "50a6ac05c9799f03c2641ca8b308303e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 49495, "upload_time": "2019-02-25T20:53:14", "url": "https://files.pythonhosted.org/packages/b9/f1/aca2c0ecde9508b44d58c8cab1f76664948e4f49ad5a853e1b1053a673cb/nameko-grpc-1.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "dfe4e58c605731ad18ba3838aaee0c16", "sha256": "c948291ef096346b00c041a40f40566e9b2bdc2df93839d6469b48a6bd48a5fe" }, "downloads": -1, "filename": "nameko_grpc-1.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "dfe4e58c605731ad18ba3838aaee0c16", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 25300, "upload_time": "2019-02-25T20:53:12", "url": "https://files.pythonhosted.org/packages/75/75/81b8dec8b05144480fbda0975aa5df2f8350bd6aa0257b48c9c166640dd5/nameko_grpc-1.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "50a6ac05c9799f03c2641ca8b308303e", "sha256": "e2eb861c40f3c80ba2fa33ffd7ba4d0dda3c01edaa3b032abaa25ac261f8475b" }, "downloads": -1, "filename": "nameko-grpc-1.0.1.tar.gz", "has_sig": false, "md5_digest": "50a6ac05c9799f03c2641ca8b308303e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 49495, "upload_time": "2019-02-25T20:53:14", "url": "https://files.pythonhosted.org/packages/b9/f1/aca2c0ecde9508b44d58c8cab1f76664948e4f49ad5a853e1b1053a673cb/nameko-grpc-1.0.1.tar.gz" } ] }