{
    "info": {
        "author": "James Woo",
        "author_email": "james.e.woo@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "License :: OSI Approved :: Apache Software License",
            "Operating System :: MacOS :: MacOS X",
            "Operating System :: Unix",
            "Programming Language :: Perl",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Unix Shell",
            "Topic :: Internet :: WWW/HTTP :: HTTP Servers",
            "Topic :: Scientific/Engineering :: Artificial Intelligence"
        ],
        "description": "# sockeye-serving\n`sockeye-serving` is a containerized service for neural machine translation that uses Amazon's `sockeye` framework as the translation engine.\nThe web server makes use of `mxnet-model-server`, which provides a management API for loading models and a prediction API for requesting translations.\n\nAny Sockeye model can be loaded via the management API.\nText preprocessing is built into the request pipeline and supports a wide variety of languages.\nSpecialized processing for specific languages can be implemented using custom handlers.\n\n## Quickstart\nThis example shows how to serve an existing model for Chinese to English translation.\nFirst, pull the latest Docker image:\n```bash\ndocker pull jwoo11/sockeye-serving\n```\n\nDownload the example model archive (MAR).\nThis is a ZIP archive containing the parameter files and scripts needed to run translation:\n* https://www.dropbox.com/s/pk7hmp7a5zjcfcj/zh.mar?dl=0\n\nExtract the MAR file to `/tmp/models`.\n This directory will be the source for a bind mount for Docker:\n```bash\nunzip -d /tmp/models/zh zh.mar\n```\n\nStart the server:\n```bash\ndocker run -itd --name sockeye_serving -p 8080:8080 -p 8081:8081 -v /tmp/models:/opt/ml/model jwoo11/sockeye-serving\n```\n\nNow, load the model using the management API. Note that the URL of the model is relative to the bind mount:\n```bash\ncurl -X POST \"http://localhost:8081/models?synchronous=true&initial_workers=1&url=zh\"\n```\nGet the status of the model with the following:\n```bash\ncurl -X GET \"http://localhost:8081/models/zh\"\n```\nThe response should look like this:\n```json\n{\n  \"modelName\": \"zh\",\n  \"modelUrl\": \"zh\",\n  \"runtime\": \"python3\",\n  \"minWorkers\": 1,\n  \"maxWorkers\": 1,\n  \"batchSize\": 1,\n  \"maxBatchDelay\": 100,\n  \"workers\": [\n    {\n      \"id\": \"9000\",\n      \"startTime\": \"2019-01-26T00:49:10.431Z\",\n      \"status\": \"READY\",\n      \"gpu\": false,\n      \"memoryUsage\": 601395200\n    }\n  ]\n}\n```\n\nTo translate text use the inference API. Notice that the port is different from above.\n```bash\ncurl -X POST \"http://localhost:8080/predictions/zh\" -H \"Content-Type: application/json\" \\\n    -d '{ \"text\": \"\u6211\u7684\u4e16\u754c\u662f\u4e00\u6b3e\u958b\u653e\u4e16\u754c\u904a\u6232\uff0c\u73a9\u5bb6\u6c92\u6709\u5177\u9ad4\u8981\u5b8c\u6210\u7684\u76ee\u6a19\uff0c\u5373\u73a9\u5bb6\u6709\u8d85\u9ad8\u7684\u81ea\u7531\u5ea6\u9078\u64c7\u5982\u4f55\u73a9\u904a\u6232\" }'\n```\n\nThe translation quality depends on the model. The provided model returns this translation:\n```json\n{\n  \"translation\": \"in my life was a life of a life of a public public, and a public, a time, a video, a play, which, it was a time of a time of a time.\"\n}\n```\n\nA better model trained on more data returns this response:\n```json\n{\n  \"translation\": \"My world is an open world game, and players have no specific goal to accomplish, that is, players have a high degree of freedom to choose how to play.\"\n}\n```\n\n## Installation\nTo install the command line clients for `sockeye-serving` run the following in a virtual environment:\n```bash\npip install sockeye-serving\n```\nIf you want to install from source, a `Pipfile` is provided.\nClone the repository and run `pipenv install`.\n\nInstallation places the command line interfaces `sockeye-serving` and `sockeye-client` on your virtual environment's path.\n\n## Command Line Interfaces\nYou can use `sockeye-serving` to easily start Docker and to make REST calls to both the management and prediction APIs.\nFirst, a configuration file must be placed in either the current directory or some place referenced by `SOCKEYE_SERVING_CONF`.\nExample properties are located in `config/sockeye-serving.conf`.\nHere's some basic usage:\n```bash\n# start the Docker container\nsockeye-serving start\n\n# deploy a model\nsockeye-serving deploy zh\n\n# list available models\nsockeye-serving list\n\n# translate text\nsockeye-serving translate zh \"my text\"\n\n# upload a file for translation\nsockeye-serving upload zh \"my_file.txt\"\n```\nRun `sockeye-serving help` for a full list of commands.\n\nThe Python client takes a YAML configuration file.\nAn example configuration is in `config/sockeye-client.yml`.\nThis client does not support restarting Docker, however, it does exercise the full API provided by `mxnet-model-server`.\nThe commands which accept query parameters are below:\n```bash\n$ sockeye-client deploy -h\nusage: sockeye-client deploy [-h] [-m MODEL_NAME] [-x HANDLER] [-r RUNTIME]\n                             [-b BATCH_SIZE] [-d MAX_BATCH_DELAY]\n                             [-i INITIAL_WORKERS] [-s] [-t RESPONSE_TIMEOUT]\n                             url\n...\n\n$ sockeye-client list -h\nusage: sockeye-client list [-h] [-l LIMIT] [-t NEXT_PAGE_TOKEN]\n...\n\n$ sockeye-client scale -h\nusage: sockeye-client scale [-h] [-a MIN_WORKER] [-b MAX_WORKER]\n                            [-n NUMBER_GPU] [-s] [-t TIMEOUT]\n                            model_name\n...\n```\nRun `sockeye-client -h` to show a full list of commands.\nFor more information on the API, see [additional documentation](#additional-documentation) for `mxnet-model-server`.\n\n## Jupyter Notebook\nIf you want to translate text with Jupyter, you can use `notebooks/machine_translation.ipynb`.\nMake sure `requests` is installed in your Python environment.\n\n## Choosing between CPUs and GPUs\n`sockeye-serving` provides different image tags for CPUs and GPUs.\nYou can set the desired tag in your `sockeye-serving.conf` file.\nYou'll also need to specify a Sockeye config file `sockeye-args.txt`.\nThis file contains arguments passed to the Sockeye translation engine.\nExample files for both CPU and GPU configs are under `config/sockeye`.\n\nTo use GPUs, ensure `nvidia-docker` is installed on the host machine.\nIn `sockeye-serving.conf` set the image tag to one with \"gpu\" in its name, such as `latest-gpu`, and set `docker_exec=\"nvidia-docker\"`.\nThen run `sockeye-serving update MODEL_NAME config/sockeye/gpu/sockeye-args.txt`.\n\nFor CPUs, use a tag without \"gpu\" in its name, such as `latest`, and use the CPU version of the Sockeye config file.\nThe changes to `sockeye-serving.conf` will be picked up when you run `sockeye-serving start`.\n\n## Initializing Models\nEach model must be initialized with a `MANIFEST.json` file in order for `mxnet-model-server` to deploy it.\nAn easy way to initialize a model is to run `sockeye-serving archive MODEL_NAME HANDLER`, where `HANDLER` is the name of a Python handler module under `src/sockeye_serving`.\nThe provided handlers include `ko_handler` (Korean), `zh_handler` (Chinese), and `default_handler` (generic).\nAfter running the archive command, your model directory should have a file `MAR-INF/MANIFEST.json` that looks like:\n```json\n{\n  \"runtime\": \"python3\",\n  \"model\": {\n    \"modelName\": \"zho\",\n    \"handler\": \"sockeye_serving.zh_handler:handle\"\n  },\n  \"modelServerVersion\": \"1.0\",\n  \"implementationVersion\": \"1.0\",\n  \"specificationVersion\": \"1.0\"\n}\n```\n\n## Enabling TLS\nThe provided configuration instructs the server to use plain HTTP.\nTo enable TLS, you can either supply a Java keystore or a private key and certificate in PEM format.\n\nUsing `config/config.properties` as a starting point, create a new `config.properties` file and save it under `/tmp/models`:\n```properties\nmodel_store=/opt/ml/model\ninference_address=https://0.0.0.0:8443\nmanagement_address=https://0.0.0.0:8444\n```\nSuppose you have a key pair residing on the host at `/path/to/certs`.\nSet the properties for the keystore:\n```properties\nkeystore=/path/to/certs/keystore.p12\nkeystore_pass=changeit\nkeystore_type=PKCS12\n```\nOr provide the path to the server's private key and certificate:\n```properties\nprivate_key_file=/path/to/certs/private.key\ncertificate_file=/path/to/certs/cert.pem\n```\nThen start the container:\n```bash\ndocker run -itd --name sockeye_serving -p 8443:8443 -p 8444:8444 \\\n    -v /path/to/certs:/path/to/certs \\\n    -v /tmp/models:/opt/ml/model jwoo11/sockeye-serving \\\n    mxnet-model-server --start --mms-config /opt/ml/model/config.properties\n```\n\nTo make requests using `curl` you should ensure that you set `--cert`, `--key`, and `--cacert` as needed.\n\n## <a name=\"additional-documentation\"></a> Additional Documentation\n\nFor more information on `mxnet-model-server`, see:\n* https://github.com/awslabs/mxnet-model-server/tree/master/docs\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/jamesewoo/sockeye-serving",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "sockeye-serving",
        "package_url": "https://pypi.org/project/sockeye-serving/",
        "platform": "",
        "project_url": "https://pypi.org/project/sockeye-serving/",
        "project_urls": {
            "Homepage": "https://github.com/jamesewoo/sockeye-serving"
        },
        "release_url": "https://pypi.org/project/sockeye-serving/2.1.0/",
        "requires_dist": [
            "model-archiver",
            "pyyaml",
            "requests"
        ],
        "requires_python": "",
        "summary": "A containerized service for neural machine translation",
        "version": "2.1.0"
    },
    "last_serial": 5621803,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f3006b356d4728d6a69a84d41cc40618",
                    "sha256": "614e0632a06d470b6be133f8f753eea1003a2f4fac97467b6d70b32230c466f6"
                },
                "downloads": -1,
                "filename": "sockeye_serving-1.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f3006b356d4728d6a69a84d41cc40618",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 71570,
                "upload_time": "2019-04-20T16:56:07",
                "url": "https://files.pythonhosted.org/packages/48/7d/cfc09928d707f662be30bb02a2221b552d14afa11de72e80b50fecb8a873/sockeye_serving-1.0.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e629a3557b6fce4818737fb8c9c622ca",
                    "sha256": "3f2f00f5417204f9ef35689f09521f6c221f39e8215f9a2c4ad33caf87a90534"
                },
                "downloads": -1,
                "filename": "sockeye-serving-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "e629a3557b6fce4818737fb8c9c622ca",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14830,
                "upload_time": "2019-04-20T16:56:09",
                "url": "https://files.pythonhosted.org/packages/d4/af/92ea7db5b9d34d60b5e4c694a74948b6d24200ed318a93e4e02bfc355af7/sockeye-serving-1.0.0.tar.gz"
            }
        ],
        "1.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1987f86cb076611ba6295568104b9d6d",
                    "sha256": "7e5dcb0c72c6311245ce24e8b6d082b7771eb073938c51360526c5fa10577e1e"
                },
                "downloads": -1,
                "filename": "sockeye_serving-1.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1987f86cb076611ba6295568104b9d6d",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 73932,
                "upload_time": "2019-04-26T01:27:39",
                "url": "https://files.pythonhosted.org/packages/71/42/84780c6ebb79b880ebf088e88cbb41102a9bc61d3c5775654491db944125/sockeye_serving-1.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "778f70acf28129186e8d795d6e634b22",
                    "sha256": "b60d74a19b653335ef753e53889d7fab7bb317b7c86ef2b64444a7157ce7d5d9"
                },
                "downloads": -1,
                "filename": "sockeye-serving-1.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "778f70acf28129186e8d795d6e634b22",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17511,
                "upload_time": "2019-04-26T01:27:41",
                "url": "https://files.pythonhosted.org/packages/5d/06/4c1160021b38244171e27bda39870f02d7888f3474c62522f3f4ac0d8d57/sockeye-serving-1.0.1.tar.gz"
            }
        ],
        "1.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "35a418a2777c6eea159b00e3c538ee8e",
                    "sha256": "64f9bce60651c88ffb5c0c4e6c44e86ade5bcc83b5c347e6a50192022dfc7506"
                },
                "downloads": -1,
                "filename": "sockeye_serving-1.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "35a418a2777c6eea159b00e3c538ee8e",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 75443,
                "upload_time": "2019-05-12T22:39:34",
                "url": "https://files.pythonhosted.org/packages/5c/14/b04010c923bdc0be42fe2ab47feca686a741184dad1801073bfae4d6c995/sockeye_serving-1.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "9246af6635d86d40b23ca3488d7b7351",
                    "sha256": "c0533ece30dadf9ab24b0819c32137465116b877c9fba9bce8b9458454cff514"
                },
                "downloads": -1,
                "filename": "sockeye-serving-1.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "9246af6635d86d40b23ca3488d7b7351",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17728,
                "upload_time": "2019-05-12T22:39:36",
                "url": "https://files.pythonhosted.org/packages/21/4d/fdffc1d96dc82d0ed08c8c689ceff0526ba748ac273d280e75481a196480/sockeye-serving-1.1.0.tar.gz"
            }
        ],
        "2.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "314aa21fbfad9d9a9b50339cc2cc1762",
                    "sha256": "a153175ea697a171365654ee7e55749d87f8fdeb09da5e9f54508385ff35d0af"
                },
                "downloads": -1,
                "filename": "sockeye_serving-2.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "314aa21fbfad9d9a9b50339cc2cc1762",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 75143,
                "upload_time": "2019-07-17T02:08:34",
                "url": "https://files.pythonhosted.org/packages/de/09/d77b5679e16e81c74665b048370c18b25b0d1a8287e7d1db513dab36346d/sockeye_serving-2.0.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "05fabef71d4584d06b1b0b73659ca97d",
                    "sha256": "43897c8c609c8c57d02c1073b658b9bd832f123a921caaa0fd41bf9e82ed6dbb"
                },
                "downloads": -1,
                "filename": "sockeye-serving-2.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "05fabef71d4584d06b1b0b73659ca97d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18551,
                "upload_time": "2019-07-17T02:08:35",
                "url": "https://files.pythonhosted.org/packages/bc/a2/9c69702d3dcade01202a19b83d6a96bc372b3f6b76112492b700f06d770b/sockeye-serving-2.0.0.tar.gz"
            }
        ],
        "2.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a961c807aca47aeb03858178d429b2ae",
                    "sha256": "1cefaac58bf203425130ce7ef6a3626de92ee4affeead4ebcd3007b120f22ccd"
                },
                "downloads": -1,
                "filename": "sockeye_serving-2.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a961c807aca47aeb03858178d429b2ae",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 75336,
                "upload_time": "2019-08-02T03:44:15",
                "url": "https://files.pythonhosted.org/packages/ed/1b/5be413b659c6e0102a5abf6c02a3b1205d42325f5deda26f4051e0f0659f/sockeye_serving-2.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "eafd9e927b1fad71f6608d1e5add348c",
                    "sha256": "2af8e03208cc0d0c17dbca695831d929ef269d95ad5233451ea55f3748ee6db8"
                },
                "downloads": -1,
                "filename": "sockeye-serving-2.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "eafd9e927b1fad71f6608d1e5add348c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18756,
                "upload_time": "2019-08-02T03:44:16",
                "url": "https://files.pythonhosted.org/packages/bf/67/26402e14c1d4585dce8fd2985f0045164edc5af25b2a8ab167de9124a863/sockeye-serving-2.1.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "a961c807aca47aeb03858178d429b2ae",
                "sha256": "1cefaac58bf203425130ce7ef6a3626de92ee4affeead4ebcd3007b120f22ccd"
            },
            "downloads": -1,
            "filename": "sockeye_serving-2.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "a961c807aca47aeb03858178d429b2ae",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 75336,
            "upload_time": "2019-08-02T03:44:15",
            "url": "https://files.pythonhosted.org/packages/ed/1b/5be413b659c6e0102a5abf6c02a3b1205d42325f5deda26f4051e0f0659f/sockeye_serving-2.1.0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "eafd9e927b1fad71f6608d1e5add348c",
                "sha256": "2af8e03208cc0d0c17dbca695831d929ef269d95ad5233451ea55f3748ee6db8"
            },
            "downloads": -1,
            "filename": "sockeye-serving-2.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "eafd9e927b1fad71f6608d1e5add348c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 18756,
            "upload_time": "2019-08-02T03:44:16",
            "url": "https://files.pythonhosted.org/packages/bf/67/26402e14c1d4585dce8fd2985f0045164edc5af25b2a8ab167de9124a863/sockeye-serving-2.1.0.tar.gz"
        }
    ]
}