{ "info": { "author": "Jay Johnson", "author_email": "jay.p.h.johnson@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "Deploying a Distributed AI Stack to Kubernetes on CentOS\n--------------------------------------------------------\n\n.. image:: https://i.imgur.com/xO4CbfN.png\n\nInstall and manage a Kubernetes cluster (version 1.13.3) with helm on a single CentOS 7 vm or in multi-host mode that runs the cluster on 3 CentOS 7 vms. Once running, you can deploy a distributed, scalable python stack capable of delivering a resilient REST service with JWT for authentication and Swagger for development. This service uses a decoupled REST API with two distinct worker backends for routing simple database read and write tasks vs long-running tasks that can use a Redis cache and do not need a persistent database connection. This is handy for not only simple CRUD applications and use cases, but also serving a secure multi-tenant environment where multiple users manage long-running tasks like training deep neural networks that are capable of making near-realtime predictions.\n\nThis guide was built for deploying the `AntiNex stack of docker containers `__ and the `Stock Analysis Engine `__ on a Kubernetes single host or multi-host cluster.\n\n- `Managing a Multi-Host Kubernetes Cluster with an External DNS Server `__\n- `Cert Manager with Let's Encrypt SSL support `__\n- `A Native Ceph Cluster for Persistent Volume Management with KVM `__\n- `A Third-party Rook Ceph Cluster for Persistent Volumes `__\n- `Minio S3 Object Store `__\n- `Redis `__\n- `Postgres `__\n- `Django REST API with JWT and Swagger `__\n- `Django REST API Celery Workers `__\n- `Jupyter `__\n- `Core Celery Workers `__\n- `pgAdmin4 `__\n- `(Optional) Splunk with TCP and HEC Service Endpoints `__\n\nGetting Started\n---------------\n\n.. note:: Please ensure for single-vm hosting that the CentOS machine has at least 4 CPU cores and more than 8 GB ram. Here is a screenshot of the CPU utilization during AI training with only 3 cores:\n\n .. image:: https://i.imgur.com/KQ7MBdM.png\n\nOverview\n========\n\nThis guide installs the following systems and a storage solution `Rook with Ceph cluster (default) `__ or NFS volumes to prepare the host for running containers and automatically running them on host startup:\n\n- Kubernetes\n- Helm and Tiller\n- `Minio S3 Storage `__\n- `Persistent Storage Volumes using Rook with Ceph cluster `__ or optional NFS Volumes mounted at: ``/data/k8/redis``, ``/data/k8/postgres``, ``/data/k8/pgadmin``\n- Flannel CNI\n\nInstall\n=======\n\nHere is a video showing how to prepare the host to run a local Kubernetes cluster:\n\n.. image:: https://asciinema.org/a/193463.png\n :target: https://asciinema.org/a/193463?autoplay=1\n :alt: Install Kubernetes\n\nPreparing the host to run Kubernetes requires run this as root\n\n::\n\n sudo su\n ./prepare.sh\n\n.. note:: This has only been tested on CentOS 7 and Ubuntu 18.04 and requires commenting out all swap entries in ``/etc/fstab`` to work\n\n.. warning:: This guide used to install the cluster on Ubuntu 18.04, but after seeing high CPU utilization after a few days of operation this guide was moved to CentOS 7. The specific issues on Ubuntu were logged in ``journalctl -xe`` and appeared to be related to \"volumes not being found\" and \"networking disconnects\".\n\nValidate\n--------\n\n#. Install Kubernetes Config\n\n Run as your user\n\n ::\n\n mkdir -p $HOME/.kube\n sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config\n sudo chown $(id -u):$(id -g) $HOME/.kube/config\n\n Or use the script:\n\n ::\n\n ./user-install-kubeconfig.sh\n\n#. Check the Kubernetes Version\n\n ::\n\n kubectl version\n Client Version: version.Info{Major:\"1\", Minor:\"11\", GitVersion:\"v1.11.1\", GitCommit:\"b1b29978270dc22fecc592ac55d903350454310a\", GitTreeState:\"clean\", BuildDate:\"2018-07-17T18:53:20Z\", GoVersion:\"go1.10.3\", Compiler:\"gc\", Platform:\"linux/amd64\"}\n The connection to the server localhost:8080 was refused - did you specify the right host or port?\n\n#. Confirm the Kubernetes Pods Are Running\n\n ::\n\n kubectl get pods -n kube-system\n\n ::\n\n NAME READY STATUS RESTARTS AGE\n coredns-78fcdf6894-k8srv 1/1 Running 0 4m\n coredns-78fcdf6894-xx8bt 1/1 Running 0 4m\n etcd-dev 1/1 Running 0 3m\n kube-apiserver-dev 1/1 Running 0 3m\n kube-controller-manager-dev 1/1 Running 0 3m\n kube-flannel-ds-m8k9w 1/1 Running 0 4m\n kube-proxy-p4blg 1/1 Running 0 4m\n kube-scheduler-dev 1/1 Running 0 3m\n tiller-deploy-759cb9df9-wxvp8 1/1 Running 0 4m\n\nDeploy Redis and Postgres and the Nginx Ingress\n-----------------------------------------------\n\nHere is a video showing how to deploy Postgres, Redis, Nginx Ingress, and the pgAdmin4 as pods in the cluster:\n\n.. image:: https://asciinema.org/a/193476.png\n :target: https://asciinema.org/a/193476?autoplay=1\n :alt: Deploy Redis and Postgres and the Nginx Ingress\n\n.. note:: Postgres, pgAdmin4 and Redis use Rook Ceph to persist data\n\nHere are the commands to deploy Postgres, Redis, Nginx Ingress, and pgAdmin4 in the cluster:\n\n.. note:: Please ensure helm is installed and the tiller pod in the ``kube-system`` namespace is the ``Running`` state or Redis will encounter deployment issues\n\nInstall Go using the `./tools/install-go.sh script `__ or with the commands:\n\n::\n\n # note go install has only been tested on CentOS 7 and Ubuntu 18.04:\n sudo su\n GO_VERSION=\"1.11\"\n GO_OS=\"linux\"\n GO_ARCH=\"amd64\"\n go_file=\"go${GO_VERSION}.${GO_OS}-${GO_ARCH}.tar.gz\"\n curl https://dl.google.com/go/${go_file} --output /tmp/${go_file}\n export GOPATH=$HOME/go/bin\n export PATH=$PATH:$GOPATH:$GOPATH/bin\n tar -C $HOME -xzf /tmp/${go_file}\n $GOPATH/go get github.com/blang/expenv\n # make sure to add GOPATH and PATH to ~/.bashrc\n\n::\n\n ./user-install-kubeconfig.sh\n ./deploy-resources.sh\n\nIf you want to deploy splunk you can add it as an argument:\n\n::\n\n ./deploy-resources.sh splunk\n\nIf you want to deploy splunk with Let's Encrypt make sure to add ``prod`` as an argument:\n\n::\n\n ./deploy-resources.sh splunk prod\n\nStart Applications\n------------------\n\nHere is a video showing how to start the Django REST Framework, Celery Workers, Jupyter, and the AntiNex Core as pods in the cluster:\n\n.. image:: https://asciinema.org/a/193485.png\n :target: https://asciinema.org/a/193485?autoplay=1\n :alt: Start Applications\n\nStart all applications as your user with the command:\n\n::\n\n ./start.sh\n\nIf you want to deploy the splunk-ready application builds, you can add it as an argument:\n\n::\n\n ./start.sh splunk\n\nIf you want to deploy the splunk-ready application builds integrated with Let's Encrypt TLS encryption, just add ``prod`` as an argument:\n\n::\n\n ./start.sh splunk prod\n\n.. note:: The `Cert Manager `__ is set to staging mode by default and requires the ``prod`` argument to prevent accidentally getting blocked due to Lets Encrypt rate limits\n\nConfirm Pods are Running\n========================\n\nDepending on how fast your network connection is the initial container downloads can take a few minutes. Please wait until all pods are ``Running`` before continuing.\n\n::\n\n kubectl get pods\n\nRun a Database Migration\n------------------------\n\nHere is a video showing how to apply database schema migrations in the cluster:\n\n.. image:: https://asciinema.org/a/193491.png\n :target: https://asciinema.org/a/193491?autoplay=1\n :alt: Run a Database Migration\n\nTo apply new Django database migrations, run the following command:\n\n::\n\n ./api/migrate-db.sh\n\nAdd Ingress Locations to /etc/hosts\n-----------------------------------\n\nWhen running locally (also known in these docs as ``dev`` mode), all ingress urls need to resolve on the network. Please append the following entries to your local ``/etc/hosts`` file on the ``127.0.0.1`` line:\n\n::\n\n sudo vi /etc/hosts\n\nAppend the entries to the existing ``127.0.0.1`` line:\n\n::\n\n 127.0.0.1 api.example.com jupyter.example.com pgadmin.example.com splunk.example.com s3.example.com ceph.example.com minio.example.com\n\nUsing the Minio S3 Object Store\n-------------------------------\n\nBy default, the Kubernetes cluster has a `Minio S3 object store running on a Ceph Persistent Volume `__. S3 is a great solution for distributing files, datasets, configurations, static assets, build artifacts and many more across components, regions, and datacenters using an S3 distributed backend. Minio can also replicate some of the `AWS Lambda event-based workflows `__ with `Minio bucket event listeners `__.\n\nFor reference, Minio was deployed using this script:\n\n::\n\n ./minio/run.sh\n\nView the Verification Tests on the Minio Dashboard\n==================================================\n\nLogin with:\n\n- access key: ``trexaccesskey``\n- secret key: ``trex123321``\n\nhttps://minio.example.com/minio/s3-verification-tests/\n\nTest Minio S3 with Bucket Creation and File Upload and Download\n===============================================================\n\n#. Run from inside the API container\n\n ::\n\n ./api/ssh.sh\n source /opt/venv/bin/activate && run_s3_test.py\n\n Example logs:\n\n ::\n\n creating test file: run-s3-test.txt\n connecting: http://minio-service:9000\n checking bucket=s3-verification-tests exists\n upload_file(run-s3-test.txt, s3-verification-tests, s3-worked-on-2018-08-12-15-21-02)\n upload_file(s3-verification-tests, s3-worked-on-2018-08-12-15-21-02, download-run-s3-test.txt)\n download_filename=download-run-s3-test.txt contents: tested on: 2018-08-12 15:21:02\n exit\n\n#. Run from outside the Kubernetes cluster\n\n .. note:: This tool requires the python ``boto3`` pip is installed\n\n ::\n\n source ./minio/envs/ext.env\n ./minio/run_s3_test.py\n\n#. Verify the files were uploaded to Minio\n\n https://minio.example.com/minio/s3-verification-tests/\n\nUsing the Rook Ceph Cluster\n---------------------------\n\nBy default, the Kubernetes cluster is running a `Rook Ceph cluster for storage `__ which provides HA persistent volumes and claims.\n\nYou can review the persistent volumes and claims using the Ceph Dashboard:\n\nhttps://ceph.example.com\n\nCreate a User\n-------------\n\nCreate the user ``trex`` with password ``123321`` on the REST API.\n\n::\n\n ./api/create-user.sh\n\nDeployed Web Applications\n-------------------------\n\nHere are the hosted web application urls. These urls are made accessible by the included nginx-ingress.\n\nView Django REST Framework\n--------------------------\n\nLogin with:\n\n- user: ``trex``\n- password: ``123321``\n\nhttps://api.example.com\n\nView Swagger\n------------\n\nLogin with:\n\n- user: ``trex``\n- password: ``123321``\n\nhttps://api.example.com/swagger\n\nView Jupyter\n------------\n\nLogin with:\n\n- password: ``admin``\n\nhttps://jupyter.example.com\n\nView pgAdmin\n------------\n\nLogin with:\n\n- user: ``admin@admin.com``\n- password: ``123321``\n\nhttps://pgadmin.example.com\n\nView Minio S3 Object Storage\n----------------------------\n\nLogin with:\n\n- access key: ``trexaccesskey``\n- secret key: ``trex123321``\n\nhttps://minio.example.com\n\nView Ceph\n---------\n\nhttps://ceph.example.com\n\nView Splunk\n-----------\n\nLogin with:\n\n- user: ``trex``\n- password: ``123321``\n\nhttps://splunk.example.com\n\nTraining AI with the Django REST API\n------------------------------------\n\nThese steps install the `AntiNex python client `__ for training a deep neural network to predict attack packets from recorded network data (all of which is already included in the docker containers).\n\n#. Create a virtual environment and install the client\n\n ::\n\n virtualenv -p python3 /opt/venv && source /opt/venv/bin/activate\n pip install antinex-client\n\n#. Watch the application logs\n\n\n From a separate terminal, you can tail the Django REST API logs with the command:\n\n ::\n\n ./api/logs.sh\n\n From a separate terminal, you can tail the Django Celery Worker logs with the command:\n\n ::\n\n ./worker/logs.sh\n\n From a separate terminal, you can tail the AntiNex Core Worker logs with the command:\n\n ::\n\n ./core/logs.sh\n\n .. note:: Use ``ctrl + c`` to stop these log tailing commands\n\nTrain a Deep Neural Network on Kubernetes\n-----------------------------------------\n\nWith virtual environment set up, we can use the client to train a deep neural network with the included datasets:\n\n.. note:: this can take a few minutes to finish depending on your hosting resources\n\n::\n\n ai -a https://api.example.com -u trex -p 123321 -s -f ./tests/scaler-full-django-antinex-simple.json\n\nWhile you wait, here is a video showing the training and get results:\n\n.. image:: https://i.imgur.com/0hcMfti.png\n :target: https://asciinema.org/a/193494?autoplay=1\n :alt: Train a Deep Neural Network on Kubernetes\n\nGet the AI Job Record\n---------------------\n\n::\n\n ai_get_job.py -a https://api.example.com -u trex -p 123321 -i 1\n\nGet the AI Training Job Results\n-------------------------------\n\n::\n\n ai_get_results.py -a https://api.example.com -u trex -p 123321 -i 1 -s\n\nStandalone Deployments\n----------------------\n\nBelow are steps to manually deploy each component in the stack with Kubernetes.\n\nDeploy Redis\n------------\n\n::\n\n ./redis/run.sh\n\nOr manually with the commands:\n\n::\n\n echo \"deploying persistent volume for redis\" \n kubectl apply -f ./redis/pv.yml\n echo \"deploying Bitnami redis stable with helm\" \n helm install \\\n --name redis stable/redis \\\n --set rbac.create=true \\\n --values ./redis/redis.yml\n\nConfirm Connectivity\n====================\n\nThe following commands assume you have ``redis-tools`` installed (``sudo apt-get install redis-tools``).\n\n::\n\n redis-cli -h $(kubectl describe pod redis-master-0 | grep IP | awk '{print $NF}') -p 6379\n 10.244.0.81:6379> info\n 10.244.0.81:6379> exit\n\nDebug Redis Cluster\n===================\n\n#. Examine Redis Master\n\n ::\n\n kubectl describe pod redis-master-0\n\n#. Examine Persistent Volume Claim\n\n ::\n\n kubectl get pvc\n NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE\n redis-ceph-data Bound pvc-1a88e3a6-9df8-11e8-8047-0800270864a8 8Gi RWO rook-ceph-block 46m\n\n#. Examine Persistent Volume\n\n ::\n\n kubectl get pv\n NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE\n pvc-1a88e3a6-9df8-11e8-8047-0800270864a8 8Gi RWO Delete Bound default/redis-ceph-data rook-ceph-block 46m\n\nPossible Errors\n===============\n\n#. Create the Persistent Volumes\n\n ::\n\n Warning FailedMount 2m kubelet, dev MountVolume.SetUp failed for volume \"redis-pv\" : mount failed: exit status 32\n\n ::\n\n ./pvs/create-pvs.sh\n\nDelete Redis\n============\n\n::\n\n helm del --purge redis\n release \"redis\" deleted\n\nDelete Persistent Volume and Claim\n==================================\n\n#. Delete Claim\n\n ::\n\n kubectl delete pvc redis-data-redis-master-0\n\n#. Delete Volume\n\n ::\n\n kubectl delete pv redis-pv\n persistentvolume \"redis-pv\" deleted\n\nDeploy Postgres\n---------------\n\nInstall Go\n==========\n\nUsing Crunchy Data's postgres containers requires having go installed. Go can be installed using the `./tools/install-go.sh script `__ or with the commands:\n\n::\n\n # note go install has only been tested on CentOS 7 and Ubuntu 18.04:\n sudo su\n GO_VERSION=\"1.11\"\n GO_OS=\"linux\"\n GO_ARCH=\"amd64\"\n go_file=\"go${GO_VERSION}.${GO_OS}-${GO_ARCH}.tar.gz\"\n curl https://dl.google.com/go/${go_file} --output /tmp/${go_file}\n export GOPATH=$HOME/go/bin\n export PATH=$PATH:$GOPATH:$GOPATH/bin\n tar -C $HOME -xzf /tmp/${go_file}\n $GOPATH/go get github.com/blang/expenv\n # make sure to add GOPATH and PATH to ~/.bashrc\n\nStart\n=====\n\nStart the `Postgres container `__ within Kubernetes:\n\n::\n\n ./postgres/run.sh\n\nDebug Postgres\n==============\n\n#. Examine Postgres\n\n ::\n\n kubectl describe pod primary\n\n Type Reason Age From Message\n ---- ------ ---- ---- -------\n Normal Scheduled 2m default-scheduler Successfully assigned default/primary to dev\n Normal Pulling 2m kubelet, dev pulling image \"crunchydata/crunchy-postgres:centos7-10.4-1.8.3\"\n Normal Pulled 2m kubelet, dev Successfully pulled image \"crunchydata/crunchy-postgres:centos7-10.4-1.8.3\"\n Normal Created 2m kubelet, dev Created container\n Normal Started 2m kubelet, dev Started container\n\n#. Examine Persistent Volume Claim\n\n ::\n\n kubectl get pvc\n NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE\n pgadmin4-http-data Bound pvc-19031825-9df8-11e8-8047-0800270864a8 400M RWX rook-ceph-block 46m\n primary-pgdata Bound pvc-17652595-9df8-11e8-8047-0800270864a8 400M RWX rook-ceph-block 46m\n\n\n#. Examine Persistent Volume\n\n ::\n\n kubectl get pv\n NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE\n pvc-17652595-9df8-11e8-8047-0800270864a8 400M RWX Delete Bound default/primary-pgdata rook-ceph-block 47m\n pvc-19031825-9df8-11e8-8047-0800270864a8 400M RWX Delete Bound default/pgadmin4-http-data rook-ceph-block 47m\n\nDeploy pgAdmin\n--------------\n\nPlease confirm go is installed with the `Install Go section `__.\n\nStart\n=====\n\nStart the `pgAdmin4 container `__ within Kubernetes:\n\n::\n\n ./pgadmin/run.sh\n\nGet Logs\n========\n\n::\n\n ./pgadmin/logs.sh\n\nSSH into pgAdmin\n================\n\n::\n\n ./pgadmin/ssh.sh\n\nDeploy Django REST API\n----------------------\n\nUse these commands to manage the `Django REST Framework pods `__ within Kubernetes.\n\nStart\n=====\n\n::\n\n ./api/run.sh\n\nRun a Database Migration\n========================\n\nTo apply a django database migration run the following command:\n\n::\n\n ./api/migrate-db.sh\n\nGet Logs\n========\n\n::\n\n ./api/logs.sh\n\nSSH into the API\n================\n\n::\n\n ./api/ssh.sh\n\nDeploy Django Celery Workers\n----------------------------\n\nUse these commands to manage the `Django Celery Worker pods `__ within Kubernetes.\n\nStart\n=====\n\n::\n\n ./worker/run.sh\n\nGet Logs\n========\n\n::\n\n ./worker/logs.sh\n\nSSH into the Worker\n===================\n\n::\n\n ./worker/ssh.sh\n\nDeploy AntiNex Core\n-------------------\n\nUse these commands to manage the `Backend AntiNex Core pods `__ within Kubernetes.\n\nStart\n=====\n\n::\n\n ./core/run.sh\n\nGet Logs\n========\n\n::\n\n ./core/logs.sh\n\nSSH into the API\n================\n\n::\n\n ./core/ssh.sh\n\nDeploy Jupyter\n--------------\n\nUse these commands to manage the `Jupyter pods `__ within Kubernetes.\n\nStart\n=====\n\n::\n\n ./jupyter/run.sh\n\nLogin to Jupyter\n================\n\nLogin with:\n\n- password: ``admin``\n\nhttps://jupyter.example.com\n\nGet Logs\n========\n\n::\n\n ./jupyter/logs.sh\n\nSSH into Jupyter\n================\n\n::\n\n ./jupyter/ssh.sh\n\nDeploy Splunk\n-------------\n\nUse these commands to manage the `Splunk container `__ within Kubernetes.\n\nStart\n=====\n\n::\n\n ./splunk/run.sh\n\nLogin to Splunk\n===============\n\nLogin with:\n\n- user: ``trex``\n- password: ``123321``\n\nhttps://splunk.example.com\n\nSearching in Splunk\n-------------------\n\nHere is the splunk searching command line tool I use with these included applications:\n\nhttps://github.com/jay-johnson/spylunking\n\nWith search example documentation:\n\nhttps://spylunking.readthedocs.io/en/latest/scripts.html#examples\n\nSearch using Spylunking\n-----------------------\n\nFind logs in splunk using the ``sp`` command line tool:\n\n::\n\n sp -q 'index=\"antinex\" | reverse' -u trex -p 123321 -a $(./splunk/get-api-fqdn.sh) -i antinex\n\nFind Django REST API Logs in Splunk\n-----------------------------------\n\n::\n\n sp -q 'index=\"antinex\" AND name=api | head 20 | reverse' -u trex -p 123321 -a $(./splunk/get-api-fqdn.sh) -i antinex\n\nFind Django Celery Worker Logs in Splunk\n----------------------------------------\n\n::\n\n sp -q 'index=\"antinex\" AND name=worker | head 20 | reverse' -u trex -p 123321 -a $(./splunk/get-api-fqdn.sh) -i antinex\n\nFind Core Logs in Splunk\n------------------------\n\n::\n\n sp -q 'index=\"antinex\" AND name=core | head 20 | reverse' -u trex -p 123321 -a $(./splunk/get-api-fqdn.sh) -i antinex\n\nFind Jupyter Logs in Splunk\n---------------------------\n\n::\n\n sp -q 'index=\"antinex\" AND name=jupyter | head 20 | reverse' -u trex -p 123321 -a $(./splunk/get-api-fqdn.sh) -i antinex\n\nExample for debugging ``sp`` splunk connectivity from inside an API Pod:\n\n::\n\n kubectl exec -it api-59496ccb5f-2wp5t -n default echo 'starting search' && /bin/bash -c \"source /opt/venv/bin/activate && sp -q 'index=\"antinex\" AND hostname=local' -u trex -p 123321 -a 10.101.107.205:8089 -i antinex\"\n\nGet Logs\n========\n\n::\n\n ./splunk/logs.sh\n\nSSH into Splunk\n===============\n\n::\n\n ./splunk/ssh.sh\n\nDeploy Nginx Ingress\n--------------------\n\nThis project is currently using the `nginx-ingress `__ instead of the `Kubernetes Ingress using nginx `__. Use these commands to manage and debug the nginx ingress within Kubernetes.\n\n.. note:: The default Yaml file annotations only work with the `nginx-ingress customizations `__\n\nStart\n=====\n\n::\n\n ./ingress/run.sh\n\nGet Logs\n========\n\n::\n\n ./ingress/logs.sh\n\nSSH into the Ingress\n====================\n\n::\n\n ./ingress/ssh.sh\n\nView Ingress Nginx Config\n-------------------------\n\nWhen troubleshooting the nginx ingress, it is helpful to view the nginx configs inside the container. Here is how to view the configs:\n\n::\n\n ./ingress/view-configs.sh\n\nView a Specific Ingress Configuration\n-------------------------------------\n\nIf you know the pod name and the namespace for the nginx-ingress, then you can view the configs from the command line with:\n\n::\n\n app_name=\"jupyter\"\n app_name=\"pgadmin\"\n app_name=\"api\"\n use_namespace=\"default\"\n pod_name=$(kubectl get pods -n ${use_namespace} | awk '{print $1}' | grep nginx | head -1)\n kubectl exec -it ${pod_name} -n ${use_namespace} cat /etc/nginx/conf.d/${use_namespace}-${app_name}-ingress.conf\n\nDeploy Splunk\n-------------\n\nStart\n=====\n\nTo deploy splunk you can add the argument ``splunk`` to the `./deploy-resources.sh splunk `__ script. Or you can manually run it with the command:\n\n::\n\n ./splunk/run.sh\n\nOr if you want to use Let's Encrypt for SSL:\n\n::\n\n ./splunk/run.sh prod\n\nDeploy Splunk-Ready Applications\n--------------------------------\n\nAfter deploying the splunk pod, you can deploy the splunk-ready applications with the command:\n\n::\n\n ./start.sh splunk\n\nGet Logs\n========\n\n::\n\n ./splunk/logs.sh\n\nSSH into Splunk\n===============\n\n::\n\n ./splunk/ssh.sh\n\nView Ingress Config\n===================\n\n::\n\n ./splunk/view-ingress-config.sh\n\nCreate your own self-signed x509 TLS Keys, Certs and Certificate Authority with Ansible\n---------------------------------------------------------------------------------------\n\nIf you have openssl installed you can use this ansible playbook to create your own certificate authority (CA), keys and certs.\n\n#. Create the CA, Keys and Certificates\n\n ::\n\n cd ansible\n ansible-playbook -i inventory_dev create-x509s.yml\n\n#. Check the CA, x509, keys and certificates for the client and server were created\n\n ::\n\n ls -l ./ssl\n\nDeploying Your Own x509 TLS Encryption files as Kubernetes Secrets\n------------------------------------------------------------------\n\nThis is a work in progress, but in ``dev`` mode the cert-manager is not in use. Instead the cluster utilizes pre-generated x509s TLS SSL files created with the `included ansible playbook create-x509s.yml `__. Once created, you can deploy them as Kubernetes secrets using the `deploy-secrets.sh `__ script and reload them at any time in the future.\n\nDeploy Secrets\n==============\n\nRun this to create the TLS secrets:\n\n::\n\n ./ansible/deploy-secrets.sh\n\nList Secrets\n============\n\n::\n\n kubectl get secrets | grep tls\n tls-ceph kubernetes.io/tls 2 36m\n tls-client kubernetes.io/tls 2 36m\n tls-database kubernetes.io/tls 2 36m\n tls-docker kubernetes.io/tls 2 36m\n tls-jenkins kubernetes.io/tls 2 36m\n tls-jupyter kubernetes.io/tls 2 36m\n tls-k8 kubernetes.io/tls 2 36m\n tls-kafka kubernetes.io/tls 2 36m\n tls-kibana kubernetes.io/tls 2 36m\n tls-minio kubernetes.io/tls 2 36m\n tls-nginx kubernetes.io/tls 2 36m\n tls-pgadmin kubernetes.io/tls 2 36m\n tls-phpmyadmin kubernetes.io/tls 2 36m\n tls-rabbitmq kubernetes.io/tls 2 36m\n tls-redis kubernetes.io/tls 2 36m\n tls-restapi kubernetes.io/tls 2 36m\n tls-s3 kubernetes.io/tls 2 36m\n tls-splunk kubernetes.io/tls 2 36m\n tls-webserver kubernetes.io/tls 2 36m\n\nReload Secrets\n==============\n\nIf you want to deploy new TLS secrets at any time, use the ``reload`` argument with the ``deploy-secrets.sh`` script. Doing so will delete the original secrets and recreate all of them using the new TLS values:\n\n::\n\n ./ansible/deploy-secrets.sh -r\n\nDeploy Cert Manager with Let's Encrypt\n--------------------------------------\n\nUse these commands to manage the `Cert Manager with Let's Encrypt SSL support `__ within Kubernetes. By default, the cert manager is deployed only in ``prod`` mode. If you run it in production mode, then it will install real, valid x509 certificates from `Let's Encrypt `__ into the nginx-ingress automatically.\n\nStart with Let's Encrypt x509 SSL Certificates\n==============================================\n\nStart the cert manager in ``prod`` mode to enable Let's Encrypt TLS Encryption with the command:\n\n::\n\n ./start.sh prod\n\nOr manually with the command:\n\n::\n\n ./cert-manager/run.sh prod\n\nIf you have splunk you can just add it to the arguments:\n\n::\n\n ./start.sh splunk prod\n\nView Logs\n=========\n\nWhen using the production mode, make sure to view the logs to ensure you are not being blocked due to rate limiting:\n\n::\n\n ./cert-manager/logs.sh\n\nStop the Cert Manager\n---------------------\n\nIf you notice things are not working correctly, you can quickly prevent yourself from getting blocked by stopping the cert manager with the command:\n\n::\n\n ./cert-manager/_uninstall.sh\n\n.. note:: If you get blocked due to rate-limits it will show up in the cert-manager logs like:\n\n ::\n\n I0731 07:53:43.313709 1 sync.go:273] Error issuing certificate for default/api.antinex.com-tls: error getting certificate from acme server: acme: urn:ietf:params:acme:error:rateLimited: Error finalizing order :: too many certificates already issued for exact set of domains: api.antinex.com: see https://letsencrypt.org/docs/rate-limits/\n E0731 07:53:43.313738 1 sync.go:182] [default/api.antinex.com-tls] Error getting certificate 'api.antinex.com-tls': secret \"api.antinex.com-tls\" not found\n\nDebugging\n=========\n\nTo reduce debugging issues, the cert manager ClusterIssuer objects use the same name for staging and production mode. This is nice because you do not have to update all the annotations to deploy on production vs staging:\n\nThe cert manager starts and defines the issuer name for both production and staging as: \n\n::\n\n --set ingressShim.defaultIssuerName=letsencrypt-issuer\n\nMake sure to set any nginx ingress annotations that need Let's Encrypt SSL encryption to these values:\n\n::\n\n annotations:\n kubernetes.io/tls-acme: \"true\"\n kubernetes.io/ingress.class: \"nginx\"\n certmanager.k8s.io/cluster-issuer: \"letsencrypt-issuer\"\n\nTroubleshooting\n---------------\n\nCustomize Minio and How to Troubleshoot\n---------------------------------------\n\nChange the Minio Access and Secret Keys\n=======================================\n\n#. Change the secrets file: ``minio/secrets/default_access_keys.yml``\n\n Change the ``access_key`` and ``secret_key`` values after generating the new base64 string values for the secrets file:\n\n ::\n\n echo -n \"NewAccessKey\" | base64\n TmV3QWNjZXNzS2V5\n # now you can replace the access_key's value in the secrets file with the string: TmV3QWNjZXNzS2V5\n\n ::\n\n echo -n \"NewSecretKey\" | base64\n TmV3U2VjcmV0S2V5\n # now you can replace the secret_key's value in the secrets file with the string: TmV3QWNjZXNzS2V5\n\n#. Deploy the secrets file\n\n ::\n\n kubectl apply -f ./minio/secrets/default_access_keys.yml\n\n#. Restart the Minio Pod\n\n ::\n\n kubectl delete pod -l app=minio\n\nIf you have changed the default access and secret keys, then you will need to export the following environment variables as needed to make sure the ``./minio/run_s3_test.py`` test script works:\n\n::\n\n export S3_ACCESS_KEY=\n export S3_SECRET_KEY=\n export S3_REGION_NAME=\n export S3_ADDRESS=\n # examples of setting up a minio env files are in: ./minio/envs\n\nView the Minio Dashboard\n========================\n\nLogin with:\n\n- access key: ``trexaccesskey``\n- secret key: ``trex123321``\n\nhttps://minio.example.com\n\nGet S3 Internal Endpoint\n========================\n\nIf you want to use the Minio S3 service within the cluster please use the endpoint:\n\n::\n\n minio-service:9000\n\nor source the internal environment file:\n\n::\n\n source ./minio/envs/int.env\n\nGet S3 External Endpoint\n========================\n\nIf you want to use the Minio S3 service from outside the cluser please use the endpoint provided by the script:\n\n::\n\n ./minio/get-s3-endpoint.sh\n # which for this documentation was the minio service's Endpoints:\n # 10.244.0.103:9000\n\nor source the external environment file:\n\n::\n\n source ./minio/envs/ext.env\n\nDebugging Steps\n===============\n\n#. Load the Minio S3 external environment variables:\n\n ::\n\n source ./minio/envs/ext.env\n\n#. Run the S3 Verification test script\n\n ::\n\n ./minio/run_s3_test.py\n\n#. Confirm Verification Keys are showing up in this Minio S3 bucket\n\n https://minio.example.com/minio/s3-verification-tests/\n\n If not please use the describe tools in ``./minio/describe-*.sh`` to grab the logs and `please file a GitHub issue `__\n\nDescribe Pod\n============\n\n::\n\n ./minio/describe-service.sh\n\nDescribe Service\n================\n\n::\n\n ./minio/describe-service.sh\n\nDescribe Ingress\n================\n\n::\n\n ./minio/describe-ingress.sh\n\nUninstall Minio\n===============\n\n::\n\n ./minio/_uninstall.sh\n\nCeph Troubeshooting\n-------------------\n\nPlease refer to the `Rook Common Issues `__ for the latest updates on how to use your Rook Ceph cluster.\n\n.. note:: By default Ceph is not hosting the S3 solution unless ``cephs3`` is passed in as an argument to ``deploy-resource.sh``.\n\nThere are included troubleshooting tools in the ``./rook`` directory with an overview of each below:\n\nValidate Ceph System Pods are Running\n=====================================\n\n::\n\n ./rook/view-system-pods.sh \n\n ----------------------------------------- \n Getting the Rook Ceph System Pods: \n kubectl -n rook-ceph-system get pod \n NAME READY STATUS RESTARTS AGE\n rook-ceph-agent-g9vzm 1/1 Running 0 7m\n rook-ceph-operator-78d498c68c-tbsdf 1/1 Running 0 7m\n rook-discover-h9wj9 1/1 Running 0 7m\n\nValidate Ceph Pods are Running\n==============================\n\n::\n\n ./rook/view-ceph-pods.sh \n\n ----------------------------------------- \n Getting the Rook Ceph Pods: \n kubectl -n rook-ceph get pod \n NAME READY STATUS RESTARTS AGE\n rook-ceph-mgr-a-9c44495df-7jksz 1/1 Running 0 6m\n rook-ceph-mon0-rxxsl 1/1 Running 0 6m\n rook-ceph-mon1-gqblg 1/1 Running 0 6m\n rook-ceph-mon2-7xfsq 1/1 Running 0 6m\n rook-ceph-osd-id-0-7d4d4c8794-kgr2d 1/1 Running 0 6m\n rook-ceph-osd-prepare-dev-kmsn9 0/1 Completed 0 6m\n rook-ceph-tools 1/1 Running 0 6m\n\nValidate Persistent Volumes are Bound\n=====================================\n\n::\n\n kubectl get pv\n NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE\n pvc-03e6e4ef-9df8-11e8-8047-0800270864a8 1Gi RWO Delete Bound default/certs-pv-claim rook-ceph-block 46m\n pvc-0415de24-9df8-11e8-8047-0800270864a8 1Gi RWO Delete Bound default/configs-pv-claim rook-ceph-block 46m\n pvc-0441307f-9df8-11e8-8047-0800270864a8 1Gi RWO Delete Bound default/datascience-pv-claim rook-ceph-block 46m\n pvc-0468ef73-9df8-11e8-8047-0800270864a8 1Gi RWO Delete Bound default/frontendshared-pv-claim rook-ceph-block 46m\n pvc-04888222-9df8-11e8-8047-0800270864a8 1Gi RWO Delete Bound default/staticfiles-pv-claim rook-ceph-block 46m\n pvc-1c3e359d-9df8-11e8-8047-0800270864a8 10Gi RWO Delete Bound default/minio-pv-claim rook-ceph-block 46m\n\nValidate Persistent Volume Claims are Bound\n===========================================\n\n::\n\n kubectl get pvc\n NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE\n certs-pv-claim Bound pvc-03e6e4ef-9df8-11e8-8047-0800270864a8 1Gi RWO rook-ceph-block 47m\n configs-pv-claim Bound pvc-0415de24-9df8-11e8-8047-0800270864a8 1Gi RWO rook-ceph-block 47m\n datascience-pv-claim Bound pvc-0441307f-9df8-11e8-8047-0800270864a8 1Gi RWO rook-ceph-block 47m\n frontendshared-pv-claim Bound pvc-0468ef73-9df8-11e8-8047-0800270864a8 1Gi RWO rook-ceph-block 47m\n minio-pv-claim Bound pvc-1c3e359d-9df8-11e8-8047-0800270864a8 10Gi RWO rook-ceph-block 46m\n\nCreate a Persistent Volume Claim\n================================\n\nGoing forward, Ceph will automatically create a persistent volume if one is not available for binding to an available Persistent Volume Claim. To create a new persistent volume, just create a claim and verify the Rook Ceph cluster created the persistent volume and both are bound to each other.\n\n::\n\n kubectl apply -f pvs/pv-staticfiles-ceph.yml\n\nVerify the Persistent Volume is Bound\n=====================================\n\n::\n\n kubectl get pv\n NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE\n pvc-77afbc7a-9ade-11e8-b293-0800270864a8 20Gi RWO Delete Bound default/staticfiles-pv-claim rook-ceph-block 2s\n\nVerify the Persistent Volume Claim is Bound\n===========================================\n\n::\n\n kubectl get pvc\n NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE\n staticfiles-pv-claim Bound pvc-77afbc7a-9ade-11e8-b293-0800270864a8 20Gi RWO rook-ceph-block 11s\n\n\nDescribe Persistent Volumes\n===========================\n\n::\n\n kubectl describe pv pvc-c88fc37b-9adf-11e8-9fae-0800270864a8\n Name: pvc-c88fc37b-9adf-11e8-9fae-0800270864a8\n Labels: \n Annotations: pv.kubernetes.io/provisioned-by=ceph.rook.io/block\n Finalizers: [kubernetes.io/pv-protection]\n StorageClass: rook-ceph-block\n Status: Bound\n Claim: default/certs-pv-claim\n Reclaim Policy: Delete\n Access Modes: RWO\n Capacity: 20Gi\n Node Affinity: \n Message: \n Source:\n Type: FlexVolume (a generic volume resource that is provisioned/attached using an exec based plugin)\n Driver: ceph.rook.io/rook-ceph-system\n FSType: xfs\n SecretRef: \n ReadOnly: false\n Options: map[clusterNamespace:rook-ceph image:pvc-c88fc37b-9adf-11e8-9fae-0800270864a8 pool:replicapool storageClass:rook-ceph-block]\n Events: \n\nShow Ceph Cluster Status\n========================\n\n::\n\n ./rook/show-ceph-status.sh \n\n ---------------------------------------------- \n Getting the Rook Ceph Status with Toolbox: \n kubectl -n rook-ceph exec -it rook-ceph-tools ceph status \n cluster:\n id: 7de1988c-03ea-41f3-9930-0bde39540552\n health: HEALTH_OK\n\n services:\n mon: 3 daemons, quorum rook-ceph-mon2,rook-ceph-mon0,rook-ceph-mon1\n mgr: a(active)\n osd: 1 osds: 1 up, 1 in\n\n data:\n pools: 1 pools, 100 pgs\n objects: 12 objects, 99 bytes\n usage: 35443 MB used, 54756 MB / 90199 MB avail\n pgs: 100 active+clean\n\nShow Ceph OSD Status\n====================\n\n::\n\n ./rook/show-ceph-osd-status.sh \n\n ---------------------------------------------- \n Getting the Rook Ceph OSD Status with Toolbox: \n kubectl -n rook-ceph exec -it rook-ceph-tools ceph osd status \n +----+-------------------------------------+-------+-------+--------+---------+--------+---------+-----------+\n | id | host | used | avail | wr ops | wr data | rd ops | rd data | state |\n +----+-------------------------------------+-------+-------+--------+---------+--------+---------+-----------+\n | 0 | rook-ceph-osd-id-0-7d4d4c8794-kgr2d | 34.6G | 53.4G | 0 | 0 | 0 | 0 | exists,up |\n +----+-------------------------------------+-------+-------+--------+---------+--------+---------+-----------+\n\nShow Ceph Free Space\n====================\n\n::\n\n ./rook/show-ceph-df.sh \n\n ---------------------------------------------- \n Getting the Rook Ceph df with Toolbox: \n kubectl -n rook-ceph exec -it rook-ceph-tools ceph df \n GLOBAL:\n SIZE AVAIL RAW USED %RAW USED \n 90199M 54756M 35443M 39.29 \n POOLS:\n NAME ID USED %USED MAX AVAIL OBJECTS \n replicapool 1 99 0 50246M 12 \n\nShow Ceph RDOS Free Space\n=========================\n\n::\n\n ./rook/show-ceph-rados-df.sh \n\n ---------------------------------------------- \n Getting the Rook Ceph rados df with Toolbox: \n kubectl -n rook-ceph exec -it rook-ceph-tools rados df \n POOL_NAME USED OBJECTS CLONES COPIES MISSING_ON_PRIMARY UNFOUND DEGRADED RD_OPS RD WR_OPS WR \n replicapool 99 12 0 12 0 0 0 484 381k 17 7168 \n\n total_objects 12\n total_used 35443M\n total_avail 54756M\n total_space 90199M\n\nOut of IP Addresses\n===================\n\nFlannel can exhaust all available ip addresses in the CIDR network range. When this happens please run the following command to clean up the local cni network files:\n\n::\n\n ./tools/reset-flannel-cni-networks.sh\n\nAntiNex Stack Status\n--------------------\n\nHere are the AntiNex repositories, documentation and build reports:\n\n.. list-table::\n :header-rows: 1\n\n * - Component\n - Build\n - Docs Link\n - Docs Build\n * - `REST API `__\n - .. image:: https://travis-ci.org/jay-johnson/train-ai-with-django-swagger-jwt.svg?branch=master\n :alt: Travis Tests\n :target: https://travis-ci.org/jay-johnson/train-ai-with-django-swagger-jwt.svg\n - `Docs `__\n - .. image:: https://readthedocs.org/projects/antinex/badge/?version=latest\n :alt: Read the Docs REST API Tests\n :target: https://readthedocs.org/projects/antinex/badge/?version=latest\n * - `Core Worker `__\n - .. image:: https://travis-ci.org/jay-johnson/antinex-core.svg?branch=master\n :alt: Travis AntiNex Core Tests\n :target: https://travis-ci.org/jay-johnson/antinex-core.svg\n - `Docs `__\n - .. image:: https://readthedocs.org/projects/antinex-core-worker/badge/?version=latest\n :alt: Read the Docs AntiNex Core Tests\n :target: http://antinex-core-worker.readthedocs.io/en/latest/?badge=latest\n * - `Network Pipeline `__\n - .. image:: https://travis-ci.org/jay-johnson/network-pipeline.svg?branch=master\n :alt: Travis AntiNex Network Pipeline Tests\n :target: https://travis-ci.org/jay-johnson/network-pipeline.svg\n - `Docs `__\n - .. image:: https://readthedocs.org/projects/antinex-network-pipeline/badge/?version=latest\n :alt: Read the Docs AntiNex Network Pipeline Tests\n :target: https://readthedocs.org/projects/antinex-network-pipeline/badge/?version=latest\n * - `AI Utils `__\n - .. image:: https://travis-ci.org/jay-johnson/antinex-utils.svg?branch=master\n :alt: Travis AntiNex AI Utils Tests\n :target: https://travis-ci.org/jay-johnson/antinex-utils.svg\n - `Docs `__\n - .. image:: https://readthedocs.org/projects/antinex-ai-utilities/badge/?version=latest\n :alt: Read the Docs AntiNex AI Utils Tests\n :target: http://antinex-ai-utilities.readthedocs.io/en/latest/?badge=latest\n * - `Client `__\n - .. image:: https://travis-ci.org/jay-johnson/antinex-client.svg?branch=master\n :alt: Travis AntiNex Client Tests\n :target: https://travis-ci.org/jay-johnson/antinex-client.svg\n - `Docs `__\n - .. image:: https://readthedocs.org/projects/antinex-client/badge/?version=latest\n :alt: Read the Docs AntiNex Client Tests\n :target: https://readthedocs.org/projects/antinex-client/badge/?version=latest\n\nReset Cluster\n-------------\n\nHere is a video showing how to reset the local Kubernetes cluster.\n\n.. image:: https://asciinema.org/a/193472.png\n :target: https://asciinema.org/a/193472?autoplay=1\n :alt: Reset Cluster\n\nPlease be careful as these commands will shutdown all containers and reset the Kubernetes cluster.\n\nRun as root:\n\n::\n\n sudo su\n kubeadm reset -f\n ./prepare.sh\n\nOr use the file:\n\n::\n\n sudo su\n ./tools/cluster-reset.sh\n\nOr the full reset and deploy once ready:\n\n::\n\n sudo su\n cert_env=dev; ./tools/reset-flannel-cni-networks.sh; ./tools/cluster-reset.sh ; ./user-install-kubeconfig.sh ; sleep 30; ./deploy-resources.sh splunk ${cert_env}\n exit\n # as your user\n ./user-install-kubeconfig.sh\n # depending on testing vs prod:\n # ./start.sh splunk\n # ./start.sh splunk prod\n\nDevelopment\n-----------\n\nRight now, the python virtual environment is only used to bring in ansible for running playbooks, but it will be used in the future with the kubernetes python client as I start using it more and more.\n\n::\n\n virtualenv -p python3 /opt/venv && source /opt/venv/bin/activate && pip install -e .\n\nTesting\n-------\n\n::\n\n py.test\n\nor\n\n::\n\n python setup.py test\n\nLicense\n-------\n\nApache 2.0 - Please refer to the LICENSE_ for more details\n\n.. _License: https://github.com/jay-johnson/deploy-to-kubernetes/blob/master/LICENSE\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jay-johnson/deploy-to-kubernetes", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "deploy-to-kubernetes", "package_url": "https://pypi.org/project/deploy-to-kubernetes/", "platform": "", "project_url": "https://pypi.org/project/deploy-to-kubernetes/", "project_urls": { "Homepage": "https://github.com/jay-johnson/deploy-to-kubernetes" }, "release_url": "https://pypi.org/project/deploy-to-kubernetes/1.0.8/", "requires_dist": [ "ansible", "colorlog", "coverage", "flake8 (<=3.4.1)", "pep8 (>=1.7.1)", "pipenv", "pycodestyle (<=2.3.1)", "pylint", "recommonmark", "sphinx", "sphinx-autobuild", "sphinx-rtd-theme", "spylunking", "unittest2", "mock" ], "requires_python": "", "summary": "Deployment tooling for managing a distributed AI stack on Kubernetes. Projects using this tool: AntiNex (https://antinex.readthedocs.io/en/latest/) and the Stock Analysis Engine (https://stock-analysis-engine.readthedocs.io/en/latest/) and works on Kubernetes 1.13.3", "version": "1.0.8" }, "last_serial": 4865705, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "204f56217f77677895dbc0b2c846f70c", "sha256": "3f13d2e97b5d5bce910e4dbcfdf840a12161bb7b5b0fc66bf91d05dbd268a88c" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "204f56217f77677895dbc0b2c846f70c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18127, "upload_time": "2018-11-02T15:32:24", "url": "https://files.pythonhosted.org/packages/0c/0e/2dce1875f6fa9e389f8304168c6f3de8bc4461e244140203d506375a2dac/deploy_to_kubernetes-1.0.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "79f049a3351d282e31a1ebbfd9ae01dc", "sha256": "6b687cc7cb790263d716162f2ca89c8f931170822682518a87aa2d8c62792800" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.0.tar.gz", "has_sig": false, "md5_digest": "79f049a3351d282e31a1ebbfd9ae01dc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38312, "upload_time": "2018-11-02T15:32:25", "url": "https://files.pythonhosted.org/packages/4b/48/1d10c3d44f0363024ca44ab76b5f69b6b29a7894885a059367d8a417c868/deploy-to-kubernetes-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "3b5735ec3d4e47b24eddc5b53701c1c0", "sha256": "979cbd82ee52334d744de707123ecca95f295ea20734d7a98fde2427c105a23a" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3b5735ec3d4e47b24eddc5b53701c1c0", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18181, "upload_time": "2018-12-07T23:40:17", "url": "https://files.pythonhosted.org/packages/6e/c6/1768fccc07441861890c24f0c7c316bb3e5f866e665f0bebfa081f5e69fc/deploy_to_kubernetes-1.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2e5aea22e0c99996f124e627f25a4a1c", "sha256": "9f48f2250a01c5e2025fcec7e675ed47c56f1da55ddbfff2d7c917451117de62" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.1.tar.gz", "has_sig": false, "md5_digest": "2e5aea22e0c99996f124e627f25a4a1c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38475, "upload_time": "2018-12-07T23:40:18", "url": "https://files.pythonhosted.org/packages/04/e1/29e67dd4589e1a8c7c5d441812dc4c244d87318a2da23a4b8dd8f4633d8d/deploy-to-kubernetes-1.0.1.tar.gz" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "84239af49c2cc51182402a69985170a7", "sha256": "9bc12aa52bb5aec1d683549c76456b0e1ae524b7e5d3030b0d938da31b1e17fb" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "84239af49c2cc51182402a69985170a7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18189, "upload_time": "2018-12-07T23:49:49", "url": "https://files.pythonhosted.org/packages/57/c9/b82afa9f8b3a63231375a568a2f64dee57a115a81325c44f59adb5b09466/deploy_to_kubernetes-1.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9c68a208a77991fc282286616c03c708", "sha256": "90996cb3d47ef137a1efae22c2fc01df34bfb1bdcaa004cac5e089d84fbbb2bc" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.2.tar.gz", "has_sig": false, "md5_digest": "9c68a208a77991fc282286616c03c708", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38506, "upload_time": "2018-12-07T23:49:51", "url": "https://files.pythonhosted.org/packages/6b/10/12470a381f26d1ae8a19a8ff470c2c8bb1a52a07f2e81217469c133c8ca7/deploy-to-kubernetes-1.0.2.tar.gz" } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "d6660f665a588f6a299a2edda8b8d577", "sha256": "ef05e1370b791d02cee7d28c0704b9e62d4cc563917e32e16cd84b2fe446cdb9" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d6660f665a588f6a299a2edda8b8d577", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18198, "upload_time": "2018-12-18T09:02:51", "url": "https://files.pythonhosted.org/packages/48/be/a7bd43fcd58b14da152e989ae31acaacba3fcb0e1c9b15c8c94106ce35d2/deploy_to_kubernetes-1.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9ab42ac632210378b7f457237aef4e4f", "sha256": "d1dcf1ba049b1e13e64a8f8fb9db3a2ccab8bb926d6f024c5d85ed299e97ef10" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.3.tar.gz", "has_sig": false, "md5_digest": "9ab42ac632210378b7f457237aef4e4f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38406, "upload_time": "2018-12-18T09:02:52", "url": "https://files.pythonhosted.org/packages/27/43/eb783f41dd1a2f63d5db54d7a9f6c24fb6d578b8287f09eadf8d9d19ad78/deploy-to-kubernetes-1.0.3.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "93be528a49c80622ce9e3130cbb4e4ee", "sha256": "2445ef07305d13ba5234d3534b4f503aa6ae3e9f8e8c49607dca6886ec0e9165" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "93be528a49c80622ce9e3130cbb4e4ee", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18198, "upload_time": "2018-12-18T22:35:25", "url": "https://files.pythonhosted.org/packages/bb/61/1a7bcd661bc3a7af5748fca64ece223eb16bf24c410fff21724ef6781ed6/deploy_to_kubernetes-1.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "db98703e4aadb67ddcd9dc84149a8fe5", "sha256": "de3ee2bb4bcbd0dccbfef2cf8606ae13fb7f9e2760709c7205905df0067c3c56" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.4.tar.gz", "has_sig": false, "md5_digest": "db98703e4aadb67ddcd9dc84149a8fe5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38410, "upload_time": "2018-12-18T22:35:27", "url": "https://files.pythonhosted.org/packages/75/38/0f0e4b6323fb027b0caf5fd5c42e568ac4df5c31c546454fa78dc056b270/deploy-to-kubernetes-1.0.4.tar.gz" } ], "1.0.5": [ { "comment_text": "", "digests": { "md5": "8239883e6334f1f4a33855c9010b8d0c", "sha256": "be3e1ad339436ce5496e300df19efedb7a6572e3e99eebeb8b1d4f90389f4687" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "8239883e6334f1f4a33855c9010b8d0c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18220, "upload_time": "2019-02-08T16:46:24", "url": "https://files.pythonhosted.org/packages/31/54/eca4de51945658f9970030b16589e8aa57d2955624d66289c1fbd3d858a4/deploy_to_kubernetes-1.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c5743f28dd8685d8636a5a7cb4bfcd6a", "sha256": "ac37f911450a7c4e4a5a4f12af10d2b227048c7b185e5f8c044a898ef40b9848" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.5.tar.gz", "has_sig": false, "md5_digest": "c5743f28dd8685d8636a5a7cb4bfcd6a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38629, "upload_time": "2019-02-08T16:46:25", "url": "https://files.pythonhosted.org/packages/ef/ab/9ed3f7f9379ae62d1fb7261979a317d91ab8dc74c4612953ae59a4961bf1/deploy-to-kubernetes-1.0.5.tar.gz" } ], "1.0.6": [ { "comment_text": "", "digests": { "md5": "26e215f99794efaf758e7313441563ea", "sha256": "aac5e2b8013125ca3de58f586df543fd32c45178ae50abba8819917d2adc57fd" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "26e215f99794efaf758e7313441563ea", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18255, "upload_time": "2019-02-25T08:34:28", "url": "https://files.pythonhosted.org/packages/f9/66/d2cfd45b4d14c8dd3b8e770795640e8f20d8c7087a9dea1c6aaacafaf923/deploy_to_kubernetes-1.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d7fba51090cd0f873c509d3c8b9d35ee", "sha256": "19e6e51df843ef4f1a5a5abcdef075d65974d2aa417b2839626d17e945e2abd2" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.6.tar.gz", "has_sig": false, "md5_digest": "d7fba51090cd0f873c509d3c8b9d35ee", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38615, "upload_time": "2019-02-25T08:34:29", "url": "https://files.pythonhosted.org/packages/dd/92/70fa49b2ad304946ab726ef95ad8214207754c14b04c2e037e51af31029c/deploy-to-kubernetes-1.0.6.tar.gz" } ], "1.0.7": [ { "comment_text": "", "digests": { "md5": "1a4e7eb19e6960904f8f34bad1b860e9", "sha256": "ecf8b28dcced0829f8071f936902ef4448093d93bc72d4e9f27257541358852a" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "1a4e7eb19e6960904f8f34bad1b860e9", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18254, "upload_time": "2019-02-25T16:35:07", "url": "https://files.pythonhosted.org/packages/8d/f4/f0b56ea6f7adbbc122357b01b8a953b338208cdb2ef5a677ce2b93d0dd50/deploy_to_kubernetes-1.0.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "50ac2008c4663699d61fd6f2095e14b9", "sha256": "7336ca2b62ffe04bc8d383738b21d92b9d05612fface9a38e945aab6393fb8f4" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.7.tar.gz", "has_sig": false, "md5_digest": "50ac2008c4663699d61fd6f2095e14b9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38614, "upload_time": "2019-02-25T16:35:08", "url": "https://files.pythonhosted.org/packages/bb/d7/e76b5193100660764277d5b5225c41d70ebf4b97c8163abf77c6efcd0906/deploy-to-kubernetes-1.0.7.tar.gz" } ], "1.0.8": [ { "comment_text": "", "digests": { "md5": "a86bbc0ce89abb2ee175417ac198636c", "sha256": "ee0c1a68e528a085723bca849af8fa1aa264e1161743b130f9163eb86196a5e0" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a86bbc0ce89abb2ee175417ac198636c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18250, "upload_time": "2019-02-25T16:59:36", "url": "https://files.pythonhosted.org/packages/b4/f4/3e9f0469a3ee433a4162e5bc6d21c05b6be73c0db49a6b4caf8912e36886/deploy_to_kubernetes-1.0.8-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "11bc4e3fe9bcf8ed4b5fd48507d7d35e", "sha256": "b0c25f0961dd3d3ff83edb1931a31279a1649584ba6c64496f0959c90dc27bd2" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.8.tar.gz", "has_sig": false, "md5_digest": "11bc4e3fe9bcf8ed4b5fd48507d7d35e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38602, "upload_time": "2019-02-25T16:59:38", "url": "https://files.pythonhosted.org/packages/ea/b9/c20d9bec057c4651c4013b48cb8a4fd58d6b8caed66bce765779e1b14975/deploy-to-kubernetes-1.0.8.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a86bbc0ce89abb2ee175417ac198636c", "sha256": "ee0c1a68e528a085723bca849af8fa1aa264e1161743b130f9163eb86196a5e0" }, "downloads": -1, "filename": "deploy_to_kubernetes-1.0.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a86bbc0ce89abb2ee175417ac198636c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18250, "upload_time": "2019-02-25T16:59:36", "url": "https://files.pythonhosted.org/packages/b4/f4/3e9f0469a3ee433a4162e5bc6d21c05b6be73c0db49a6b4caf8912e36886/deploy_to_kubernetes-1.0.8-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "11bc4e3fe9bcf8ed4b5fd48507d7d35e", "sha256": "b0c25f0961dd3d3ff83edb1931a31279a1649584ba6c64496f0959c90dc27bd2" }, "downloads": -1, "filename": "deploy-to-kubernetes-1.0.8.tar.gz", "has_sig": false, "md5_digest": "11bc4e3fe9bcf8ed4b5fd48507d7d35e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38602, "upload_time": "2019-02-25T16:59:38", "url": "https://files.pythonhosted.org/packages/ea/b9/c20d9bec057c4651c4013b48cb8a4fd58d6b8caed66bce765779e1b14975/deploy-to-kubernetes-1.0.8.tar.gz" } ] }