{ "info": { "author": "Sense Inc.", "author_email": "support@senseplatform.com", "bugtrack_url": null, "classifiers": [], "description": "Utilities for `IPython `__ on `Sense `__\n=======================================================================================\n\n|Build Status|\n\nThis package complements Sense's `REST\nAPI `__ by wrapping and\nsimplifying some of the most common operations, such as launching and\nstopping worker dashboards. Its primary purpose is to support other\npackages that implement higher-level approaches to cluster computing.\n\nInstallation\n------------\n\nThis package is preinstalled on Sense. You can import it with\n\n.. code:: python\n\n import sense\n\nTo install it elsewhere use\n\n.. code:: bash\n\n pip install sense\n\nExample\n-------\n\nThis example launches several worker dashboards and communicates with\nthem using\n`ZeroMQ `__\nover the project's private network.\n\n.. code:: python\n\n import time\n import sense\n\n n_workers = 3\n\n # Use 'install' to install package 'pyzmq' from PyPI to the project.\n sense.install('pyzmq')\n import zmq\n\n # Create the ZeroMQ server.\n context = zmq.Context()\n socket = context.socket(zmq.REP)\n\n # Use 'get_network_info' to find out the private IP address of the current\n # dashboard in the project's virtual private network.\n address = \"tcp://\" + sense.network_info()['project_ip'] + \":5000\"\n socket.bind(address)\n\n # Define code the worker dashboards should execute on startup.\n # Each worker will attempt to connect to the ZeroMQ server whose \n # address is stored in its 'SERVER' environment variable and then will send\n # a message to the server.\n worker_code = \"\"\"\n import os\n # Because pyzmq was previously installed to the project, workers don't\n # need to reinstall it.\n import zmq\n import sense\n\n # Connect to the master\n context = zmq.Context()\n print \"Connecting to master...\"\n socket = context.socket(zmq.REQ)\n socket.connect(os.environ['SERVER'])\n\n # Send a message\n socket.send(\"Sense is so easy!\")\n\n # Wait for a reply\n message = socket.recv()\n print \"Received reply: \", message\n \"\"\"\n\n # Use 'launch_workers' to start three small worker dashboards. The above\n # code is sent to each, and the current dashboard's project IP address is\n # stored in each worker's environment as 'SERVER', so each worker will contact\n # the current dashboard.\n workers = sense.launch_workers(n=n_workers, \n size=0, \n code=worker_code, \n env={\"SERVER\": address})\n\n # Listen for worker messages.\n for i in range(0, n_workers):\n # Wait for next request from client\n message = socket.recv()\n print \"Received request: \", message\n time.sleep (1) \n socket.send(\"I agree.\")\n\n sense.stop_workers()\n\nAPI\n---\n\ninstall\n~~~~~~~\n\nInstalls a Python package to the project with\n`pip `__ using the `user\nscheme `__.\n\n.. code:: python\n\n import sense\n sense.install(package_name, flags=[], arguments={})\n\nIf you prefer, you can also install packages by running a shell command\nfrom IPython using the ! prefix:\n\n::\n\n !pip install pyzmq --user\n\nOptions:\n\n- **flags**: A list of strings to pass to pip as flags. For example,\n ``[\"U\", \"use-mirrors\"]`` would translate to the command-line flags\n ``-U --use-mirrors``.\n- **arguments**: A dict containing arguments to pass to pip. For\n example, ``{\"d\": \"./downloads\", \"mirrors\": \"http://URL\"}`` would\n translate to the command-line arguments\n ``-d ./downloads --mirrors=http://URL``.\n\nOnce installed, any of the project's dashboards can import the package.\n\nnetwork\\_info\n~~~~~~~~~~~~~\n\nReturns the current dashboard's contact information in a dict with keys\n``public_dns``, ``public_port_mapping``, ``ssh_password`` and\n``project_ip``.\n\n.. code:: python\n\n import sense\n network_info = sense.network_info()\n\nEvery project has its own `virtual private network\n(VPN). `__ The\nproject IP address is bound to the project VPN and is only accessible to\nother dashboards in the same project. The project VPN makes it easy to\nuse cluster computing frameworks that lack built-in security features,\nsuch as\n`MPI `__.\n\nThe public DNS hostname, public port mapping and SSH password describe\nhow the current dashboard can be contacted from outside the project. The\npublic port mapping is a dict whose keys and values are integers. Only\nports that are keys of the public port mapping can be accessed via the\npublic DNS hostname. If you run a service on port 3000, for example, it\ncan be accessed from anywhere on the internet on the public DNS hostname\nand port ``public_port_mapping[3000]``.\n\nIf required, you can SSH to dashboards using the public DNS hostname and\nport ``public_port_mapping[22]`` with username \"sense\" and the SSH\npassword.\n\nlaunch\\_workers\n~~~~~~~~~~~~~~~\n\nLaunches worker dashboards into the cluster.\n\n.. code:: python\n\n import sense\n worker_info = sense.launch_workers(n, \n size=0, \n engine=\"sense-ipython-engine\", \n script=\"\", \n code=\"\", \n env={})\n\nIn Sense, a cluster is a group of dashboards with the same master\ndashboard. Worker dashboards multiplex their outputs to the master and\nare cleaned up automatically when the master is stopped or fails. These\nfeatures make it easy to manage, monitor and debug distributed\napplications on Sense.\n\nThe parameters are:\n\n- **n**: The number of workers to launch.\n- **size** (optional): The size of the workers, 0 to 16.\n- **engine** (optional): The name of the `npm `__\n module to use as the engine. Defaults to \"sense-ipython-engine\", but\n workers can run other engines too.\n- **script** (optional): A script file that the worker should execute\n on launch. The path is relative to the project's home folder.\n- **code** (optional): Code that the worker should execute on launch.\n If both are provided, script has precedence over code.\n- **env** (optional): A dict containing environment variables that\n should be set on the workers before any code is executed. This is the\n preferred way to send a master's contact information information to\n workers.\n\nThe return value is a list of dicts. Each dict describes one of the\nworkers that was launched and contains keys such as ``\"id\"``,\n``\"engine\"``, ``\"status\"``, etc. The full format is documented\n`here. `__\n\nlist\\_workers\n~~~~~~~~~~~~~\n\nReturns information on the worker dashboards in the cluster in a list of\ndicts like those returned by launch\\_workers.\n\n.. code:: python\n\n import sense\n worker_info = sense.list_workers()\n\nget\\_master\n~~~~~~~~~~~\n\nReturns information on the cluster's master dashboard in a dict like the\nones returned by launch\\_workers.\n\n.. code:: python\n\n import sense\n master_info = sense.get_master()\n\nstop\\_workers\n~~~~~~~~~~~~~\n\nStops worker dashboards.\n\nDashboards' numerical IDs are available at key ``id`` in the dicts\nreturned by list\\_workers and launch\\_workers. The return value is a\ndict of the same type.\n\n.. code:: python\n\n import sense\n\n # To stop specific workers:\n worker_info = sense.stop_workers(id1, id2, ...)\n\n # To stop all workers in the cluster:\n worker_info = sense.stop_workers()\n\nget\\_auth\n~~~~~~~~~\n\nReturns authentication information for the `REST\nAPI `__ as a dict with keys\n``\"user\"`` and ``\"password\"``.\n\nSense's REST API gives you complete programmatic control over virtually\nevery aspect of Sense. Most REST calls require `Basic\nAuthentication `__.\nTo make authenticated REST calls, supply the information returned by\nget\\_auth your HTTP client of choice, such as the Python\n`requests `__ package.\n\nBy default get\\_auth uses the environment variable SENSE\\_API\\_TOKEN for\nauthentication. This token restricts access to the current project. For\naccess across projects, you can pass in credentials manually or set\nSENSE\\_USERNAME and SENSE\\_PASSWORD in the environment. To better\nunderstand these options, read the `Understanding Project\nSecurity `__\ndocumentation.\n\nAuthenticated REST Example\n^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThis example retrieves information about the current project:\n\n.. code:: python\n\n import sense\n import requests\n auth = sense.get_auth()\n url = \"https://api.senseplatform.com/users/\" + os.environ[\"SENSE_OWNER_ID\"] + \\\n \"/projects/\" + os.environ[\"SENSE_PROJECT_ID\"]\n response = requests.get(url, auth=(auth[\"user\"], auth[\"password\"])).json()\n\nThe environment variables used in this example are common to all\ndashboards, across all engines, and are documented\n`here `__.\n\nRich Dashboard Output\n---------------------\n\nIPython provides its own `rich display\nsystem `__,\nso unlike its `R `__ and\n`JavaScript `__\ncounterparts this package does not provide any rich output functions.\nUse IPython's rich display system to display HTML, images and more in a\ndashboard.\n\nSupport\n=======\n\n- **Documentation**: http://help.senseplatform.com\n- **Email**: support@senseplatform.com\n- **Twitter**: https://twitter.com/senseplatform\n- **Google Group**:\n https://groups.google.com/forum/?fromgroups#!forum/sense-users\n- **IRC**: ``#senseplatform`` on ``irc.freenode.net``\n\nLicense\n=======\n\nMIT\n\n.. |Build Status| image:: https://travis-ci.org/SensePlatform/sense-ipython-module.png\n :target: https://travis-ci.org/SensePlatform/sense-ipython-module\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://www.senseplatform.com", "keywords": null, "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "sense", "package_url": "https://pypi.org/project/sense/", "platform": "Sense", "project_url": "https://pypi.org/project/sense/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://www.senseplatform.com" }, "release_url": "https://pypi.org/project/sense/0.1.2/", "requires_dist": null, "requires_python": null, "summary": "Standard Sense utilities for Python", "version": "0.1.2" }, "last_serial": 907701, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "7038ca9e3659c1fa1fca5ebf61c812eb", "sha256": "a880f61ae6a112833bfcb8cedad1f393314f77573ce256b1ea795c6b4e5cc0f4" }, "downloads": -1, "filename": "sense-0.1.tar.gz", "has_sig": false, "md5_digest": "7038ca9e3659c1fa1fca5ebf61c812eb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3273, "upload_time": "2013-10-30T19:55:33", "url": "https://files.pythonhosted.org/packages/5e/c3/26fae06a43dfa5a16be7ddca3d06606f082d2866e4c9d7b3a57bfb2633d3/sense-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "43ab43ab66cda161318567cda2785222", "sha256": "cec5404a449e865f6723b29a2c4dc45ed5ea4de7922f7a5f687b864c495bac0e" }, "downloads": -1, "filename": "sense-0.1.1.tar.gz", "has_sig": false, "md5_digest": "43ab43ab66cda161318567cda2785222", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6661, "upload_time": "2013-10-30T20:21:18", "url": "https://files.pythonhosted.org/packages/6d/80/e56c01ee8ba0c8d8871a073c22609704d7b155b1f19d51ee084a086ee1f3/sense-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "1d6f88c7dd59af2a7446cf1895d2b39a", "sha256": "ac6a4c209a0c8464388434aa2bb2164ac1e56b5f15ae73bb941a73ec39799e54" }, "downloads": -1, "filename": "sense-0.1.2.tar.gz", "has_sig": false, "md5_digest": "1d6f88c7dd59af2a7446cf1895d2b39a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6730, "upload_time": "2013-10-30T20:29:47", "url": "https://files.pythonhosted.org/packages/c5/05/56eb81f640f37290f899d15915e9cd81e9dba693dc294366099040804c00/sense-0.1.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1d6f88c7dd59af2a7446cf1895d2b39a", "sha256": "ac6a4c209a0c8464388434aa2bb2164ac1e56b5f15ae73bb941a73ec39799e54" }, "downloads": -1, "filename": "sense-0.1.2.tar.gz", "has_sig": false, "md5_digest": "1d6f88c7dd59af2a7446cf1895d2b39a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6730, "upload_time": "2013-10-30T20:29:47", "url": "https://files.pythonhosted.org/packages/c5/05/56eb81f640f37290f899d15915e9cd81e9dba693dc294366099040804c00/sense-0.1.2.tar.gz" } ] }