{ "info": { "author": "Jim Fan", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Programming Language :: Python :: 3", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "Tensorplex: distributed Tensorboard and distributed logging\n===========================================================\n\nTensorplex is a multiplexed extension of the popular Tensorboard\nvisualization tool. When you have a cluster, you can collect the\nlearning curves from multiple running nodes and display them\nside-by-side on a single tensorboard web page.\n\nTensorplex makes extensive use of ZeroMQ under the hood, an efficient,\nrobust, and lightweight distributed communication protocol.\n\n``Loggerplex`` is a subcomponent of Tensorplex that does lightweight\ndistributed logging. It collects the real-time logs from multiple nodes\nand send them to a single master node for persistent book-keeping.\n\nTensorplex is not tied to Tensorflow and can be used with any machine\nlearning frameworks that support numpy.\n\nTHIS DOC IS INCOMPLETE. MORE COMING SOON.\n\nInstallation\n------------\n\n.. code:: bash\n\n git clone https://github.com/StanfordVL/Tensorplex.git\n pip install -e Tensorplex/\n\nDemo\n----\n\nGo to ``Tensorplex/examples/``. Change the tensorboard log folder in\n``run_server.py`` script.\n\nIn one command line window, run ``python run_server.py``. Then in\nanother window, run ``python run_client.py``. The server script should\nprint out a list of ``dones``.\n\nUse ``tensorboard --logdir ~/Temp/tensorplex/ --port 8009`` to view the\nresults.\n\nManual\n------\n\nTensorplex requires one long-running server script. Client scripts can\nconnect and disconnect (e.g. client crashes) without impacting the\nserver.\n\nTensorplex server\n~~~~~~~~~~~~~~~~~\n\nThere are 3 steps to create the server script.\n\nFirst, initialize a ``Tensorplex`` object with the root logging folder.\nDifferent clients will write to different sub-folders that are created\nautomatically. ``max_processes`` is the number of processes that the\nserver uses internally. Set it to 4 should be a sweet spot.\n\n.. code:: python\n\n\n tplex = Tensorplex(\n '~/my/logging/folder',\n max_processes=4,\n )\n\nSecond, register the client groups, which helps group Tensorflow curves\ninto the same or different graph windows. The \"client IDs\" (explained\nlater) in your client scripts must be consistent with the groups you\nregister in the server.\n\nThere are 3 types of client groups:\n\n1. ``register_normal_group(name)``: each graph will have only one curve\n in a normal group.\n2. ``register_indexed_group(name, bin_size)``: each graph will have at\n most ``bin_size`` number of curves. Suppose you launch 42 agents with\n ``bin_size=10``, the curves of agent 0-9 will be displayed in the\n same graph window; likewise, the curves of 10-19, 20-29, 30-39, 40-41\n will be grouped in their respective graphs.\n3. ``register_combined_group(name, group_criterion)``: TODO\n\nTo register multiple groups, you can chain the commands:\n\n.. code:: python\n\n (tplex\n .register_normal_group('learner') # 1 curve per graph\n .register_indexed_group('agent', 8) # 8 agent learning curves per graph\n .register_indexed_group('eval', 4) # 4 eval curves per graph\n .register_combined_group('eval', get_eval_bin_name)\n )\n\nThird, you specify a port and launch the server. The script will be\nblocking:\n\n.. code:: python\n\n tplex.start_server(8008)\n # block main thread forever\n\nTensorplex client\n~~~~~~~~~~~~~~~~~\n\nEvery ``TensorplexClient`` object must have a client ID that looks like\n``/``, i.e. two string names separated by\n``/``.\n\n.. code:: python\n\n client = TensorplexClient(\n client_id='agent/0', # for indexed group\n # client_id='learner/system_stats', # for normal group, here client_name is `system_stats`\n host='123.45.6.7', # server address to connect\n port=8008, # server port to connect\n )\n\nThen you can write statistics to TensorplexClient\n\n.. code:: python\n\n # most fundamental method\n client.add_scalar(tag, 3.1415, integer_step)\n # add_scalars is equivalent to multiple add_scalar() in one line\n client.add_scalars({tag: 3.1415, tag2: 2.71828, tag3: 42}, integer_step)\n\nThere are\n\nNote that ``tag`` in ``add_scalar`` behaves differently for different\nclient group types.\n\nFor normal group,\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/SurrealAI/Tensorplex", "keywords": "Machine Learning,Distributed Computing", "license": "GPLv3", "maintainer": "", "maintainer_email": "", "name": "tensorplex", "package_url": "https://pypi.org/project/tensorplex/", "platform": "", "project_url": "https://pypi.org/project/tensorplex/", "project_urls": { "Homepage": "http://github.com/SurrealAI/Tensorplex" }, "release_url": "https://pypi.org/project/tensorplex/0.9.post1/", "requires_dist": null, "requires_python": ">=3.5", "summary": "Distributed wrapper for Tensorboard", "version": "0.9.post1" }, "last_serial": 4414325, "releases": { "0.0.0": [ { "comment_text": "", "digests": { "md5": "80ba1c10209b62163f21859351b3d526", "sha256": "c82aa3deff9b593bd0c7d9caa2202c5402992d4aa86acc9456fc00a7b42d97d7" }, "downloads": -1, "filename": "tensorplex-0.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "80ba1c10209b62163f21859351b3d526", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 2015, "upload_time": "2017-11-17T18:35:41", "url": "https://files.pythonhosted.org/packages/97/9a/c2650365ccb86d5c4825d29bf6f49eb2891bb2955eb66bcf64ba8f46388e/tensorplex-0.0.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "363ffda39a0e0bf36f25a8417ff457fb", "sha256": "46a787b42b9741bc2ac013eb7210857b3a30d6aaad510fc0c5a935bf3354dc4a" }, "downloads": -1, "filename": "tensorplex-0.0.0.tar.gz", "has_sig": false, "md5_digest": "363ffda39a0e0bf36f25a8417ff457fb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1035, "upload_time": "2017-11-17T18:35:42", "url": "https://files.pythonhosted.org/packages/4e/af/1486ace5e2e84c988cb3ed30fcf8f6b0d5bf009fafb66330e619e4ede786/tensorplex-0.0.0.tar.gz" } ], "0.9": [ { "comment_text": "", "digests": { "md5": "822c47f835bc7927581a4906cb110306", "sha256": "99fbd8934b05c3c0900765ac4769f31d21dd2f12dd904eca59652f3719f5ce1a" }, "downloads": -1, "filename": "Tensorplex-0.9-py3-none-any.whl", "has_sig": false, "md5_digest": "822c47f835bc7927581a4906cb110306", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 27003, "upload_time": "2018-10-14T01:03:57", "url": "https://files.pythonhosted.org/packages/c7/24/fd3592191572b72d853856c4b3be1f666263574acf9ea4ed946aa145b84f/Tensorplex-0.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9ac18076176410ca14e15014264e343d", "sha256": "2990abb78c3a8f8c5cb42763e8e3275bcca3a659043aa211fbc796ccf154e371" }, "downloads": -1, "filename": "Tensorplex-0.9.tar.gz", "has_sig": false, "md5_digest": "9ac18076176410ca14e15014264e343d", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 22172, "upload_time": "2018-10-14T01:03:59", "url": "https://files.pythonhosted.org/packages/33/2b/3a9b5e3775f2561205625c753c45cf432457c3d48829d7435508991b2ee2/Tensorplex-0.9.tar.gz" } ], "0.9.post1": [ { "comment_text": "", "digests": { "md5": "d38bc309ca1bc70e35e91457387c8fa0", "sha256": "754385c2c9f85e02243d12377b2bab041f5d25dde79f55b6d4268e4ab91fa8f7" }, "downloads": -1, "filename": "Tensorplex-0.9.post1-py3-none-any.whl", "has_sig": false, "md5_digest": "d38bc309ca1bc70e35e91457387c8fa0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 27062, "upload_time": "2018-10-14T01:06:08", "url": "https://files.pythonhosted.org/packages/6c/65/c84034380e9c58fd8aabded39f7ee8bdc770b1cbb9ea21e03d107e5fd411/Tensorplex-0.9.post1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "437f5f151604aa585440c19d1c0b6c73", "sha256": "37c44fa6618bb7f0fc5be236ef2dbfb891c81ddca379630198170709129a2170" }, "downloads": -1, "filename": "Tensorplex-0.9.post1.tar.gz", "has_sig": false, "md5_digest": "437f5f151604aa585440c19d1c0b6c73", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 22182, "upload_time": "2018-10-14T01:06:09", "url": "https://files.pythonhosted.org/packages/f0/eb/d94ee47b8b307294aa1439abe33c41ab8455c4b7eb150d5d84330cea3054/Tensorplex-0.9.post1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d38bc309ca1bc70e35e91457387c8fa0", "sha256": "754385c2c9f85e02243d12377b2bab041f5d25dde79f55b6d4268e4ab91fa8f7" }, "downloads": -1, "filename": "Tensorplex-0.9.post1-py3-none-any.whl", "has_sig": false, "md5_digest": "d38bc309ca1bc70e35e91457387c8fa0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 27062, "upload_time": "2018-10-14T01:06:08", "url": "https://files.pythonhosted.org/packages/6c/65/c84034380e9c58fd8aabded39f7ee8bdc770b1cbb9ea21e03d107e5fd411/Tensorplex-0.9.post1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "437f5f151604aa585440c19d1c0b6c73", "sha256": "37c44fa6618bb7f0fc5be236ef2dbfb891c81ddca379630198170709129a2170" }, "downloads": -1, "filename": "Tensorplex-0.9.post1.tar.gz", "has_sig": false, "md5_digest": "437f5f151604aa585440c19d1c0b6c73", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 22182, "upload_time": "2018-10-14T01:06:09", "url": "https://files.pythonhosted.org/packages/f0/eb/d94ee47b8b307294aa1439abe33c41ab8455c4b7eb150d5d84330cea3054/Tensorplex-0.9.post1.tar.gz" } ] }