{ "info": { "author": "Nextdoor Engineering", "author_email": "eng@nextdoor.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Topic :: Software Development" ], "description": "# Nextdoor Service Registry\n\n*nd_service_registry* is a Python module that provides a simple way to leverage\n*Apache Zookeeper* as a dynamic configuration and service registry.\n\nThe goal of the package is to provide a single foundational class that can be\nleveraged by any Python program for registering and monitoring services through\nZookeeper.\n\nThe current use cases are:\n\n * Register a server providing a service (and subsequently retrieve them)\n * Store a simple dict/key value in Zookeeper (and retreive it)\n * Execute callback methods whenever any node list or data setting changes\n * Get a temporary (but global!) lock while you do something\n\nThe main benefit of using this module is if you have several different tools\nin your network that all need to interact with Zookeeper in a common way. The\nmost common functions are handled in this singular module allowing you to focus\non your own app development more.\n\n## Installation\n\nTo install, run\n\n python setup.py install\n\nor\n\n pip install nd_service_registry\n\n## Usage\n\n### Instantiating KazooServiceRegistry\n\nCreate a logger object\n\n >>> import logging\n >>> logging.basicConfig(level=logging.DEBUG)\n\nTo create your initial connection object\n\n >>> import nd_service_registry\n >>> nd = nd_service_registry.KazooServiceRegistry()\n\nThe KazooServiceRegistry object is a child of nd\\_service\\_registry that\nconforms to our ServiceRegistry specs, while leveraging Kazoo as the backend.\nThe object handles all of your connection states - there is no need to\nstart/stop or monitor the connection state at all.\n\n### Registering Data\n\n#### Ephemeral Nodes\n\nRegistering a node in Zookeeper thats Ephemeral (disappears when the node\ngoes offline, or if the connection is lost) is highly useful for keeping\ntrack of which servers are offering specific services.\n\n >>> nd.set_node('/production/ssh/server1:22')\n >>> nd.set_node('/production/ssh/server2:22')\n >>> nd.set_node('/production/ssh/server3:22')\n >>> nd.set_node('/production/web/server2:22',\n data={'type': 'apache'})\n >>> nd.get('/production/ssh')\n {'children': [u'server1:22', u'server2:22', u'server3:22' ],\n 'data': None,\n 'path': '/production/ssh',\n 'stat': ZnodeStat(czxid=27, mzxid=27, ctime=1355533229452,\n mtime=1355533229452, version=0, cversion=5,\n aversion=0, ephemeralOwner=0, dataLength=0,\n numChildren=3, pzxid=45)}\n\n\nTo register the host as providing a particular service\n\n >>> nd.set_node('/services/ssh/server1:22', data={ 'foo': 'bar'})\n\nGetting a list of servers at a path\n\n >>> nd.get('/services/ssh')\n {'children': {u'server1:22': {u'foo': u'bar',\n u'created': u'2012-12-15 00:45:03',\n u'pid': 10733}},\n 'data': None,\n 'path': '/services/ssh',\n 'stat': ZnodeStat(czxid=6, mzxid=6, ctime=1355532303688,\n mtime=1355532303688, version=0, cversion=1,\n aversion=0, ephemeralOwner=0, dataLength=0,\n numChildren=1, pzxid=7)}\n\n#### Arbitrary Data\n\nWhen you want to store arbitrary (but simple!) data objects in, its simple!\n\n >>> sr.set_data('/config/my_app', data={'state': 'enabled'})\n True\n >>> sr.get('/config/my_app')\n {'children': []\n 'data': {u'created': u'2014-06-03 13:37:53',\n u'pid': 2546,\n u'state': u'enabled'},\n 'path': '/config/my_app',\n 'stat': ZnodeStat(czxid=76, mzxid=77, ctime=1401827873764,\n mtime=1401827873766, version=1, cversion=0,\n aversion=0, ephemeralOwner=0, dataLength=62,\n numChildren=0, pzxid=76)}\n\n### Distributed Locks\n\nOne of Zookeepers great features is using it as a global lock manager. We provide\ntwo models for getting a lock. In one model, your lock is only active as long as\nyour code is running\n\n >>> with nd.get_lock('/foo', simultaneous=1):\n ... \n ...\n >>>\n\nAnother example is explicitly locking a path for some period of time, then\nreleasing it explicitly (eg, locking during one method, and waiting for an\nentirely different method to handle the unlock)\n\n >>> nd.acquire_lock('/foo', simultaneous=1)\n >>> \n >>> nd.release_lock('/foo')\n\n### Django\n\nWe initially wrote this code to simplify our use of Zookeeper in Django.\nIncluded is a very simple Django utility package that makes it dead simple\nto use Zookeeper inside Django for retreiving information. To use, add\nthe following to your *settings* module in Django:\n\n from nd_service_registry.contrib.django import utils as ndsr\n ndsr.SERVER = '127.0.0.1'\n ndsr.PORT = '2181'\n ndsr.TIMEOUT = 5\n ndsr.CACHEFILE = '/tmp/serviceregistry.cache'\n\nExample usage to grab a single string and use it as a setting:\n \n MY_CRED = ndsr.get('/creds/some_cred')['data']['mycred']\n\nExample use in your code, calling a method every time a value is updated\nin Zookeeper: \n\n >>> def do_something(data):\n ... print \"New server data: %s\" % data\n ...\n >>> ndsr.get('/services/staging/uswest2/memcache', callback=do_something)\n New server data: { 'path': '/services/staging/uswest2/memcache',\n 'stat': ZnodeStat(czxid=8589934751, mzxid=8589934751,\n ctime=1354785240728,\n mtime=1354785240728, version=0,\n cversion=45, aversion=0,\n ephemeralOwner=0, dataLength=0,\n numChildren=1, pzxid=30064903926),\n 'data': None,\n 'children': { u'ec2-123-123-123-123.us-west-2:11211': {\n u'created': u'2013-01-08 16:51:12',\n u'pid': 3246\n }\n }\n }\n\nWarning: LC\\_ALL and LANG settings\n Due to an unknown bug, if Django cannot find your LC\\_ALL LOCALE settings\n (which often default to 'C'), *nd_service_registry* or *kazoo* crash and\n burn during the init phase. Its unknown why at this point, but we've found\n that its best to *unset LC\\_ALL* and set *LANG=en_US:UTF-8* (or some other\n valid setting) before you start up your Django app.\n\n If you use Celery, set these options in */etc/default/celeryd*.\n\n If you use uWSGI, set them in your uWSGI config file.\n\n Running the Django shell\n\n # unset LC_ALL; LANG=en_US:UTF-8 python manage.py shell\n\n\n## Connection Handling\n\nThe ServiceRegistry object tries everything that it can to make sure that\nthe backend Zookeeper connection is always up and running.\n\n### Fork Behavior\n\nIn the event that your code has created an ServiceRegistry object but then\ngone and forked the process (celery, as an example), we do our best to\ndetect this and re-create the connection, watchers and registrations.\n\nWhen we detect a fork (more on that below), we re-create our Zookeeper\nconnection, and then re-create all Watcher and Registration objects as well.\n\n### Fork Detection\n\nDetecting the fork is extremely tricky... we can only really detect it when\nyou call the module for new data. This means that if you have created a\nWatcher or Registration object, those objects will not be aware of the fork\n(and thus the loss of their connection to Zookeeper) until you make another\ncall to them.\n\nBecause of this, I strongly recommend that if you can detect the fork from\nwithin your application (Django signals perhaps?), you should immediately call\nthe *rebuild()* method on your ServiceRegistry object.\n\n >>> from nd_service_registry import KazooServiceRegistry\n >>> k = KazooServiceRegistry()\n >>> do_fork()\n >>> k.rebuild()\n\n## Exceptions\n\nThe goal of this module is to be as self-contained as possible and require\nas little code in your app as possible. To that end, we *almost never* raise\nan Exception once the module is loaded up and connected.\n\nWe do raise a few exceptions, and each one is documented here. Whenever we\ncan though, we instead just *return False* as a way of indicating that we were\nunable to perform your command now ... but that we will take care of it later.\nWhenever we do this, we throw a WARNING log message as well.\n\n### nd\\_service\\_registry.exceptions.NoConnection\n\nThrown if you attempt any operation that requires immediate access to the\nbackend Zookeeper service. Either a *set()* operation, or a *get()*\noperation on a path for the first time.\n\nAlso thrown during initial connection to Zookeeper, if *lazy=False*.\n\n(It should be noted, a *get()* will actually return the cached results even\nif Zookeeper is down. This allows the service to fail temporarily in the\nbackground but your app is still able to get the *last known* results.)\n\n### nd\\_service\\_registry.exceptions.ReadOnly\n\nIf *readonly=True*, any operation that would result in a *write* will throw\nthis exception. Most notably, a *set()* operation will fail with this\nexception if *readonly=True*.\n\n## API Documentation\n\nDetailed implementation details and instructions are in the individual\nlibrary files.\n\n## Development\n\n### Unit Tests\n\nRunning them\n\n $ python setup.py test\n running test\n test_creates_simple_json_outputformat (nd_service_registry.bin.ndsr.get_tests.GetTests) ... ok\n ...\n ...\n test_default_data_produces_expected_dict (nd_service_registry.funcs_tests.FuncsTests) ... ok\n test_encode_creates_dict_from_single_string (nd_service_registry.funcs_tests.FuncsTests) ... ok\n Ran 15 tests in 0.108s\n\n OK\n\n### Integration Tests\n\nIngegration tests download and install Zookeeper in a temporary path in your\nworkspace. They can be executed like this\n\n $ make integration\n running integration\n ...\n Make sure that the enter/exit functionality works in non-blocking ... ok\n Test that a blocking Lock works ... ok\n test_decode_converts_json_to_dict (nd_service_registry.funcs_tests.FuncsTests) ... ok\n\n ----------------------------------------------------------------------\n Ran 19 tests in 3.360s\n\n OK\n\n## Apps using nd\\_service\\_registry\n\n * [zk_monitor](http://github.com/nextdoor/zkmonitor): Zookeeper Path Monitoring Agent\n * [zk_watcher](http://github.com/nextdoor/zkwatcher): Registers third-party (memcache/etc)\n services in Zookeeper.", "description_content_type": null, "docs_url": null, "download_url": "http://pypi.python.org/pypi/nd_service_registry#downloads", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Nextdoor/ndserviceregistry", "keywords": "zookeeper apache zk", "license": "Apache License, Version 2.0", "maintainer": null, "maintainer_email": null, "name": "nd_service_registry", "package_url": "https://pypi.org/project/nd_service_registry/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/nd_service_registry/", "project_urls": { "Download": "http://pypi.python.org/pypi/nd_service_registry#downloads", "Homepage": "https://github.com/Nextdoor/ndserviceregistry" }, "release_url": "https://pypi.org/project/nd_service_registry/1.1.0/", "requires_dist": null, "requires_python": null, "summary": "Nextdoor Service Registry Module", "version": "1.1.0" }, "last_serial": 2664345, "releases": { "0.2.0": [ { "comment_text": "", "digests": { "md5": "c8477a4c4b6af049cae3d3b9b6b416f7", "sha256": "6fe21c6cdf77ca3b51f9d0e1695f48bdd9b3fb6ff3250e631c920f2372358fe2" }, "downloads": -1, "filename": "nd_service_registry-0.2.0.tar.gz", "has_sig": false, "md5_digest": "c8477a4c4b6af049cae3d3b9b6b416f7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18825, "upload_time": "2013-01-02T21:23:44", "url": "https://files.pythonhosted.org/packages/7f/17/136f2f6be563aeea84eba651b68ed6472209ec574c9a9c23822d2a1850a5/nd_service_registry-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "73c437b9a1d871fd867dd0408a089285", "sha256": "9191fd5e5bdf77579428ef267886b041823c6f554d3559275c7c3a2d0ec4522d" }, "downloads": -1, "filename": "nd_service_registry-0.2.1.tar.gz", "has_sig": false, "md5_digest": "73c437b9a1d871fd867dd0408a089285", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17966, "upload_time": "2013-01-05T22:02:40", "url": "https://files.pythonhosted.org/packages/89/c2/c0b5a8f367384bc8d4002211303db3cf607dda1c34ae7698b93a14d35c89/nd_service_registry-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "e3d94d8d642cb47c3a1de0eb15b9aba5", "sha256": "dea562b1daf4a25f5d6dbbdbd47c28a70cbffa04fd748b0838ffdde54da4fa24" }, "downloads": -1, "filename": "nd_service_registry-0.2.2.tar.gz", "has_sig": false, "md5_digest": "e3d94d8d642cb47c3a1de0eb15b9aba5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20962, "upload_time": "2013-01-11T23:30:58", "url": "https://files.pythonhosted.org/packages/92/b9/ea67317580d00961e3c9358950c56c4eebcf57a23417c5404574e38333e2/nd_service_registry-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "58840f177ef1281eb96c33f048e441df", "sha256": "bc485465441c93b0dd797c18080ec0059a6a4ca66d41ba563957a1138d15a251" }, "downloads": -1, "filename": "nd_service_registry-0.2.3.tar.gz", "has_sig": false, "md5_digest": "58840f177ef1281eb96c33f048e441df", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23820, "upload_time": "2013-03-26T20:44:32", "url": "https://files.pythonhosted.org/packages/88/cb/17210e6544426af5dcd825bc47ecb47880f2fac2c4dff5b5bd9d07c21970/nd_service_registry-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "f114f4427852440e8b55a0478fa93c8d", "sha256": "74bd210b0e74f84e49e6718bc6fb8d7f13cc1a9f84d12f44ef963db1a0dec7fc" }, "downloads": -1, "filename": "nd_service_registry-0.2.4.tar.gz", "has_sig": false, "md5_digest": "f114f4427852440e8b55a0478fa93c8d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24776, "upload_time": "2013-06-25T20:20:00", "url": "https://files.pythonhosted.org/packages/16/6c/e780ed05c78448677853027dc9e71ce5fa123f1c5a1716e45d7afafbe143/nd_service_registry-0.2.4.tar.gz" } ], "0.2.5": [ { "comment_text": "", "digests": { "md5": "939a6bf2b3708d4686b47938aa81898b", "sha256": "5beb7b0c9d84dac03138535fbd31858670100e15eeb9a136905101f72fa642c4" }, "downloads": -1, "filename": "nd_service_registry-0.2.5.tar.gz", "has_sig": false, "md5_digest": "939a6bf2b3708d4686b47938aa81898b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24793, "upload_time": "2013-08-26T16:49:05", "url": "https://files.pythonhosted.org/packages/0c/22/70ff2d419069033d8659b42d74f1e873cf53d02bfbb9c0a549ee34a8d806/nd_service_registry-0.2.5.tar.gz" } ], "0.2.6": [ { "comment_text": "", "digests": { "md5": "e12b7506e42d5edb1043ea67e394222f", "sha256": "939b28450168b986b3e5f18272a9859ffce67a37b2e42fea9fc815c71e8455f2" }, "downloads": -1, "filename": "nd_service_registry-0.2.6.tar.gz", "has_sig": false, "md5_digest": "e12b7506e42d5edb1043ea67e394222f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24835, "upload_time": "2013-09-05T17:17:08", "url": "https://files.pythonhosted.org/packages/5e/94/0f072b5cd454a430a2e13739bfefadadc266612732d04a2eba16d3b1ad1a/nd_service_registry-0.2.6.tar.gz" } ], "0.2.7": [ { "comment_text": "", "digests": { "md5": "b3247eebdc2a128b9ed664cd313499fb", "sha256": "ce73f34bc44dfeb6c5521d97ba81fbbe1745eb76c049f488bfcbb19fb97359e1" }, "downloads": -1, "filename": "nd_service_registry-0.2.7.tar.gz", "has_sig": false, "md5_digest": "b3247eebdc2a128b9ed664cd313499fb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24946, "upload_time": "2013-09-23T16:55:42", "url": "https://files.pythonhosted.org/packages/ef/2e/356577986d14804b53f69849a7fd6bb3120ee607ed233adfba8b7d13293d/nd_service_registry-0.2.7.tar.gz" } ], "0.2.8": [ { "comment_text": "", "digests": { "md5": "dab440635a8e7b2e20b175afd8ddf8da", "sha256": "1c086af43f465831fce44f773f13be89a55dc9dd962cbefa7eb7cdef75e87ded" }, "downloads": -1, "filename": "nd_service_registry-0.2.8.tar.gz", "has_sig": false, "md5_digest": "dab440635a8e7b2e20b175afd8ddf8da", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24954, "upload_time": "2013-10-28T21:25:54", "url": "https://files.pythonhosted.org/packages/62/72/08007acc8ab61c7f6d51e16c2b0e63fb598a7e3ab4fd8f186d55514b83f6/nd_service_registry-0.2.8.tar.gz" } ], "0.2.9": [ { "comment_text": "", "digests": { "md5": "317ac9217b72fe2c50a23ecca8b86bfb", "sha256": "02e3e6a0c38cb1bcc11ef63c7c33621bf0afd7f3b365ed0b82d0556af48b250e" }, "downloads": -1, "filename": "nd_service_registry-0.2.9.tar.gz", "has_sig": false, "md5_digest": "317ac9217b72fe2c50a23ecca8b86bfb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24967, "upload_time": "2013-11-11T16:38:31", "url": "https://files.pythonhosted.org/packages/a0/29/9f9ae27c2c54917e4c017130e5dc11ff500886d5cbda36d2966aeb4ee3c8/nd_service_registry-0.2.9.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "e3a4f59012e446ac24f7a4ea5ec392b4", "sha256": "fc8c761f7af18134e39586f81e1444211c77ddd649b500cb986a2424b085ccaf" }, "downloads": -1, "filename": "nd_service_registry-0.3.0.tar.gz", "has_sig": false, "md5_digest": "e3a4f59012e446ac24f7a4ea5ec392b4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42222, "upload_time": "2014-06-04T21:45:07", "url": "https://files.pythonhosted.org/packages/bb/94/56cddec14cf05b104ca3a2524930a7a47b273f835e6f69634184e1160808/nd_service_registry-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "b58a71a4e87692a17ea3b8f3e8a5a0d0", "sha256": "7a2bfd45f0e0cf9df171024b959ab93d41b1eb1febbfa64724fd662c687ac3a4" }, "downloads": -1, "filename": "nd_service_registry-0.3.1.tar.gz", "has_sig": false, "md5_digest": "b58a71a4e87692a17ea3b8f3e8a5a0d0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42244, "upload_time": "2014-06-04T21:54:07", "url": "https://files.pythonhosted.org/packages/a1/c0/3ceedb7dbe94b456be25175367d2209a6a97d8ff1f5244263643490f9a27/nd_service_registry-0.3.1.tar.gz" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "35a2aacd03a8ec0cd475d20d3557a7b0", "sha256": "96465f3ab19727afc417efc4dd455bac29149e18237b9ff3f2a768bd4ea04c35" }, "downloads": -1, "filename": "nd_service_registry-0.3.2.tar.gz", "has_sig": false, "md5_digest": "35a2aacd03a8ec0cd475d20d3557a7b0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42340, "upload_time": "2014-06-20T22:08:42", "url": "https://files.pythonhosted.org/packages/bb/78/38ad3f8bd58df53468b005223efe48a2a5a4e01d705f54f13adacc4b1a1f/nd_service_registry-0.3.2.tar.gz" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "c4a4ffbd0cd45728abc137e9a09f4bf0", "sha256": "5dc8263c3a21344d79ea477d45964ce9e21a429f67741cbe615926e27b9c98a0" }, "downloads": -1, "filename": "nd_service_registry-1.0.0.tar.gz", "has_sig": false, "md5_digest": "c4a4ffbd0cd45728abc137e9a09f4bf0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 44328, "upload_time": "2014-10-13T17:57:17", "url": "https://files.pythonhosted.org/packages/f7/0c/908e08d48edc3268fa0f6c5da7f92f6bb9305f78f8645951200e75c53233/nd_service_registry-1.0.0.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "ed206cd678a351e511049d3be815a991", "sha256": "9b1e3cb8ae2730229aea6fef791f4c70b46fcc2ad1351091150b05db139fda6a" }, "downloads": -1, "filename": "nd_service_registry-1.1.0.tar.gz", "has_sig": false, "md5_digest": "ed206cd678a351e511049d3be815a991", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 44584, "upload_time": "2017-02-23T21:14:03", "url": "https://files.pythonhosted.org/packages/c2/31/5428bd9182db105ed9f822f03b3f1aaf4e64d6224b4294f137b943d072be/nd_service_registry-1.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ed206cd678a351e511049d3be815a991", "sha256": "9b1e3cb8ae2730229aea6fef791f4c70b46fcc2ad1351091150b05db139fda6a" }, "downloads": -1, "filename": "nd_service_registry-1.1.0.tar.gz", "has_sig": false, "md5_digest": "ed206cd678a351e511049d3be815a991", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 44584, "upload_time": "2017-02-23T21:14:03", "url": "https://files.pythonhosted.org/packages/c2/31/5428bd9182db105ed9f822f03b3f1aaf4e64d6224b4294f137b943d072be/nd_service_registry-1.1.0.tar.gz" } ] }