{ "info": { "author": "Nextdoor Engineering", "author_email": "eng@nextdoor.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Software Development :: Object Brokering", "Topic :: System :: Distributed Computing" ], "description": "# Nextdoor Scheduler\n\n![Apache](https://img.shields.io/hexpm/l/plug.svg) \n[![Build Status](https://travis-ci.org/Nextdoor/ndscheduler.svg)](https://travis-ci.org/Nextdoor/ndscheduler)\n\n``ndscheduler`` is a flexible python library for building your own cron-like system to schedule jobs, which is to run a tornado process to serve REST APIs and a web ui. It's like [LLVM](http://llvm.org/) that provides modular and reusable components for building a compiler. \n\nCheck out our blog post - [We Don't Run Cron Jobs at Nextdoor](https://engblog.nextdoor.com/we-don-t-run-cron-jobs-at-nextdoor-6f7f9cc62040#.d2erw1pl6)\n\n**``ndscheduler`` currently supports Python 2 & 3 on Mac OS X / Linux.**\n\n## Table of contents\n\n * [Key Abstractions](#key-abstractions)\n * [Try it NOW](#try-it-now)\n * [How to build Your own cron-replacement](#how-to-build-your-own-cron-replacement)\n * [Install ndscheduler](#install-ndscheduler)\n * [Three things](#three-things)\n * [Reference Implementation](#reference-implementation) \n * [Contribute code to ndscheduler](#contribute-code-to-ndscheduler)\n * [REST APIs](#rest-apis)\n * [Web UI](#web-ui)\n\n## Key Abstractions\n\n* [Core](https://github.com/Nextdoor/ndscheduler/tree/master/ndscheduler/core): a bunch of resuable components\n * [Datastore](https://github.com/Nextdoor/ndscheduler/tree/master/ndscheduler/core/datastore): manages database connections and makes queries; could support Postgres, MySQL, and sqlite.\n * Job: represents a schedule job and decides how to run a paricular job.\n * Execution: represents an instance of job execution.\n * AuditLog: logs when and who runs what job.\n * [ScheduleManager](https://github.com/Nextdoor/ndscheduler/blob/master/ndscheduler/core/scheduler_manager.py): access Datastore to manage jobs, i.e., schedule/modify/delete/pause/resume a job.\n* [Server](https://github.com/Nextdoor/ndscheduler/tree/master/ndscheduler/server): a tornado server that runs ScheduleManager and provides REST APIs and serves UI.\n* [Web UI](https://github.com/Nextdoor/ndscheduler/tree/master/ndscheduler/static): a single page HTML app; this is a default implementation.\n\n## Try it NOW\n\nFrom source code:\n\n git clone https://github.com/Nextdoor/ndscheduler.git\n cd ndscheduler\n make simple\n\nOr use docker:\n\n docker run -it -p 8888:8888 wenbinf/ndscheduler\n\nOpen your browser and go to [localhost:8888](http://localhost:8888). \n\n**Demo**\n(Click for fullscreen play)\n![ndscheduler demo](https://giant.gfycat.com/NastyBossyBeaver.gif)\n\n## How to build Your own cron-replacement\n\n### Install ndscheduler\nUsing pip (from GitHub repo)\n\n #\n # Put this in requirements.txt, then run\n # pip install -r requirements.txt\n #\n\n # If you want the latest build\n git+https://github.com/Nextdoor/ndscheduler.git#egg=ndscheduler\n\n # Or put this if you want a specific commit\n git+https://github.com/Nextdoor/ndscheduler.git@5843322ebb440d324ca5a66ba55fea1fd00dabe8\n\n # Or put this if you want a specific tag version\n git+https://github.com/Nextdoor/ndscheduler.git@v0.1.0#egg=ndscheduler\n\n #\n # Run from command line\n #\n\n pip install -e git+https://github.com/Nextdoor/ndscheduler.git#egg=ndscheduler\n\n(We'll upload the package to PyPI soon.)\n\n### Three things\n\nYou have to implement three things for your scheduler, i.e., ``Settings``, ``Server``, and ``Jobs``.\n\n**Settings**\n\nIn your implementation, you need to provide a settings file to override default settings (e.g., [settings in simple_scheduler](https://github.com/Nextdoor/ndscheduler/blob/master/simple_scheduler/settings.py)). You need to specify the python import path in the environment variable ``NDSCHEDULER_SETTINGS_MODULE`` before running the server.\n\nAll available settings can be found in [default_settings.py](https://github.com/Nextdoor/ndscheduler/blob/master/ndscheduler/default_settings.py) file.\n\n**Server**\n\nYou need to have a server file to import and run ``ndscheduler.server.server.SchedulerServer``.\n\n**Jobs**\n\nEach job should be a standalone class that is a subclass of ``ndscheduler.job.JobBase`` and put the main logic of the job in ``run()`` function.\n\nAfter you set up ``Settings``, ``Server`` and ``Jobs``, you can run the whole thing like this:\n\n NDSCHEDULER_SETTINGS_MODULE=simple_scheduler.settings \\\n PYTHONPATH=.:$(PYTHONPATH) \\\n\t\t python simple_scheduler/scheduler.py\n\n### Reference Implementation\n\nSee code in the [simple_scheduler/](https://github.com/Nextdoor/ndscheduler/tree/master/simple_scheduler) directory for inspiration :)\n\nRun it\n\n make simple\n\nAccess the web ui via [localhost:8888](http://localhost:8888)\n\nThe reference implementation also comes with [several sample jobs](https://github.com/Nextdoor/ndscheduler/tree/master/simple_scheduler/jobs).\n* AwesomeJob: it just prints out 2 arguments you pass in.\n* SlackJob: it sends a slack message periodically, for example, team standup reminder.\n* ShellJob: it runs an executable command, for example, run curl to crawl web pages.\n* CurlJob: it's like running [curl](http://curl.haxx.se/) periodically.\n\nAnd it's [dockerized](https://github.com/Nextdoor/ndscheduler/tree/master/simple_scheduler/docker).\n\n## Contribute code to ndscheduler\n\n**Install dependencies**\n\n # Each time we introduce a new dependency in setup.py, you have to run this\n make install\n\n**Run unit tests**\n\n make test\n\n**Clean everything and start from scratch**\n\n make clean\n\nFinally, send pull request. Please make sure the [CI](https://travis-ci.org/Nextdoor/ndscheduler) passes for your PR.\n\n## REST APIs\n\nPlease see [README.md in ndscheduler/server/handlers](https://github.com/Nextdoor/ndscheduler/blob/master/ndscheduler/server/handlers/README.md).\n\n## Web UI\n\nWe provide a default implementation of web ui. You can replace the default web ui by overwriting these settings\n\n STATIC_DIR_PATH = :static asset directory paths:\n TEMPLATE_DIR_PATH = :template directory path:\n APP_INDEX_PAGE = :the file name of the single page app's html:\n\n### The default web ui\n\n**List of jobs**\n\n![List of jobs](http://i.imgur.com/dGILbkZ.png)\n\n**List of executions**\n\n![List of executions](http://i.imgur.com/JpjzrlU.png)\n\n**Audit Logs**\n\n![Audit logs](http://i.imgur.com/eHLzHhw.png)\n\n**Modify a job**\n\n![Modify a job](http://i.imgur.com/aWv6xOR.png)\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "http://pypi.python.org/pypi/ndscheduler#downloads", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Nextdoor/ndscheduler", "keywords": "scheduler nextdoor cron python", "license": "Apache License, Version 2", "maintainer": "", "maintainer_email": "", "name": "ndscheduler", "package_url": "https://pypi.org/project/ndscheduler/", "platform": "", "project_url": "https://pypi.org/project/ndscheduler/", "project_urls": { "Download": "http://pypi.python.org/pypi/ndscheduler#downloads", "Homepage": "https://github.com/Nextdoor/ndscheduler" }, "release_url": "https://pypi.org/project/ndscheduler/0.3.0/", "requires_dist": [ "APScheduler (==3.0.0)", "SQLAlchemy (==1.0.0)", "future (==0.15.2)", "tornado (==4.3.0)", "python-dateutil (==2.2)", "funcsigs ; extra == 'python_version_3.3_'" ], "requires_python": "", "summary": "ndscheduler: A cron-replacement library from Nextdoor", "version": "0.3.0" }, "last_serial": 4536386, "releases": { "0.3.0": [ { "comment_text": "", "digests": { "md5": "697a8bd51414279299a7229ef08ffc02", "sha256": "69fe97f33cd04a1061a3552da5dabf7e3fe9d58c7d0b89c16a8d6b46a2bcbb36" }, "downloads": -1, "filename": "ndscheduler-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "697a8bd51414279299a7229ef08ffc02", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 369200, "upload_time": "2018-11-28T00:18:14", "url": "https://files.pythonhosted.org/packages/a2/c3/2928f0e73589d190da76f45e6316b8c535b83671972dcb945fbd7dc1c5a0/ndscheduler-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "42423e034dcf2cffa1c8344b208f8eb6", "sha256": "1e6334b1268673929bf63e77a9ab354ea70e1105a253d40533a27142f674d656" }, "downloads": -1, "filename": "ndscheduler-0.3.0-py3.6.egg", "has_sig": false, "md5_digest": "42423e034dcf2cffa1c8344b208f8eb6", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": null, "size": 414216, "upload_time": "2018-11-28T00:18:16", "url": "https://files.pythonhosted.org/packages/f8/db/65f0d67a95d9d95a30d00504d86e26bba10b2095d5d5fca4e7a55599b557/ndscheduler-0.3.0-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "6d6e53baa91348e689eb33d29b53f355", "sha256": "bd7405fdd398b3f04c308bb9cb962c0ddf34cb6cf7f65c4c89e5cce027983b83" }, "downloads": -1, "filename": "ndscheduler-0.3.0.tar.gz", "has_sig": false, "md5_digest": "6d6e53baa91348e689eb33d29b53f355", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 325242, "upload_time": "2018-11-28T00:18:18", "url": "https://files.pythonhosted.org/packages/37/2e/5c29b00338f6831ac0076f9b38576a8004b7c22f19722afce9a86f91cc4f/ndscheduler-0.3.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "697a8bd51414279299a7229ef08ffc02", "sha256": "69fe97f33cd04a1061a3552da5dabf7e3fe9d58c7d0b89c16a8d6b46a2bcbb36" }, "downloads": -1, "filename": "ndscheduler-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "697a8bd51414279299a7229ef08ffc02", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 369200, "upload_time": "2018-11-28T00:18:14", "url": "https://files.pythonhosted.org/packages/a2/c3/2928f0e73589d190da76f45e6316b8c535b83671972dcb945fbd7dc1c5a0/ndscheduler-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "42423e034dcf2cffa1c8344b208f8eb6", "sha256": "1e6334b1268673929bf63e77a9ab354ea70e1105a253d40533a27142f674d656" }, "downloads": -1, "filename": "ndscheduler-0.3.0-py3.6.egg", "has_sig": false, "md5_digest": "42423e034dcf2cffa1c8344b208f8eb6", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": null, "size": 414216, "upload_time": "2018-11-28T00:18:16", "url": "https://files.pythonhosted.org/packages/f8/db/65f0d67a95d9d95a30d00504d86e26bba10b2095d5d5fca4e7a55599b557/ndscheduler-0.3.0-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "6d6e53baa91348e689eb33d29b53f355", "sha256": "bd7405fdd398b3f04c308bb9cb962c0ddf34cb6cf7f65c4c89e5cce027983b83" }, "downloads": -1, "filename": "ndscheduler-0.3.0.tar.gz", "has_sig": false, "md5_digest": "6d6e53baa91348e689eb33d29b53f355", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 325242, "upload_time": "2018-11-28T00:18:18", "url": "https://files.pythonhosted.org/packages/37/2e/5c29b00338f6831ac0076f9b38576a8004b7c22f19722afce9a86f91cc4f/ndscheduler-0.3.0.tar.gz" } ] }