{ "info": { "author": "Scott Robertson", "author_email": "scott@triv.io", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Web Environment", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Software Development" ], "description": "![Trivio Logo](http://www.triv.io/images/trivio_logo.png)\n\n\nExample Workflow\n===\n\n\nWe're making managing a big data project as simple as a \"git push\". Think of us as \"Heroku for Big Data\"\n\n\nWith triv.io you can transform massive amounts of poorly structured data on a regular basis. Regardless of the size and whether your data exists in log files, multiple databases or even at the end of a web API we help you store, transform and share your data with others.\n\nInstall triv.io and login\n---\n``` \n$ pip install trivio\n$ trivio login\n Triv.io uses github for authentication. You'll need a github \n username and password to proceed. Authentication is done directly\n with the github servers, we never see or store your user name and\n password. Don't believe us? Have a look at the trivio client source.\n \n GitHub username: \n GitHub password:\n``` \n\nCreate your first project\n---\n``` \n$ trivio create https://github.com/trivio/hello_world\nCreating project hello_world\nYou do not have write access to this project\nwould you like to fork it?[Yn]: Y\n\nforking project to https://github.com/username/hello_world\ncloning https://github.com/username/hello_world\nYour project is ready for use\n``` \n\nTry it out\n---\n```\n~ $ cd hello_world \nhello_world $ trivio simulate word_counts\nblah 1\nfoo 120\nbaz 767\n```\n\nMake some changes\n---\nUsing your favorite editor let's make word_counts use a better tokenizer\n\n```\nhello_world $ ls\n word_counts.pyj\nhello_world $ vi word_counts.pyj\n```\n\nChange the file to look like this\n\n```\n@word_counts.map\ndef map(doc, params, tokenizer = re.compile('[^A-Z0-9_.]+', flags=re.I)):\n for word in tokenizer.split(doc['payload']):\n yield word, 1\n```\n\nNow run simulate again to verify the results\n\n```\nhello_world $ trivio simulate word_counts\nblah 1\nfoo 120\nbaz 767\n```\n\nPush your changes\n---\nOnce you've tweaked your algorithm you need to push those changes to trivio\n\n```\nhello_world $ trivio push \"Used a better tokenizer\"\nchanging to srobertson\n[master 37200ed] pushing\n 1 files changed, 1 insertions(+), 1 deletions(-)\nCounting objects: 5, done.\nDelta compression using up to 8 threads.\nCompressing objects: 100% (3/3), done.\nWriting objects: 100% (3/3), 301 bytes, done.\nTotal 3 (delta 2), reused 0 (delta 0)\nTo git@github.com:srobertson/common_crawl_index.git\n 4ebf177..37200ed master -> master\n\nSuccessfully pushed your changes to trivio.\n\nYour project is paused use 'trivio step' or 'trivio resume' to build using\nyour changes.\n\nhello_world $\n```\n\n\nCompute a single segment\n---\nBy default, all projects start off \"paused\", this prevents them from running up your compute bills until you're satisfied your code works the way you expect it to. You've already used the `trivio simulate` method which samples your data and returns you results. The next step is to build a segment (\"a portion of your table\"see triv.io data model for more details)\n\n```\n$ trivio step word_counts\nBuilding next segment 2009-9-17 00:00:00\n$\n```\n\nNotice that this command returns immediately, building a segment can be quite involved. You may be producing terabytes of data which you wouldn't want to download to your workstation.\n\nMonitoring Build Progress\n---\n```\n$ trivio watch\n\nTable Queued Built Failed \n---------------------------------------------------\n.../common-crawl/parsed-output 0 1 0\n\n\nNext build: Never (project paused)\n```\n\nThis will monitor your progress of your project. When the watch command spits out Next build Never. Hit `ctrl-c` to get back to your command line. As noted your project is paused so it won't build anything else until you either tell trivio to build a single segment with the step command or tell trivio to unpause your project. We'll get to unpausing in a moment\n\n\nQuerying tables\n---\n\nNow that trivio has had a chance to build part of your table we can query it for data. Tri.vio uses a sql like syntax for querying\n\n```\n$ trivio query \"select * from word_counts\"\n\n+----------+-------+\n| key | value |\n+----------+-------+\n| Zachary | 22731 |\n| Alfred | 20477 |\n| Gregory | 17179 |\n| Ned | 16860 |\n| Ulrich | 15300 |\n| Thomas | 14995 |\n+----------+-------+\n```\n\nPut Triv.io on Autopilot\n---\n\nOnce you're happy with your project, you can tell triv.io to keep your tables up to date automatically based on the schedules you defined in your scripts.\n\n```\n$ trivio resume\nAuto building all tables, use 'trivio watch' to monitor progress\n```\n\nWhere to go from here\n---\nThat's the basic workflow\n\n* [Trivio data model](http://docs.triv.io/)\n* [Talking to new storages](http://docs.triv.io/)\n* [Interpreting mime types](http://docs.triv.io/)\n* Check out our [blog](http://docs.triv.io/) for cool projects, tips and tricks\n\n\n\n\n[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/trivio/trivio.client/trend.png)](https://bitdeli.com/free \"Bitdeli Badge\")\n\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/trivio/trivio.client", "keywords": null, "license": "UNKNOWN", "maintainer": null, "maintainer_email": null, "name": "trivio", "package_url": "https://pypi.org/project/trivio/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/trivio/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/trivio/trivio.client" }, "release_url": "https://pypi.org/project/trivio/0.2.5/", "requires_dist": null, "requires_python": null, "summary": "triv.io command line client", "version": "0.2.5" }, "last_serial": 800919, "releases": { "0.2.0": [ { "comment_text": "", "digests": { "md5": "3e7dcddd04670190473f7ae81aca93b1", "sha256": "b6979a959a158159e0614ecdc9412d806be42a9e6e2b25f0a703b08a917aa468" }, "downloads": -1, "filename": "trivio-0.2.0.tar.gz", "has_sig": false, "md5_digest": "3e7dcddd04670190473f7ae81aca93b1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6421, "upload_time": "2013-02-14T20:07:28", "url": "https://files.pythonhosted.org/packages/35/b5/0373a425886df8dfb2a3bd92f639c3aead4fa863a3fd06ffd94380189e7b/trivio-0.2.0.tar.gz" } ], "0.2.0a": [ { "comment_text": "", "digests": { "md5": "7e62aafde6945cc6796e777a99cf659b", "sha256": "64e310870d45e76828b13df7f12ffc06ed59b30e947f8505ac90e76b0acccccf" }, "downloads": -1, "filename": "trivio-0.2.0a.tar.gz", "has_sig": false, "md5_digest": "7e62aafde6945cc6796e777a99cf659b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6444, "upload_time": "2013-02-14T20:13:23", "url": "https://files.pythonhosted.org/packages/b6/57/fd097f4d1150b7d8172a6a57eac6cc06e7f3c88214050757d807082c9051/trivio-0.2.0a.tar.gz" } ], "0.2.0b": [ { "comment_text": "", "digests": { "md5": "9d79f53384c94bac92a240454e9f2222", "sha256": "68ab514a42ae2de14e4a34e14dba7925dbaab77b84da53bb35a893667711f5f0" }, "downloads": -1, "filename": "trivio-0.2.0b.tar.gz", "has_sig": false, "md5_digest": "9d79f53384c94bac92a240454e9f2222", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8695, "upload_time": "2013-02-14T20:36:36", "url": "https://files.pythonhosted.org/packages/c0/b2/940191120c95b5a4df8f78b5e290f2c15d2f17be4318349fb529dbebb34d/trivio-0.2.0b.tar.gz" } ], "0.2.0c": [ { "comment_text": "", "digests": { "md5": "12fa20dd61f1ce7e28a3a78c06d98e33", "sha256": "80a35e8e9055c4b3bfdb33cf0ef3bfff24d8f02688b083a2cf1f005f0a1f0938" }, "downloads": -1, "filename": "trivio-0.2.0c.tar.gz", "has_sig": false, "md5_digest": "12fa20dd61f1ce7e28a3a78c06d98e33", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8710, "upload_time": "2013-02-14T20:44:18", "url": "https://files.pythonhosted.org/packages/bb/63/d638a382debabca32ed7aa538dbc5870af3c504e3555d213aa7a238fd2f3/trivio-0.2.0c.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "69e6f81e2b3afa5b323a4f5a0cb79574", "sha256": "36c1a5964c191787744d169c445fe01e8eb781a60eb3a463a1551262eeeb99cf" }, "downloads": -1, "filename": "trivio-0.2.1.tar.gz", "has_sig": false, "md5_digest": "69e6f81e2b3afa5b323a4f5a0cb79574", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8713, "upload_time": "2013-02-14T20:45:31", "url": "https://files.pythonhosted.org/packages/ff/d0/60be64fea91864f4b92ae4216b82f4b13c73d3a76dbb1a1272df1f7fcbe9/trivio-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "ec825a4eaacd4cf73ed95526d2482a2f", "sha256": "7fa710497aa9ea6397419be23314a52375b16e7ecd0f1d0b1d44d9e61c330715" }, "downloads": -1, "filename": "trivio-0.2.2.tar.gz", "has_sig": false, "md5_digest": "ec825a4eaacd4cf73ed95526d2482a2f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8728, "upload_time": "2013-02-14T20:47:03", "url": "https://files.pythonhosted.org/packages/7b/ac/a315ac59f44d43a0473b51e5f62256ff4f13e1ac859bb8fd901258c6fc9b/trivio-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "d831c62c57fb5ca64db4b599cf22e694", "sha256": "6e74757fb3fa7de80d3749eb8b1c0c8c7b05ab0afa9b0c0e080b7915bbd5ec03" }, "downloads": -1, "filename": "trivio-0.2.3.tar.gz", "has_sig": false, "md5_digest": "d831c62c57fb5ca64db4b599cf22e694", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8749, "upload_time": "2013-02-15T20:17:32", "url": "https://files.pythonhosted.org/packages/2e/93/12cb9e411a3058a76ea3c618b3d4a13e57b0cdfd0bd70be93a6498bbfdda/trivio-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "973c108ac7745eaf4fe8e1242a25b6dc", "sha256": "1c0de54ae6a9cfd44b87f3bd05c602e563f5fdb724a9da4633a4725393b577a5" }, "downloads": -1, "filename": "trivio-0.2.4.tar.gz", "has_sig": false, "md5_digest": "973c108ac7745eaf4fe8e1242a25b6dc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8590, "upload_time": "2013-02-15T20:26:07", "url": "https://files.pythonhosted.org/packages/ce/0f/8f0fd446ef3121c068f4da5f0f33ef98370a7676b7f5cb0cda2c9c485a2e/trivio-0.2.4.tar.gz" } ], "0.2.5": [ { "comment_text": "", "digests": { "md5": "d07c521883ad3b5339ffb321d1bb5fc8", "sha256": "4bd26ae6dd806a1804c366d4052a6a84d481b4b3aa88c2a177a34d18ef8294e2" }, "downloads": -1, "filename": "trivio-0.2.5.tar.gz", "has_sig": false, "md5_digest": "d07c521883ad3b5339ffb321d1bb5fc8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9422, "upload_time": "2013-03-22T20:47:17", "url": "https://files.pythonhosted.org/packages/42/a1/0162d0698ae1d0d75251d4a726b645abb1aa04469fbf68661c7be86b4479/trivio-0.2.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d07c521883ad3b5339ffb321d1bb5fc8", "sha256": "4bd26ae6dd806a1804c366d4052a6a84d481b4b3aa88c2a177a34d18ef8294e2" }, "downloads": -1, "filename": "trivio-0.2.5.tar.gz", "has_sig": false, "md5_digest": "d07c521883ad3b5339ffb321d1bb5fc8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9422, "upload_time": "2013-03-22T20:47:17", "url": "https://files.pythonhosted.org/packages/42/a1/0162d0698ae1d0d75251d4a726b645abb1aa04469fbf68661c7be86b4479/trivio-0.2.5.tar.gz" } ] }