{ "info": { "author": "Jing Wang", "author_email": "99jingw@gmail.com", "bugtrack_url": null, "classifiers": [ "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: System Administrators", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Programming Language :: SQL", "Topic :: Text Processing", "Topic :: Utilities" ], "description": "==========\nShellQuery\n==========\n\nEver piped together some awful combination of ``grep``, ``sed``, ``awk``, ``sort`` to run a quick-and-dirty analysis? Now you can do those same awful transformations using SQL! Many studies |weaselwords|_ have shown that the aforementioned tools are difficult to use, and moreover, most people prefer SQL |citationneeded|_.\n\nShellQuery (``shq``) is a command line tool for running SQL against plain text. It lets you express logic in SQL instead of stringing together bash commands.\n\nUsage\n=====\n\nSuppose you're running a web server, and you want to get some quick stats on your traffic. Your logs look something like the following::\n\n [pid: 2302|app: 0|req: 12/60] 127.0.0.1 () {38 vars in 569 bytes} [Sat Dec 6 21:19:12 2014] GET /posts/new => generated 5851 bytes in 960 msecs (HTTP/1.1 200) 4 headers in 124 bytes (1 switches on core 0)\n [pid: 2304|app: 0|req: 18/61] 127.0.0.1 () {36 vars in 530 bytes} [Sat Dec 6 21:19:10 2014] GET /posts => generated 631 bytes in 3779 msecs (HTTP/1.1 200) 4 headers in 123 bytes (1 switches on core 0)\n [pid: 2305|app: 0|req: 8/62] 127.0.0.1 () {36 vars in 538 bytes} [Sat Dec 6 21:19:17 2014] GET /posts/123 => generated 7757 bytes in 294 msecs (HTTP/1.1 200) 4 headers in 124 bytes (1 switches on core 0)\n\nAwful bash to get requests by total time spent for all lines saying \"generated *X* bytes in *T* msecs\"::\n\n $ grep generated webserver.log | awk '{ sum[$18] += $24 } END { for (k in sum) { print sum[k], k; } }' | sort -n | tail\n 29 /error/style/black.css\n 535 /posts/9\n 609 /posts/99\n 720 /posts/1\n 737 /posts/123\n 806 /posts/3\n 1157 /posts/5\n 7579 /posts/new\n 7933 /posts/a\n 56594 /posts\n\nLess awful SQL to do the same thing::\n\n $ grep generated webserver.log | shq 'SUM(c24) AS s, c18 GROUP BY c18 ORDER BY s DESC LIMIT 10'\n 56594\t/posts\n 7933\t/posts/a\n 7579\t/posts/new\n 1157\t/posts/5\n 806\t/posts/3\n 737\t/posts/123\n 720\t/posts/1\n 609\t/posts/99\n 535\t/posts/9\n 29\t/error/style/black.css\n\nNote that you may omit the ``SELECT`` and ``FROM`` clauses.\n\nSee ``shq -h`` for additional usage information.\n\nInstalling\n==========\n\n- With pip: ``sudo pip install shellquery``\n- Manually: ``curl -o ~/bin/shq https://raw.githubusercontent.com/jingw/shellquery/master/shellquery.py && chmod +x ~/bin/shq``\n\nYou'll need Python 2.7 or Python 3. (Of course, you might want to double check the download to make sure I'm not giving you malware.)\n\nDevelopment testing\n===================\n\n.. image:: https://travis-ci.org/jingw/shellquery.svg?branch=master\n :target: https://travis-ci.org/jingw/shellquery\n\n.. image:: http://codecov.io/github/jingw/shellquery/coverage.svg?branch=master\n :target: http://codecov.io/github/jingw/shellquery?branch=master\n\nPython 3::\n\n virtualenv3 --no-site-packages env3\n source env3/bin/activate\n pip3 install -r dev_requirements.txt\n py.test\n\nAnd again for Python 2 (after ``deactivate``)::\n\n virtualenv2 --no-site-packages env2\n source env2/bin/activate\n pip2 install -r dev_requirements.txt\n py.test\n\n.. |weaselwords| replace:: :sup:`[weasel words]`\n.. _weaselwords: https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Words_to_watch#Unsupported_attributions\n\n.. |citationneeded| replace:: :sup:`[citation needed]`\n.. _citationneeded: https://en.wikipedia.org/wiki/Wikipedia:Citation_needed", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jingw/shellquery", "keywords": "", "license": "MIT License", "maintainer": "", "maintainer_email": "", "name": "ShellQuery", "package_url": "https://pypi.org/project/ShellQuery/", "platform": "", "project_url": "https://pypi.org/project/ShellQuery/", "project_urls": { "Homepage": "https://github.com/jingw/shellquery" }, "release_url": "https://pypi.org/project/ShellQuery/0.1.4/", "requires_dist": null, "requires_python": "", "summary": "Command line plain text SQL", "version": "0.1.4" }, "last_serial": 5402817, "releases": { "0.1.4": [ { "comment_text": "", "digests": { "md5": "a55fd6a58d2d5ea1297d20e135c8f159", "sha256": "c0e1f8818f1ed47b1a44e36317f9bc577b7dd09fb0861de89a9d721370890a74" }, "downloads": -1, "filename": "ShellQuery-0.1.4.tar.gz", "has_sig": false, "md5_digest": "a55fd6a58d2d5ea1297d20e135c8f159", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7263, "upload_time": "2019-06-15T03:04:29", "url": "https://files.pythonhosted.org/packages/db/22/7834c96a2a14c3c7ff06cd786ace29517d87a6048d7dd95b96fc5ff3c6d8/ShellQuery-0.1.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a55fd6a58d2d5ea1297d20e135c8f159", "sha256": "c0e1f8818f1ed47b1a44e36317f9bc577b7dd09fb0861de89a9d721370890a74" }, "downloads": -1, "filename": "ShellQuery-0.1.4.tar.gz", "has_sig": false, "md5_digest": "a55fd6a58d2d5ea1297d20e135c8f159", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7263, "upload_time": "2019-06-15T03:04:29", "url": "https://files.pythonhosted.org/packages/db/22/7834c96a2a14c3c7ff06cd786ace29517d87a6048d7dd95b96fc5ff3c6d8/ShellQuery-0.1.4.tar.gz" } ] }