{ "info": { "author": "Alec Thomas", "author_email": "alec@swapoff.org", "bugtrack_url": null, "classifiers": [], "description": "PAWK - A Python line processor (like AWK)\n=========================================\n\nPAWK aims to bring the full power of Python to AWK-like line-processing.\n\nHere are some quick examples to show some of the advantages of pawk over\nAWK.\n\nThe first example transforms ``/etc/hosts`` into a JSON map of host to\nIP:\n\n::\n\n cat /etc/hosts | pawk -B 'd={}' -E 'json.dumps(d)' '!/^#/ d[f[1]] = f[0]'\n\nBreaking this down:\n\n1. ``-B 'd={}'`` is a begin statement initializing a dictionary,\n executed once before processing begins.\n2. ``-E 'json.dumps(d)'`` is an end statement expression, producing the\n JSON representation of the dictionary ``d``.\n3. ``!/^#/`` tells pawk to match any line *not* beginning with ``#``.\n4. ``d[f[1]] = f[0]`` adds a dictionary entry where the key is the\n second field in the line (the first hostname) and the value is the\n first field (the IP address).\n\nAnd another example showing how to bzip2-compress + base64-encode a\nfile:\n\n::\n\n cat pawk.py | pawk -E 'base64.encodestring(bz2.compress(t))'\n\nAWK example translations\n~~~~~~~~~~~~~~~~~~~~~~~~\n\nMost basic AWK constructs are available. You can find more idiomatic\nexamples below in the example section, but here are a bunch of awk\ncommands and their equivalent pawk commands to get started with:\n\nPrint lines matching a pattern:\n\n::\n\n ls -l / | awk '/etc/'\n ls -l / | pawk '/etc/'\n\nPrint lines *not* matching a pattern:\n\n::\n\n ls -l / | awk '!/etc/'\n ls -l / | pawk '!/etc/'\n\nField slicing and dicing (here pawk wins because of Python's array\nslicing):\n\n::\n\n ls -l / | awk '/etc/ {print $5, $6, $7, $8, $9}'\n ls -l / | pawk '/etc/ f[4:]'\n\nBegin and end end actions (in this case, summing the sizes of all\nfiles):\n\n::\n\n ls -l | awk 'BEGIN {c = 0} {c += $5} END {print c}'\n ls -l | pawk -B 'c = 0' -E 'c' 'c += int(f[4])'\n\nPrint files where a field matches a numeric expression (in this case\nwhere files are > 1024 bytes):\n\n::\n\n ls -l | awk '$5 > 1024'\n ls -l | pawk 'int(f[4]) > 1024'\n\nMatching a single field (any filename with \"t\" in it):\n\n::\n\n ls -l | awk '$NF ~/t/'\n ls -l | pawk '\"t\" in f[-1]'\n\nInstallation\n------------\n\nIt should be as simple as:\n\n::\n\n pip install pawk\n\nBut if that doesn't work, just download the ``pawk.py``, make it\nexecutable, and place it somewhere in your path.\n\nExpression evaluation\n---------------------\n\nPAWK evaluates a Python expression or statement against each line in\nstdin. The following variables are available in local context:\n\n- ``line`` - Current line text, including newline.\n- ``l`` - Current line text, excluding newline.\n- ``n`` - The current 1-based line number.\n- ``f`` - Fields of the line (split by the field separator ``-F``).\n- ``nf`` - Number of fields in this line.\n- ``m`` - Tuple of match regular expression capture groups, if any.\n\nAdditionally, if the flag ``-H, --header`` is provided, each field in\nthe first row of the input will be treated as field variable names in\nsubsequent rows. The header is not output. For example, given the input:\n\n::\n\n count name\n 12 bob\n 34 fred\n\nWe could do:\n\n::\n\n $ pawk -H '\"%s is %s\" % (name, count)' < input.txt\n bob is 12\n fred is 34\n\nModule references will be automatically imported if possible.\nAdditionally, the ``--import [,,...]`` flag can be used\nto import symbols from a set of modules into the evaluation context.\n\neg. ``--import os.path`` will import all symbols from ``os.path``, such\nas ``os.path.isfile()``, into the context.\n\nOutput\n------\n\nLine actions\n~~~~~~~~~~~~\n\nThe type of the evaluated expression determines how output is displayed:\n\n- ``tuple`` or ``list``: the elements are converted to strings and\n joined with the output delimiter (``-O``).\n- ``None`` or ``False``: nothing is output for that line.\n- ``True``: the original line is output.\n- Any other value is converted to a string.\n\nStart/end blocks\n~~~~~~~~~~~~~~~~\n\nThe rules are the same as for line actions with one difference. Because\nthere is no \"line\" that corresponds to them, an expression returning\nTrue is ignored.\n\n::\n\n $ echo -ne 'foo\\nbar' | pawk -E t\n foo\n bar\n\nCommand-line usage\n------------------\n\n::\n\n Usage: cat input | pawk [] \n\n A Python line-processor (like awk).\n\n See https://github.com/alecthomas/pawk for details. Based on\n http://code.activestate.com/recipes/437932/.\n\n Options:\n -h, --help show this help message and exit\n -I , --in_place=\n modify given input file in-place\n -i , --import=\n comma-separated list of modules to \"from x import *\"\n from\n -F input delimiter\n -O output delimiter\n -L output line separator\n -B , --begin=\n begin statement\n -E , --end=\n end statement\n -s, --statement DEPRECATED. retained for backward compatibility\n --strict abort on exceptions\n\nExamples\n--------\n\nLine processing\n~~~~~~~~~~~~~~~\n\nPrint the name and size of every file from stdin:\n\n::\n\n find . -type f | pawk 'f[0], os.stat(f[0]).st_size'\n\n **Note:** this example also shows how pawk automatically imports\n referenced modules, in this case ``os``.\n\nPrint the sum size of all files from stdin:\n\n::\n\n find . -type f | \\\n pawk \\\n --begin 'c=0' \\\n --end c \\\n 'c += os.stat(f[0]).st_size'\n\nShort-flag version:\n\n::\n\n find . -type f | pawk -B c=0 -E c 'c += os.stat(f[0]).st_size'\n\nWhole-file processing\n~~~~~~~~~~~~~~~~~~~~~\n\nIf you do not provide a line expression, but do provide an end\nstatement, pawk will accumulate each line, and the entire file's text\nwill be available in the end statement as ``t``. This is useful for\noperations on entire files, like the following example of converting a\nfile from markdown to HTML:\n\n::\n\n cat README.md | \\\n pawk --end 'markdown.markdown(t)'\n\nShort-flag version:\n\n::\n\n cat README.md | pawk -E 'markdown.markdown(t)'\n", "description_content_type": null, "docs_url": null, "download_url": "http://github.com/alecthomas/pawk", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/alecthomas/pawk", "keywords": "", "license": "PSF", "maintainer": "", "maintainer_email": "", "name": "pawk", "package_url": "https://pypi.org/project/pawk/", "platform": "any", "project_url": "https://pypi.org/project/pawk/", "project_urls": { "Download": "http://github.com/alecthomas/pawk", "Homepage": "http://github.com/alecthomas/pawk" }, "release_url": "https://pypi.org/project/pawk/0.6.5/", "requires_dist": null, "requires_python": "", "summary": "PAWK - A Python line processor (like AWK)", "version": "0.6.5" }, "last_serial": 2816537, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "c6a93381e64d86b8629f450863461f19", "sha256": "137c48445cff7ab58bfbe53d2af0c52760f93f883447f4b8b1096fc533608fe5" }, "downloads": -1, "filename": "pawk-0.1.tar.gz", "has_sig": false, "md5_digest": "c6a93381e64d86b8629f450863461f19", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3256, "upload_time": "2013-02-10T02:09:55", "url": "https://files.pythonhosted.org/packages/66/92/fcf16f232c08c72af31a9ceed956334c7e7a095c09009296b63288a05576/pawk-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "3427a5dfaf0f0038437471943690dcfd", "sha256": "f33cbbc1e9a1e44cd71b75a658d113f0476ebe0c106f8a214f2bc2caa99a53b8" }, "downloads": -1, "filename": "pawk-0.2.tar.gz", "has_sig": false, "md5_digest": "3427a5dfaf0f0038437471943690dcfd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3865, "upload_time": "2013-02-10T04:36:02", "url": "https://files.pythonhosted.org/packages/3c/e8/a7315b014e0fe4e31e9f7a3b1d0428072db8ea3b372494bfdb3d101f3f77/pawk-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "7a41cf230b4a743630646b331469b942", "sha256": "dc46be572c5fb539202c439276f17ce922c1674d3987cf4eac22d0924701284b" }, "downloads": -1, "filename": "pawk-0.3-py2.7.egg", "has_sig": false, "md5_digest": "7a41cf230b4a743630646b331469b942", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 6334, "upload_time": "2013-02-10T18:14:09", "url": "https://files.pythonhosted.org/packages/8f/93/5dabb57b16b5e333e0ea7b68ad921952b1cbf25ad4df385ab72c1363c2b2/pawk-0.3-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "b97ae4b664de6da23517a0d91734ae81", "sha256": "ec4725174a4674a7ab107be6dfe42301230b9461f939ba0366372697771cdcf7" }, "downloads": -1, "filename": "pawk-0.3.tar.gz", "has_sig": false, "md5_digest": "b97ae4b664de6da23517a0d91734ae81", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4715, "upload_time": "2013-02-10T18:13:59", "url": "https://files.pythonhosted.org/packages/c8/2e/c007c6f389a07ce123a2ecd47564719d40f64aa4e0d345f42766810c4545/pawk-0.3.tar.gz" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "130bf4e3c33756aadb149409db85f595", "sha256": "852397dfb2d621208d15101396ac51f0329b4449c0910cfc61ac0ebcdcf6ce96" }, "downloads": -1, "filename": "pawk-0.4-py2.7.egg", "has_sig": false, "md5_digest": "130bf4e3c33756aadb149409db85f595", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 6452, "upload_time": "2013-02-11T01:35:11", "url": "https://files.pythonhosted.org/packages/7c/ee/c370b742d62f2270cda37b292035add69cb5e3ad1010ce0ffe823c2d8159/pawk-0.4-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "5fd7ea8ee6a9ac7949819b99582477d1", "sha256": "b199b2dd2e98345095f65906f00e8c71735d864d118d7006ef8bcb2bd00655d1" }, "downloads": -1, "filename": "pawk-0.4.tar.gz", "has_sig": false, "md5_digest": "5fd7ea8ee6a9ac7949819b99582477d1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4727, "upload_time": "2013-02-11T01:35:08", "url": "https://files.pythonhosted.org/packages/69/a7/54477670ff0dce120d92fa969f8e7bcae437f8f3518adae691dce8126e08/pawk-0.4.tar.gz" } ], "0.5": [ { "comment_text": "", "digests": { "md5": "e3f7a2cbfe7904bf107d38f4d9218fe5", "sha256": "f921bf3e764f91d76b8b36b5ce885ffe2b6f832df912d0925150a95334b93bed" }, "downloads": -1, "filename": "pawk-0.5.tar.gz", "has_sig": false, "md5_digest": "e3f7a2cbfe7904bf107d38f4d9218fe5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5258, "upload_time": "2013-07-31T18:32:03", "url": "https://files.pythonhosted.org/packages/35/bd/378c0e2c674a703e37ce520b04a03399b69ec94cdc57c22db167a08d52b6/pawk-0.5.tar.gz" } ], "0.6": [ { "comment_text": "", "digests": { "md5": "d5fa14a9a18db14d426f040872b06b45", "sha256": "18624c4c0b90dd6f7f0af1227a3bab0e848008efcdfae4d0d53a1d3b5fe616d4" }, "downloads": -1, "filename": "pawk-0.6.tar.gz", "has_sig": false, "md5_digest": "d5fa14a9a18db14d426f040872b06b45", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5668, "upload_time": "2016-01-19T05:10:41", "url": "https://files.pythonhosted.org/packages/ab/5a/c22c6ff37c74196950fad281b1094547e06065d5c922d8b679458e5f37b5/pawk-0.6.tar.gz" } ], "0.6.1": [], "0.6.2": [ { "comment_text": "", "digests": { "md5": "89bd2f971a052d0d7eb92147f3755388", "sha256": "14d5da18e39325a78708b8df01068c257a0a96e529eb9554c553dca28030abb2" }, "downloads": -1, "filename": "pawk-0.6.2.tar.gz", "has_sig": false, "md5_digest": "89bd2f971a052d0d7eb92147f3755388", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6598, "upload_time": "2017-02-14T07:13:17", "url": "https://files.pythonhosted.org/packages/4d/41/ca78414100a3278cfce17e5ab79d41e0940e2d56ef0b2ebfdac1fa14cde6/pawk-0.6.2.tar.gz" } ], "0.6.3": [ { "comment_text": "", "digests": { "md5": "a90500e116125a6c0233391f2f867706", "sha256": "28f14806f695fa9594beb96dbf7c9e7c8cf88eeea3f22d25b041c8260038f024" }, "downloads": -1, "filename": "pawk-0.6.3.tar.gz", "has_sig": false, "md5_digest": "a90500e116125a6c0233391f2f867706", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7250, "upload_time": "2017-02-14T07:15:32", "url": "https://files.pythonhosted.org/packages/19/2b/c81c1ad0680f3828cb557567dbfb821035cd678de2366bd1bed1a5df5ec8/pawk-0.6.3.tar.gz" } ], "0.6.4": [ { "comment_text": "", "digests": { "md5": "765ff598d6cc29c8ae4e2c3e8169692b", "sha256": "495ba0e24384a10e28ecb59af55cabdc1ba64db634fe9f9205a229a3f9430dd4" }, "downloads": -1, "filename": "pawk-0.6.4.tar.gz", "has_sig": false, "md5_digest": "765ff598d6cc29c8ae4e2c3e8169692b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6820, "upload_time": "2017-04-20T10:06:22", "url": "https://files.pythonhosted.org/packages/4a/12/4e1ec051dbb8b3e7c0d27a1ac270b9125c60e15db20dd9ab7607a30dead6/pawk-0.6.4.tar.gz" } ], "0.6.5": [ { "comment_text": "", "digests": { "md5": "1b7e6aecd233936cc1f687c05c30ff4c", "sha256": "2548b71ef7dba2976a711b341dbc7c8769f31f6c11d037b62963a6551ad74dda" }, "downloads": -1, "filename": "pawk-0.6.5.tar.gz", "has_sig": false, "md5_digest": "1b7e6aecd233936cc1f687c05c30ff4c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7636, "upload_time": "2017-04-20T10:08:08", "url": "https://files.pythonhosted.org/packages/ce/d5/70fc0b6d979f6adbe5d07d6093479118b75b3ab1ea977d575579c0d2f966/pawk-0.6.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1b7e6aecd233936cc1f687c05c30ff4c", "sha256": "2548b71ef7dba2976a711b341dbc7c8769f31f6c11d037b62963a6551ad74dda" }, "downloads": -1, "filename": "pawk-0.6.5.tar.gz", "has_sig": false, "md5_digest": "1b7e6aecd233936cc1f687c05c30ff4c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7636, "upload_time": "2017-04-20T10:08:08", "url": "https://files.pythonhosted.org/packages/ce/d5/70fc0b6d979f6adbe5d07d6093479118b75b3ab1ea977d575579c0d2f966/pawk-0.6.5.tar.gz" } ] }