{
    "info": {
        "author": "Gary Lee",
        "author_email": "garywlee@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "What is cmdlet?\r\n===============\r\n\r\nCmdlet provides pipe-like mechanism to cascade functions and generators.\r\nIt uses symbol(\\ **\\|**) to convert function to Pipe object and cascade\r\nthem. This sequence of commands can be executed and evaluated later.\r\nJust like pipe mechanism in Unix shell. For example:\r\n\r\n.. code:: python\r\n\r\n    from cmdlet.cmds import *\r\n\r\n    # Create piped commands.\r\n    cmds = range(10) | pipe.filter(lambda x: x > 5) | format('item#%d')\r\n\r\n    # Execute commands and return the last processed data.\r\n    run(cmds)\r\n    # >>> 'item#9'\r\n\r\n    # Execute commands and return processed data in a list.\r\n    result(cmds)\r\n    # >>> ['item#6', 'item#7', 'item#8', 'item#9']\r\n\r\n    # Execute commands and return iterator for processed data.\r\n    for data in cmds:\r\n        print data\r\n    # >>> item#6\r\n    # >>> item#7\r\n    # >>> item#8\r\n    # >>> item#9\r\n\r\nFirst, we created commands and used **\\|** to cascade them. Then, we can\r\nexecute commands by run(), result() or iterator.\r\n\r\ncmdlet can convert corresponding types to Pipe object automatically. In\r\nabove example, range(10) is a iterator not a Pipe object. Because second\r\nitem is a Pipe object(made by pipe.filter), it turns out first item to\r\nbe converted to a Pipe object automatically.\r\n\r\nThere are many useful utilities in cmdlet.cmds modules. They can provide\r\na great convenience to build up useful pipes. Here is a example:\r\n\r\n.. code:: python\r\n\r\n    from cmdlet.cmds import *\r\n\r\n    query_topic =\r\n        'find ./mydoc -name \"*.txt\" -print' |\r\n        readline(end=10) |\r\n        match(r'^[tT]opic:\\s*(?P<topic>.+)\\s*', to=dict) |\r\n        values('topic')\r\n\r\n    for topic in query_topic:\r\n        print topic\r\n\r\nIn above example, the goal is to query topic from article files. To\r\nachieve the goal, we have to:\r\n\r\n1. Search text files in a given folder.\r\n2. Read first 10 lines from each file.\r\n3. Find the line that matched 'topic: foo bar' pattern.\r\n4. Extract the topic string.\r\n\r\nWith the utilities provided by *cmdlet.cmds*, we only need to write a\r\nfew of code. The first string which starts with 'find' is a normal shell\r\nscript. It is converted to *sh* pipe automatically and executed with\r\nsystem shell. The *readline* pipe can open files whose name passed from\r\nsh pipe. *match* pipe and *values* pipe work together to extract topic\r\nfrom file content.\r\n\r\nAbove example shows not only small code but also readability. It's\r\nreally easy to understand the purpose of source code.\r\n\r\nNOTE: > When using cmdlet's pipe mechanism, make sure one of your >\r\n**first two pipe items** is a valid Pipe object.\r\n\r\nThere is another advantage to use cmdlet. The pipe object is evaluated\r\nwhen calling result, run or iter. It implies you can reuse them. Let's\r\nmodify previous example.\r\n\r\n.. code:: python\r\n\r\n    from cmdlet.cmds import *\r\n\r\n    # Separate from query_topic command.\r\n    extract_topic =\r\n        readline(end=10) |\r\n        match(r'^[tT]opic:\\s*(?P<topic>.+)\\s*', to=dict) |\r\n        values('topic')\r\n\r\n    for topic in ('find ./mydoc1 -name \"*.txt\" -print' | extract_topic):\r\n        print topic\r\n\r\n    for topic in ('find ../mydoc2 -name \"*.md\" -print' | extract_topic):\r\n        print topic\r\n\r\nRun piped commands and get result\r\n=================================\r\n\r\nThere are 3 ways to execute piped commands and get the result.\r\n\r\n1. Use **run(cmds)** or **cmds.run()** to execute cmds and get the last\r\n   processed data. Use this if you don't need all processed data. Or,\r\n   the tasks you need to do have been done by cascaded Pipe objects.\r\n\r\n2. Use **result(cmds)** or **cmds.result()** to get the processed data\r\n   in a list. Use this method when you need to take all processed data\r\n   to other mechanisms.\r\n\r\n3. Use cmds as a **iterator** to handle the processed data one by one.\r\n   It treats cascaded Pipe objects as a pre-processing function. Use it\r\n   to process data and invoke it by a for loop to do the last processing\r\n   by yourself.\r\n\r\nFunction wrapper\r\n================\r\n\r\nFunction should not be used in pipes directly, unless using auto-type\r\nconversion. Cmdlet provides a set of basic wrappers to wrap function to\r\nPipe object.\r\n\r\npipe.func(generator\\_function)\r\n------------------------------\r\n\r\nThe most basic wrapper. In Python, generator function is a function with\r\nyield statement in it. The generator\\_function defined here is a Python\r\ngenerator function with at least one argument. The first argument is a\r\ngenerator object passed by previous Pipe object. generator\\_function can\r\ntake it as input or just leave it. It looks like:\r\n\r\n.. code:: python\r\n\r\n    # Generator function which use prev as input.\r\n    @pipe.func\r\n    def my_generator(prev):\r\n        for data in prev:\r\n            # ... Put some code to process data ...\r\n            yield new_data\r\n\r\n.. code:: python\r\n\r\n    # Generator function which ignore input.\r\n    @pipe.func\r\n    def my_generator_ignore_prev(prev):\r\n        while True:\r\n            # ... Generate data and break loop in some conditions. ...\r\n            yield data\r\n\r\nFor example:\r\n\r\n.. code:: python\r\n\r\n    @pipe.func\r\n    def randint_generator(prev, num):\r\n        for i in range(num):\r\n            yield random.randint(0, 1000)\r\n\r\n    @pipe.func\r\n    def power(prev, th):\r\n        for n in prev:\r\n            yield n ** th\r\n\r\n    cmds = randint_generator(10) | power\r\n    ans = result(cmds)\r\n    # Equals to:\r\n    # ans = []\r\n    # for i in range(10):\r\n    #     ans.append(random.randint(0, 1000)\r\n\r\npipe.map(function)\r\n------------------\r\n\r\nWrap function to a mapper. The input is a normal function with at least\r\none argument for data input. The returned value will be passed to next\r\nPipe object. It looks like:\r\n\r\n.. code:: python\r\n\r\n    @pipe.map\r\n    def my_mapper(data):\r\n        # ... Put some code to process data ...            \r\n        return new_data\r\n\r\nFor example:\r\n\r\n.. code:: python\r\n\r\n    @pipe.func\r\n    def randint_generator(prev, num):\r\n        for i in range(num):\r\n            yield random.randint(0, 1000)\r\n\r\n    @pipe.map\r\n    def power(n, th):\r\n        return n ** th\r\n\r\n    cmds = randint_generator(10) | power\r\n    ans = result(cmds)\r\n    # Equals to:\r\n    # ans = []\r\n    # for i in range(10):\r\n    #     ans.append(random.randint(0, 1000)\r\n\r\nThe power pipe can also be written in this way:\r\n\r\n.. code:: python\r\n\r\n    power = pipe.map(lambda n, th: n ** th)\r\n\r\nAnything returned by mapper will be sent to next Pipe object. If mapper\r\nreturn None, next Pipe object will receive None. That is, you can't use\r\nmapper to filter data out. That's why we have pipe.filter.\r\n\r\npipe.filter(function)\r\n---------------------\r\n\r\nWrap function to a filter. Filter is a function with at least one\r\nargument as data input. Filter should return Boolean value, True or\r\nFalse. If True, data from previous Pipe object is allowed to pass\r\nthrough. If False, data is dropped. It looks like:\r\n\r\n.. code:: python\r\n\r\n    @pipe.filter\r\n    def my_filter(data):\r\n        # Handle data and check conditions.\r\n        if you_should_not_pass:\r\n            return False\r\n        else:\r\n            return True\r\n\r\nFor example:\r\n\r\n.. code:: python\r\n\r\n    @pipe.filter\r\n    def less_than(data, thrd):\r\n        return data < thrd\r\n\r\n    cmds = range(10) | less_than(3)\r\n    ans = result(cmds)\r\n    # Equals to:\r\n    # ans = []\r\n    # thrd = 3\r\n    # for n in range(10):\r\n    #     if n < thrd:\r\n    #          ans.append()\r\n\r\nYou can write filter pipe in this way:\r\n\r\n.. code:: python\r\n\r\n    less_than = pipe.filter(lambda data, thrd: data < thrd)\r\n\r\npipe.reduce(function)\r\n---------------------\r\n\r\nWrap function as a reducer. A reducer is a function which has at least\r\ntwo arguments. The first one is used as accumulated result, the second\r\none is the data to be processed. A optional keyword argument *init* can\r\nbe used to specify initial value to accumulated result. It looks like:\r\n\r\n.. code:: python\r\n\r\n    @pipe.reduce\r\n    def my_reducer(accum_result, data):\r\n        # Calculate new accum_result according to data.\r\n        return accum_result\r\n\r\nFor example:\r\n\r\n.. code:: python\r\n\r\n    @pipe.reduce\r\n    def count_mod(accum_result, data, mod_by):\r\n        if (data % mod_by) == 0:\r\n            return accum_result\r\n        else:\r\n            return accum_result + 1\r\n\r\n    cmds = range(1000) | count_mod(10, init=0)\r\n\r\npipe.stopper(function)\r\n----------------------\r\n\r\nWrap function as a stopper. Stopper is used to stop the pipe execution.\r\nIt returns true to stop the pipe execution. Return false to pass data to\r\nnext. It looks like:\r\n\r\n.. code:: python\r\n\r\n    @pipe.stopper\r\n    def my_stopper(data):\r\n        if check_stop_criteria(data):\r\n            return True\r\n        return False\r\n\r\nThe usage of wrapper\r\n--------------------\r\n\r\nHere is a example to show how to use function wrapper.\r\n\r\n.. code:: python\r\n\r\n    from random import randint\r\n    from cmdlet.cmds import *\r\n\r\n    @pipe.func\r\n    def random_number(prev, amount):\r\n        for i in range(amount):\r\n            yield randint(0, 100000)\r\n\r\n    @pipe.filter\r\n    def in_range(data, lower_bound, upper_bound):\r\n        return data >= lower_bound and data <= upper_bound\r\n\r\n    @pipe.reduce\r\n    def count(accum_result, data):\r\n        return accum_result + 1\r\n\r\n    @pipe.map\r\n    def format_output(data, format):\r\n        return format % data\r\n\r\n    # Generate 1000 random number and count how many of them between 100 and 500.\r\n    # Then, format the result to 'ans=%d'.\r\n    cmds = random_number(1000) | in_range(100, 500) | count(init=0) | format_output('ans=%d')\r\n\r\n    print cmds.run()\r\n    # >>> ans=40\r\n\r\nIf wrapped code is just a expression, following code shows another way\r\nto make them:\r\n\r\n.. code:: python\r\n\r\n    in_range = pipe.filter(lambda data: data >= lower_bound and data <= upper_bound)\r\n    count = pipe.reduce(lambda accum_result, data: accum_result + 1)\r\n    format_output = pipe.reduce(lambda data, format: format % data)\r\n\r\nNOTE: > As you might already noticed, the number of argument using in\r\npiped commands > is different from the definition of wrapped function.\r\nYou should know your > function is wrapped to a Pipe object. The\r\nfunction is not invoked when > cascading pipes. It is called when using\r\nrun(), result() or iteration. The > arguments will be stored in Pipe\r\nobject and append to the argument list of > wrapped function when it is\r\ninvoked.\r\n\r\nAuto-type conversion\r\n--------------------\r\n\r\nIf the operand of **\\|** operator is not a Pipe object, cmdlet will call\r\nproper creator to convert and wrap it to a Pipe object. The data type of\r\noperand must be registered in cmdlet. Otherwise, exception\r\n*UnregisteredPipeType* will be raised.\r\n\r\ncmdlet.cmds has registered some basic types by default. You can use them\r\ndirectly.\r\n\r\n+------------+------------+-----------------------------------------------+\r\n| Type       | wrapper    | Description                                   |\r\n+============+============+===============================================+\r\n| type       | pipe.map   | Convert processed data to specified type      |\r\n+------------+------------+-----------------------------------------------+\r\n| function   | pipe.map   | Wrap function as a mapper.                    |\r\n+------------+------------+-----------------------------------------------+\r\n| method     | pipe.map   | Wrap method as a mapper.                      |\r\n+------------+------------+-----------------------------------------------+\r\n| tuple      | seq        | Wrap tuple to gernator.                       |\r\n+------------+------------+-----------------------------------------------+\r\n| list       | seq        | Wrap list to gernator.                        |\r\n+------------+------------+-----------------------------------------------+\r\n| str        | sh         | Wrap string to command line and execute it.   |\r\n+------------+------------+-----------------------------------------------+\r\n| unicode    | sh         | Wrap string to command line and execute it.   |\r\n+------------+------------+-----------------------------------------------+\r\n| file       | fileobj    | Wrap file object for read/write operation.    |\r\n+------------+------------+-----------------------------------------------+\r\n\r\ncmdlet.cmds utilities.\r\n======================\r\n\r\ncmdlet.cmds has predefined some commands. Here are brief descriptions.\r\n\r\nPipe commnds for iterable object.\r\n---------------------------------\r\n\r\n+------------+--------------------------------------------------------------+\r\n| Command    | Description                                                  |\r\n+============+==============================================================+\r\n| pack       | Take N elements from pipe and group them into one element.   |\r\n+------------+--------------------------------------------------------------+\r\n| enum       | Generate (index, value) pair from previous pipe.             |\r\n+------------+--------------------------------------------------------------+\r\n| counter    | Count the number of data from previous pipe.                 |\r\n+------------+--------------------------------------------------------------+\r\n| flatten    | Flatten the data passed from previous pipe.                  |\r\n+------------+--------------------------------------------------------------+\r\n| items      | Extract (key, value) pair from a dict-like object.           |\r\n+------------+--------------------------------------------------------------+\r\n| seq        | Extract any iterable object.                                 |\r\n+------------+--------------------------------------------------------------+\r\n| attr       | Extract the value of given attribute from previous pipe.     |\r\n+------------+--------------------------------------------------------------+\r\n| attrs      | Extract the value of given attributes from previous pipe.    |\r\n+------------+--------------------------------------------------------------+\r\n| attrdict   | Extract the value of given attributes from previous pipe.    |\r\n+------------+--------------------------------------------------------------+\r\n\r\nPipe commands for file\r\n----------------------\r\n\r\n+------------+---------------------------------------------+\r\n| Command    | Description                                 |\r\n+============+=============================================+\r\n| stdout     | Output data from previous pipe to stdout.   |\r\n+------------+---------------------------------------------+\r\n| stderr     | Output data from previous pipe to stderr.   |\r\n+------------+---------------------------------------------+\r\n| readline   | Read data from file line by line.           |\r\n+------------+---------------------------------------------+\r\n| fileobj    | Read/write file with pipe data.             |\r\n+------------+---------------------------------------------+\r\n\r\nPipe commands for shell\r\n-----------------------\r\n\r\n+-----------+-----------------------------------------------------------+\r\n| Command   | Description                                               |\r\n+===========+===========================================================+\r\n| sh        | Execute system shell script to handle the stdin/stdout.   |\r\n+-----------+-----------------------------------------------------------+\r\n\r\nPipe commands for strings\r\n-------------------------\r\n\r\nAlias of string method\r\n~~~~~~~~~~~~~~~~~~~~~~\r\n\r\n+--------------------+-----------------------------------------------------+\r\n| Command            | Description                                         |\r\n+====================+=====================================================+\r\n| upper              | alias of string.upper                               |\r\n+--------------------+-----------------------------------------------------+\r\n| lower              | alias of string.lower                               |\r\n+--------------------+-----------------------------------------------------+\r\n| capwords           | alias of string.capwords                            |\r\n+--------------------+-----------------------------------------------------+\r\n| capitalize         | alias of string.capitalize                          |\r\n+--------------------+-----------------------------------------------------+\r\n| lstrip             | alias of string.lstrip                              |\r\n+--------------------+-----------------------------------------------------+\r\n| rstrip             | alias of string.rstrip                              |\r\n+--------------------+-----------------------------------------------------+\r\n| strip              | alias of string.strip                               |\r\n+--------------------+-----------------------------------------------------+\r\n| expandtabs         | alias of string.expandtabs                          |\r\n+--------------------+-----------------------------------------------------+\r\n| strip              | alias of string.strip                               |\r\n+--------------------+-----------------------------------------------------+\r\n| find               | alias of string.find                                |\r\n+--------------------+-----------------------------------------------------+\r\n| format             | alias of % operator of string (not string.format)   |\r\n+--------------------+-----------------------------------------------------+\r\n| rfind              | alias of string.rfind                               |\r\n+--------------------+-----------------------------------------------------+\r\n| count              | alias of string.count                               |\r\n+--------------------+-----------------------------------------------------+\r\n| split              | alias of string.split                               |\r\n+--------------------+-----------------------------------------------------+\r\n| rsplit             | alias of string.rsplit                              |\r\n+--------------------+-----------------------------------------------------+\r\n| swapcase           | alias of string.swapcase                            |\r\n+--------------------+-----------------------------------------------------+\r\n| translate          | alias of string.translate                           |\r\n+--------------------+-----------------------------------------------------+\r\n| ljust              | alias of string.ljust                               |\r\n+--------------------+-----------------------------------------------------+\r\n| rjust              | alias of string.rjust                               |\r\n+--------------------+-----------------------------------------------------+\r\n| center             | alias of string.center                              |\r\n+--------------------+-----------------------------------------------------+\r\n| zfill              | alias of string.zfill                               |\r\n+--------------------+-----------------------------------------------------+\r\n| replace            | alias of string.replace                             |\r\n+--------------------+-----------------------------------------------------+\r\n| join               | alias of string.join                                |\r\n+--------------------+-----------------------------------------------------+\r\n| substitute         | alias of string.Template.substitute                 |\r\n+--------------------+-----------------------------------------------------+\r\n| safe\\_substitute   | alias of string.Template.safe\\_substitute           |\r\n+--------------------+-----------------------------------------------------+\r\n\r\nString split, search and match\r\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n\r\n+------------+------------------------------------------------------------------+\r\n| Command    | Description                                                      |\r\n+============+==================================================================+\r\n| grep       | Grep strings with regular expression.                            |\r\n+------------+------------------------------------------------------------------+\r\n| match      | Grep strings with regular expression and generate MatchObject.   |\r\n+------------+------------------------------------------------------------------+\r\n| wildcard   | Grep strings with wildcard character.                            |\r\n+------------+------------------------------------------------------------------+\r\n| resplit    | Split strings with regular expression.                           |\r\n+------------+------------------------------------------------------------------+\r\n| sub        | Substitute strings with regular expression.                      |\r\n+------------+------------------------------------------------------------------+\r\n| subn       | Substitute strings with regular expression.                      |\r\n+------------+------------------------------------------------------------------+",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "https://github.com/GaryLee/cmdlet/tarball/v0.3.0",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/GaryLee/cmdlet",
        "keywords": "pipe,generator,iterator",
        "license": "UNKNOWN",
        "maintainer": null,
        "maintainer_email": null,
        "name": "cmdlet",
        "package_url": "https://pypi.org/project/cmdlet/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/cmdlet/",
        "project_urls": {
            "Download": "https://github.com/GaryLee/cmdlet/tarball/v0.3.0",
            "Homepage": "https://github.com/GaryLee/cmdlet"
        },
        "release_url": "https://pypi.org/project/cmdlet/0.3.0/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Cmdlet provides pipe-like mechanism to cascade functions and generators.",
        "version": "0.3.0"
    },
    "last_serial": 1833122,
    "releases": {
        "0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d3ca093abc6792afba8af6bfdf7c5c11",
                    "sha256": "1b77ae6996a396f72f39f5716b370f1a4d362f5ca6fa67da56a5cfd83242b713"
                },
                "downloads": -1,
                "filename": "cmdlet-0.2.zip",
                "has_sig": false,
                "md5_digest": "d3ca093abc6792afba8af6bfdf7c5c11",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19940,
                "upload_time": "2015-11-24T16:58:41",
                "url": "https://files.pythonhosted.org/packages/e7/01/af2ead5687594bf1609b738d39d378083559d37b47180f6fc077b93ed5d7/cmdlet-0.2.zip"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0ca6532695e713916439ff72e370c09d",
                    "sha256": "6b1e196b389b0a7772f87868bd63e4fc8a4c201ea39a091fc63d1f4b1216108a"
                },
                "downloads": -1,
                "filename": "cmdlet-0.2.2.zip",
                "has_sig": false,
                "md5_digest": "0ca6532695e713916439ff72e370c09d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20072,
                "upload_time": "2015-11-25T01:05:14",
                "url": "https://files.pythonhosted.org/packages/d1/76/7cfadf37a29dd0a505883eee49a94cdd7a738312e3a47ac945e2c112e910/cmdlet-0.2.2.zip"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bfed8a65d09e23a5c5517d7122212e73",
                    "sha256": "6b3ec3254595ccb9fd03afb8f4fb268cd4394bb67f6aebe50c3666574d690f81"
                },
                "downloads": -1,
                "filename": "cmdlet-0.3.0.zip",
                "has_sig": false,
                "md5_digest": "bfed8a65d09e23a5c5517d7122212e73",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25783,
                "upload_time": "2015-11-25T13:43:15",
                "url": "https://files.pythonhosted.org/packages/22/9c/08ec48169f7d0c9cbf122d40451e89fdca430fe96c6d28de9eee059e0d91/cmdlet-0.3.0.zip"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "bfed8a65d09e23a5c5517d7122212e73",
                "sha256": "6b3ec3254595ccb9fd03afb8f4fb268cd4394bb67f6aebe50c3666574d690f81"
            },
            "downloads": -1,
            "filename": "cmdlet-0.3.0.zip",
            "has_sig": false,
            "md5_digest": "bfed8a65d09e23a5c5517d7122212e73",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 25783,
            "upload_time": "2015-11-25T13:43:15",
            "url": "https://files.pythonhosted.org/packages/22/9c/08ec48169f7d0c9cbf122d40451e89fdca430fe96c6d28de9eee059e0d91/cmdlet-0.3.0.zip"
        }
    ]
}