{
    "info": {
        "author": "Jim Fan",
        "author_email": "",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Console",
            "Programming Language :: Python :: 3",
            "Topic :: Scientific/Engineering :: Artificial Intelligence"
        ],
        "description": "Installation\n============\n\nPyTorch is great. PyTorch on steroids is even better.\n\nTHIS DOC IS INCOMPLETE. MORE COMING SOON.\n\n::\n\n    git clone https://github.com/SurrealAI/TorchX.git torchx\n    pip install -e torchx/\n\nKeras-inspired API\n==================\n\nOne problem with PyTorch is that you have to specify the shapes for each\nmodule, even though some of the shape parameters can be inferred from\nupstream modules. This is especially annoying if you need to tune the\nnetwork architecture.\n\nConsider this sequential CNN architecture for 10-way classification:\n\n.. code:: python\n\n    import torch.nn as nn\n\n    model = nn.Sequential(\n        nn.Conv2d(3, 20, kernel_size=5, stride=2, padding=1),\n        nn.ReLU(),\n        nn.Conv2d(20, 30, kernel_size=7),  # 20 is redundant, can be inferred from above\n        nn.BatchNorm2d(30),  # 30 from the previous conv\n        nn.ReLU(),\n        nn.Conv2d(30, 40, kernel_size=3),  # 30 can be inferred from second conv\n        nn.BatchNorm2d(40),  # 40 from the previous conv\n        nn.ReLU(),\n        Flatten(),  # you have to write your own flatten\n        # Dim after flatten has to be manually calculated (nontrivial!)\n        # What's worse, every time you change some layer's architectural \n        # parameter above,  you will have to recalculate.\n        nn.Linear(1960, 80),\n        nn.ReLU(),\n        nn.Linear(80, 10),  # 80 is once again redundant\n    )\n\n    x = torch.randn((8, 3, 32, 32))\n    y = model(x)\n    print(y.size())  # (8, 10)\n\nTorchX Layer class\n------------------\n\nTorchX features a shape inference engine. The modules will not be\ninstantiated until they have enough information to be constructed. To\naccomplish this, TorchX provides the ``layers`` package that contains\nmost of the modules in ``torch.nn``, but are wrapped as subclasses of\n``torchx.layers.Layer`` instead.\n\nAll ``Layer``\\ s inherit from the standard ``nn.Module``, so they are\nperfectly interoperable with PyTorch in case you'd like to switch back\nand forth. Except for the \"Functional API\" (discussed later), TorchX\n``Layer``\\ s can be interleaved with standard modules when you define\nyour own ``nn.Module``.\n\nWhat's more, you can always retrieve the underlying torch module by\n``mylayer.native_module()``\n\nTo use a single layer:\n\n.. code:: python\n\n    import torchx.layers as L\n\n    x = torch.zeros((16, 10))  # batch size 16, input feature size 10\n    model = L.Linear(20)  # output feature size 20\n    y = model(x) # model weight and bias are lazily instantiated when you invoke it\n    print(y.size())  # (16, 20)\n\nTorchX Sequential API\n---------------------\n\nJust like the builtin ``torch.nn.Sequential``, TorchX features a\nSequential container that eliminates the tedious shape tracking once and\nfor all.\n\nWe take the CNN architecture in the previous section and rewrite it with\nTorchX:\n\n.. code:: python\n\n    import torchx.layers as L\n\n    model = L.Sequential(\n        L.Conv2d(20, kernel_size=5, stride=2, padding=1),\n        L.ReLU(),\n        L.Conv2d(30, kernel_size=7),  # just tell me the output channel size\n        L.BatchNorm2d(),  # input channel dim is inferred\n        L.ReLU(),\n        L.Conv2d(40, kernel_size=3),\n        L.BatchNorm2d(),\n        L.ReLU(),\n        L.Flatten(),  # output dim after flatten is calculated by TorchX\n        L.Linear(80),  # just tell me the hidden size! \n        L.ReLU(),\n        L.Linear(10),\n    )\n\n    x = torch.randn((8, 3, 32, 32))\n    y = model(x)\n    print(y.size())  # (8, 10)\n\nNo sweat!\n\nLayer serialization\n-------------------\n\nTODO: explain more\n\nEach TorchX Layer implements ``to_spec()`` and ``from_spec()`` that\ndumps and constructs a layer from dict. You can specify a new model\narchitecture easily with a JSON/YAML file.\n\nTorchX Functional API\n---------------------\n\nNow we want to define more complex connectivity than ``nn.Sequential``,\nsuch as multi-input multi-output models, directed acyclic graphs, or\nmodels with shared layers. In standard PyTorch, you typically have to\nfollow 3 steps:\n\n1. Subclass ``nn.Module``.\n2. In ``__init__()``, define all the layers with learnable parameters as\n   class attributes. You have to manually declare or calculate all the\n   shapes upfront.\n3. Override ``forward()`` method to specify the connectivity of your\n   network.\n\nThis design gives rise to redundancy and inconvenience, especially when\nyou want to change any significant part of the architecture. You will\nhave to update the attribute declarations in ``__init__``, recalculate\nthe shapes, and make sure the corresponding lines in ``forward()`` are\nkept consistent.\n\nWe illustrate with a diamond-shaped CNN followed by 2 FCs:\n\n.. code:: python\n\n    class MyModel(nn.Module):\n        def __init__(self):\n            super().__init__()\n            # note the unnecessary shape parameter duplications, similar to Sequential\n            self.conv1 = nn.Conv2d(3, 10, kernel_size=5, stride=2)\n            # diamond edges:\n            self.conv2_1 = nn.Conv2d(10, 30, kernel_size=7, padding=2)\n            self.conv2_2 = nn.Conv2d(10, 30, kernel_size=5, padding=1)\n            self.fc1 = nn.Linear(4320, 80)  # 4320 is a non-trivial calculate!\n            self.fc2 = nn.Linear(80, 10)\n\n        def forward(self, x):\n            # omitting nonlinearities for simplicity\n            x = self.conv1(x)\n            branch1 = self.conv2_1(x)\n            branch2 = self.conv2_2(x)\n            x = branch1 * branch2\n            x = x.view(x.size(0), -1)  # flatten\n            x = self.fc1(x)\n            x = self.fc2(x)\n            return x\n\n\n    model = MyModel()\n\n    x = torch.randn((8, 3, 32, 32))\n    y = model(x)\n    print(y.size())  # (8, 10)\n\nNow let's attempt to tune the architecture.\n\nSuppose you want to change the output channel of ``conv1`` to 20, you\nwill have to change *three* places in the code: (1) ``conv1``'s output\nchannel, (2) ``conv2_1``'s input channel, and (3) ``conv2_2``'s input\nchannel.\n\nTo add another FC layer between ``fc1`` and ``fc2``, you will have to\nchange *four* places: (1) ``fc1``'s output dim, (2) ``fc2``'s input dim,\n(3) define a new ``fc3`` in ``__init__``, and (4) add\n``x = self.fc3(x)`` to ``forward()``.\n\nBecause the module definitions and their connectivity are decoupled, you\nwill have to scroll back and forth between ``__init__`` and ``forward``\nto know what exactly are ``conv2_1`` and ``fc3``. This is not a problem\nhere, but would be a headache in bigger modules that span hundreds of\nlines.\n\nIn TorchX, we introduce the **functional API** to automate shape\ndeduction and bring module definitions and connectivity together. We\nstart with a lightweight markup object, ``Placeholder`` that represents\nthe input tensor shape.\n\n.. code:: python\n\n    import torchx.layers as L\n\n    # input image shape: (batch_size, channel, H, W)\n    xp_input = L.Placeholder((8, 3, 32, 32))\n\n    # definition and connectivity right next to each other!\n    # layers take a placeholder and return another placeholder\n    xp = L.Conv2d(10, kernel_size=5, stride=2)(xp_input)\n    # omitting nonlinearities for simplicity\n    branch1 = L.Conv2d(30, kernel_size=7, padding=2)(xp)  # no need to specify input channel\n    branch2 = L.Conv2d(30, kernel_size=5, padding=1)(xp)\n    xp = branch1 * branch2\n    xp = xp.flatten()\n    xp = L.Linear(80)(xp)  # no need to calculate the flattened shape\n    xp = L.Linear(10)(xp)\n\n    # `inputs` and `outputs` keywords can each take a single placeholder, \n    # a list, or even a dict of placeholders. \n    # this defines the signature of model.__call__()\n    model = L.Functional(inputs=xp_input, outputs=xp)\n\n    # model is now fully instantiated, we can give it real tensors\n    x = torch.randn((8, 3, 32, 32))\n    y = model(x)\n    print(y.size())  # (8, 10)\n\nThe functional API makes updating architecture so much easier. Let's\nrepeat the exercise above and note the difference from standard PyTorch:\n\nTo change the output channel of ``branch1`` to 20, you only need to\ntouch one line:\n\n.. code:: python\n\n    xp = L.Conv2d(20, kernel_size=5, stride=2)(xp_input)  # change 10 to 20\n    branch1 = L.Conv2d(30, kernel_size=7, padding=2)(xp)  # unchanged\n    branch2 = L.Conv2d(30, kernel_size=5, padding=1)(xp)  # unchanged\n\nTo add another FC layer between ``fc1`` and ``fc2``, just add one line:\n\n.. code:: python\n\n    xp = L.Linear(80)(xp)  # unchanged\n    xp = L.Linear(50)(xp)  # added line: Linear layer of 50 hidden units\n    xp = L.Linear(10)(xp)  # unchanged\n\nThe functional model is also a subclass of ``nn.Module``, so it plays\nwell with your regular pytorch code. You can stuff it into a regular\n``nn.Module`` definition; it will have all the learnable parameters\nproperly registered.\n\nNon-standard layers\n-------------------\n\nTimeDistributed\n~~~~~~~~~~~~~~~\n\nTODO (give example)\n\nThis container is useful for RNNs. It applies a sequence of layers to\nevery temporal slice of an input.\n\nThe input should be at least 3D, and the dimension at index one will be\nconsidered to be the temporal dimension:\n``(batch_size, time_steps, ...features...)``\n\nLambda\n~~~~~~\n\nTODO\n\nTorchX GPU and Distributed\n==========================\n\nGPU scoping\n-----------\n\nUse ``torchx.device_scope(device_id)`` context manager.\n\n``device_id`` can be any of the following:\n\n-  ``int >= 0``: single GPU index\n-  ``-1``: CPU\n-  ``\"cuda:<n>\"``: single GPU at index ``n``\n-  ``\"gpu:<n>\"``: single GPU at index ``n``, alias of ``cuda:<n>``\n-  ``\"cpu\"``: CPU\n-  list of ints, e.g. ``[0, 3, 5]``: distribute ``torchx.DataParallel``\n   over multiple GPUs at index 0, 3, and 5. More about this later.\n-  ``\"cuda:all\"``: distribute ``torchx.DataParallel`` over all available\n   GPUs on your machine.\n\nAll PyTorch constructor functions within the scope will create tensors\non the designated device. Examples are ``torch.zeros``,\n``torch.ones_like``, ``<mytensor>.new_zeros()``.\n\n.. code:: python\n\n    import torchx as tx\n\n    with tx.device_scope(2):\n        torch.zeros((3, 6))  # on GPU 2 \n\n    with tx.device_scope('gpu:2'):\n        torch.empty(0).new_ones((3, 6))  # on GPU 2 \n\n    with tx.device_scope(-1):\n        torch.ones((3, 6))  # on CPU\n\ntorchx.nn.Module\n----------------\n\nTorchX provides ``torchx.nn.Module`` inherits from the standard\n``nn.Module``. It is a strict superset of features compared to\n``nn.Module``. Besides providing convenient methods like\n``.clip_grad_value()`` and ``.soft_update()``, TorchX Modules are also\naware of ``torchx.device_scope``. When you call the module (upon\n``__call__``) on an input tensor, the module will transfer itself to the\ncurrent device in scope. Standard ``nn.Module`` cannot do that.\n\n.. code:: python\n\n    import torchx as tx\n    import torch.nn as nn  # builtin\n    import torchx.nn as nnx\n\n    # use it the same way as nn.Module\n    class MyModel(nnx.Module):\n        def __init__(self):\n            super().__init__()\n            self.fc1 = nn.Linear(10, 20)\n            self.fc2 = nn.Linear(20, 30)\n\n        def forward(self, x):\n            # omitting nonlinearities for simplicity\n            return self.fc2(self.fc1(x))\n\n\n    with tx.device_scope(3):\n        x = torch.zeros((4, 10))  # on GPU 3\n        model = MyModel()  # still on CPU\n        y = model(x)  # this call automatically transfers model to GPU 3\n        print(y)  # shape (4, 30), on GPU 3\n\nDataParallel\n------------\n\nTODO\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://github.com/SurrealAI/TorchX",
        "keywords": "Deep Learning,Machine Learning",
        "license": "GPLv3",
        "maintainer": "",
        "maintainer_email": "",
        "name": "torchx",
        "package_url": "https://pypi.org/project/torchx/",
        "platform": "",
        "project_url": "https://pypi.org/project/torchx/",
        "project_urls": {
            "Homepage": "http://github.com/SurrealAI/TorchX"
        },
        "release_url": "https://pypi.org/project/torchx/0.9/",
        "requires_dist": null,
        "requires_python": ">=3.5",
        "summary": "PyTorch on steroids",
        "version": "0.9"
    },
    "last_serial": 4414326,
    "releases": {
        "0.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "54855e1e13b0c506838ae9e6fc486203",
                    "sha256": "4ebc0edc7e4c18582fe7a85a63e6906f407b6db61a226222773630b71a738fc1"
                },
                "downloads": -1,
                "filename": "torchx-0.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "54855e1e13b0c506838ae9e6fc486203",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 3384,
                "upload_time": "2018-03-28T05:49:22",
                "url": "https://files.pythonhosted.org/packages/a1/2b/a97e393abba376608b8fe38ff73da042ba96d10236de9b65cb87441f6439/torchx-0.0.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "613266d84fc01f0ae2c43af729726757",
                    "sha256": "c86ab693b762fc31f9ac724d7cf9b8986d2e0693218405b76154b82b24cbf8d1"
                },
                "downloads": -1,
                "filename": "torchx-0.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "613266d84fc01f0ae2c43af729726757",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1050,
                "upload_time": "2018-03-28T05:49:23",
                "url": "https://files.pythonhosted.org/packages/77/7c/290bf36c70f06ea4096d1b19acc8f56c1f9b763785c45b721d6863b5c26d/torchx-0.0.0.tar.gz"
            }
        ],
        "0.9": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f739931c0d5f8809d932dc20ed05e860",
                    "sha256": "f106736882b9e2bbd7288da4dded3d33d156e393aa7e79477f70d237c98c6bc5"
                },
                "downloads": -1,
                "filename": "torchx-0.9-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f739931c0d5f8809d932dc20ed05e860",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5",
                "size": 47323,
                "upload_time": "2018-10-14T01:01:09",
                "url": "https://files.pythonhosted.org/packages/eb/9c/a571131bf94e7695c3dd0b56e75895cd08dd9960d7b2e799e5f5c1a14298/torchx-0.9-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "182dbe2f0634f36ba64ed547069f7f5c",
                    "sha256": "8a09258a315d83de4bca16d93cb14aed27de5c429fb2f0e3057cc7df7dba46d1"
                },
                "downloads": -1,
                "filename": "torchx-0.9.tar.gz",
                "has_sig": false,
                "md5_digest": "182dbe2f0634f36ba64ed547069f7f5c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5",
                "size": 51266,
                "upload_time": "2018-10-14T01:01:11",
                "url": "https://files.pythonhosted.org/packages/96/38/83ddfaf6ad3eabd47c4882afd88fd9de923c3d49150f801156e15555135c/torchx-0.9.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "f739931c0d5f8809d932dc20ed05e860",
                "sha256": "f106736882b9e2bbd7288da4dded3d33d156e393aa7e79477f70d237c98c6bc5"
            },
            "downloads": -1,
            "filename": "torchx-0.9-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "f739931c0d5f8809d932dc20ed05e860",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.5",
            "size": 47323,
            "upload_time": "2018-10-14T01:01:09",
            "url": "https://files.pythonhosted.org/packages/eb/9c/a571131bf94e7695c3dd0b56e75895cd08dd9960d7b2e799e5f5c1a14298/torchx-0.9-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "182dbe2f0634f36ba64ed547069f7f5c",
                "sha256": "8a09258a315d83de4bca16d93cb14aed27de5c429fb2f0e3057cc7df7dba46d1"
            },
            "downloads": -1,
            "filename": "torchx-0.9.tar.gz",
            "has_sig": false,
            "md5_digest": "182dbe2f0634f36ba64ed547069f7f5c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.5",
            "size": 51266,
            "upload_time": "2018-10-14T01:01:11",
            "url": "https://files.pythonhosted.org/packages/96/38/83ddfaf6ad3eabd47c4882afd88fd9de923c3d49150f801156e15555135c/torchx-0.9.tar.gz"
        }
    ]
}