{
    "info": {
        "author": "Daniel Watson",
        "author_email": "daniel.watson@nyu.edu",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "# tf-inputs\n\nThis package provides easy-to-write input pipelines for TensorFlow that automatically\nintegrate with the `tf.data` API.\n\n## Overview\n\nA quick, full example of a training script with an optimized input pipeline:\n\n```python\nimport tensorflow as tf\nimport tf_inputs as tfi\n\n# Recursively find all files inside directory and parse them with `parse_fn`.\ninputs = tfi.Input.from_directory(\n    \"/path/to/data_dir\", parse_fn=tf.image.decode_png, batch_size=16,\n    num_parallel_calls=4\n)\n\n# Supposing `my_model_fn` builds the computational graph of some image model.\n# Built Keras style-- calling the instance returns the iterator input tensor,\n# and until this is done, no ops are added to the computational graph.\ntrain_op, outputs = my_model_fn(inputs())\n\n# Training loop.\nwith tf.Session().as_default():\n    inputs.initialize()  # or `sess.run(inputs.initializer)` is fine too\n    while True:\n        try:\n            inputs.run(train_op)  # replace `sess.run` with `inputs.run`\n        except tf.errors.OutOfRangeError:\n            break\n```\n\nYou may still use `sess.run` if you prefer, though we override it to automatically\nhandle `feed_dict` passing for TensorFlow's feedable iterators and placeholders. If you\nneed to pass an explicit session you may also use `inputs.run(ops, session=sess)`.\n\n## Installation\n\n`tf-inputs` supports TensorFlow 1.13 and python 3.7. We use no other 3rd party python\nmodules. Make sure to have your favorite TensorFlow binary installed (i.e., `tensorflow`,\n`tensorflow-gpu`, or your own custom wheel built from source) prior to installing\n`tf-inputs`.\n\n```\npip install tf-inputs\n```\n\n## Switch between training and validation datasets\n\nThis can get quite messy with the `tf.data` API. See the\n[documentation](https://www.tensorflow.org/guide/datasets#creating_an_iterator)\nyourself. `tf-inputs` handles it this way:\n\n```python\ntrain_inputs = tfi.Input.from_directory(\"/data/training\", **options)\nvalid_inputs = tfi.Input.from_directory(\"/data/validation\", **options)\ninputs = tfi.TrainValidInput(train_inputs, valid_inputs)\n\n...\n\nwith tf.Session().as_default():\n    inputs.initialize()\n    inputs.run([train_op, output_op])  # receives a training batch\n    inputs.run(output_op, valid=True)  # receives a validation batch\n```\n\nIf you do not have separate datasets for training and validation, you may use:\n\n```python\ninputs = tfi.TrainValidSplit(inputs, num_valid_examples)\n```\n\n## Methods to read data\n\n`tf-inputs` supports a variety of ways to read data besides `Input.from_directory`:\n\n```python\n# Provide the file paths yourself:\ninputs = tfi.Input.from_file_paths([\"data/file1.txt\", \"data/file2.txt\"], **options)\n```\n\n```python\n# Provide the `tf.data.Dataset` instance yourself (yielding single input elements):\ninputs = tfi.Input.from_dataset(dataset, **blah)\n```\n\n```python\n# Same as above, but preventing any graph building a priori:\ninputs = tfi.Input.from_dataset_fn(get_dataset, **blah)\n```\n\n```python\n# Lowest level: subclass `tfi.Input` and override `read_data` to return the dataset:\nclass MyInput(tfi.Input):\n    def __init__(self, my_arg, my_kwarg=\"foo\", **kwargs):\n        super().__init__(**kwargs)\n        self.my_arg = myarg\n        ...\n\n    def read_data(self):\n        return tf.data.Dataset.from_tensor_slices(list(range(self.my_arg)))\n```\n\nUsually there is no need to use the lower level methods. One common example is when the\nuser wishes to yield `(input, label)` pairs and they live in different files. You may\nuse `tfi.Zip` for this, as long as the number of elements match:\n\n```python\n# Multi task learning training input pipeline.\nsentences_en = tfi.Input.from_directory(\"data/training/english\")\nsentences_fr = tfi.Input.from_directory(\"data/training/french\")\nsentiment_labels = tfi.Input.from_directory(\"data/training/labels\")\n\ninputs = tfi.Zip(images, sentences_fr, sentiment_labels)\n\ndef my_model(inputs, training=True):\n    if training:\n        x, y1, y2 = inputs\n    ...\n```\n\n## Training over multiple epochs\n\nJust catch the `tf.errors.OutOfRangeError` and restart the iterator:\n\n```python\n# Inside a `tf.Session`:\ninputs.initialize()\nwhile epoch < max_epochs:\n    try:\n        inputs.run(train_op)\n    except tf.errors.OutOfRangeError:\n        inputs.initialize()\n        epochs += 1\n```\n\n## Multiple elements per file\n\nJust set `flatten=True` flag with `Input.from_directory` or `Input.from_file_paths`:\n\n```python\n# Inputs split by an arbitrary delimiter in a text file:\ninputs = tfi.Input.from_directory(\n    'path/to/text/files', batch_size=32, flatten=True,\n    parse_fn=lambda x: tf.string_split(x, delimiter='\\n\\n'),\n)\n```\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/danielwatson6/tf-inputs.git",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "tf-inputs",
        "package_url": "https://pypi.org/project/tf-inputs/",
        "platform": "",
        "project_url": "https://pypi.org/project/tf-inputs/",
        "project_urls": {
            "Homepage": "https://github.com/danielwatson6/tf-inputs.git"
        },
        "release_url": "https://pypi.org/project/tf-inputs/0.2.3/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Input pipelines for TensorFlow that make sense.",
        "version": "0.2.3"
    },
    "last_serial": 5208556,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2f5310035b7bcfcf6f775d8d0dd2c9b5",
                    "sha256": "d8b8159c9c7b2811654c8f3650db443c169bae3389a42be7624ab68c6a7e59a3"
                },
                "downloads": -1,
                "filename": "tf_inputs-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "2f5310035b7bcfcf6f775d8d0dd2c9b5",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9971,
                "upload_time": "2019-03-10T10:48:50",
                "url": "https://files.pythonhosted.org/packages/dd/69/c55f23ebec14edaa1b2c61d491ecb9edb29c6ae5922d5193c7182b42769a/tf_inputs-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7cc2ac18c630190bebf831c36b407075",
                    "sha256": "93eecd255ede8f7ab5b14de09d481403a9a60628bdbb2d92790c57a492cdae9e"
                },
                "downloads": -1,
                "filename": "tf-inputs-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "7cc2ac18c630190bebf831c36b407075",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10126,
                "upload_time": "2019-03-10T10:48:52",
                "url": "https://files.pythonhosted.org/packages/a3/16/00911d237d9c85174045cbd242b5f1a305ea6b4185209a118ba15496c06c/tf-inputs-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b159d521923ec17e7f817ca60abbab13",
                    "sha256": "dfcb4285a3ccae07b624c6553750b4530187e54ddda57a8f81e7c2610ad7b90e"
                },
                "downloads": -1,
                "filename": "tf_inputs-0.2.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b159d521923ec17e7f817ca60abbab13",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9242,
                "upload_time": "2019-03-15T08:39:40",
                "url": "https://files.pythonhosted.org/packages/90/99/22913c7163b77b3da9e26df5515389c596ed59eab72a21b96f33c591245b/tf_inputs-0.2.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "a3ee66d5cf348fb2e99c27891e6e857c",
                    "sha256": "a967c1434751b5fda225a05ec7e5ccb9b92e542b40e7cf3b1942d217543081d3"
                },
                "downloads": -1,
                "filename": "tf-inputs-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "a3ee66d5cf348fb2e99c27891e6e857c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8880,
                "upload_time": "2019-03-15T08:39:42",
                "url": "https://files.pythonhosted.org/packages/ed/8f/6468fe3b4802f183840f141d2b7f42911699aa38eea5ed779829df259098/tf-inputs-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e8fae5b6dfccb47da7411086edfc95e0",
                    "sha256": "f9f67ef3a70a8f7db81e6b82db83f40d930ca15e03eb2c44ed2ba428776263af"
                },
                "downloads": -1,
                "filename": "tf_inputs-0.2.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e8fae5b6dfccb47da7411086edfc95e0",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9330,
                "upload_time": "2019-03-20T10:13:35",
                "url": "https://files.pythonhosted.org/packages/d0/b3/8453dc871622635dcb5538b023fd1ec35bb69d0a9bd14df621a0259f0c79/tf_inputs-0.2.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7e83bbf17ff7ad8386c120b0af62e75b",
                    "sha256": "cc38fc844a86fcf1b40a4e928fc8a50489013be38ecf18213e2f3b5374e6b9d5"
                },
                "downloads": -1,
                "filename": "tf-inputs-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "7e83bbf17ff7ad8386c120b0af62e75b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9511,
                "upload_time": "2019-03-20T10:13:37",
                "url": "https://files.pythonhosted.org/packages/35/de/fc493590352535f926c3f5d7ed7021f06030e1a8648547cafb875cca1077/tf-inputs-0.2.1.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5bb43403eb3031a556e810d2bb0026c6",
                    "sha256": "64c863020b445c8e3e7ce59c7f6b54e4fdd218a144079e7d8ee69aafbe632853"
                },
                "downloads": -1,
                "filename": "tf_inputs-0.2.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "5bb43403eb3031a556e810d2bb0026c6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9453,
                "upload_time": "2019-04-14T08:21:04",
                "url": "https://files.pythonhosted.org/packages/f1/4b/2d94064b4b12740a6fc1457358d10d9881cb2e5254fb46654c8dc8a48080/tf_inputs-0.2.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "47a223ecbd5a5301602de2f73e080a1e",
                    "sha256": "65f6aaeb7c4fd2effaf8b4ecc8acaa344fe3b25f011d7af632a831a39a026325"
                },
                "downloads": -1,
                "filename": "tf-inputs-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "47a223ecbd5a5301602de2f73e080a1e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9704,
                "upload_time": "2019-04-14T08:21:06",
                "url": "https://files.pythonhosted.org/packages/00/13/88b44be170f80fdd4988948136c91e55b568e8322eec2078b0d68a733d06/tf-inputs-0.2.2.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "99430de8749b0225fef62707e81b6929",
                    "sha256": "f758d1ed184f31ece2bfecee6a745c80d59bcb006d9c858290ddb5e92d67682a"
                },
                "downloads": -1,
                "filename": "tf_inputs-0.2.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "99430de8749b0225fef62707e81b6929",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9877,
                "upload_time": "2019-04-30T14:47:33",
                "url": "https://files.pythonhosted.org/packages/36/f0/948bf5683d1b70d5e7639a94f4986d68ff900f876a3199279458774f794b/tf_inputs-0.2.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "4211204e415415db25176bddc2485abe",
                    "sha256": "b29cc31b66d131ec34e73d41c146d0bce1cfd716655dcd6cd3482d0ab0a87fa1"
                },
                "downloads": -1,
                "filename": "tf-inputs-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "4211204e415415db25176bddc2485abe",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10160,
                "upload_time": "2019-04-30T14:47:37",
                "url": "https://files.pythonhosted.org/packages/73/27/86e199869c4601fabefb891230783e732e1e79f239b1d0c9b508191b4a93/tf-inputs-0.2.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "99430de8749b0225fef62707e81b6929",
                "sha256": "f758d1ed184f31ece2bfecee6a745c80d59bcb006d9c858290ddb5e92d67682a"
            },
            "downloads": -1,
            "filename": "tf_inputs-0.2.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "99430de8749b0225fef62707e81b6929",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 9877,
            "upload_time": "2019-04-30T14:47:33",
            "url": "https://files.pythonhosted.org/packages/36/f0/948bf5683d1b70d5e7639a94f4986d68ff900f876a3199279458774f794b/tf_inputs-0.2.3-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "4211204e415415db25176bddc2485abe",
                "sha256": "b29cc31b66d131ec34e73d41c146d0bce1cfd716655dcd6cd3482d0ab0a87fa1"
            },
            "downloads": -1,
            "filename": "tf-inputs-0.2.3.tar.gz",
            "has_sig": false,
            "md5_digest": "4211204e415415db25176bddc2485abe",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 10160,
            "upload_time": "2019-04-30T14:47:37",
            "url": "https://files.pythonhosted.org/packages/73/27/86e199869c4601fabefb891230783e732e1e79f239b1d0c9b508191b4a93/tf-inputs-0.2.3.tar.gz"
        }
    ]
}