{
"info": {
"author": "Paul Kosek",
"author_email": "pckosek@fcps.edu",
"bugtrack_url": null,
"classifiers": [
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3"
],
"description": "# easy_tfrecords\n\n### this package is designed to assist reading and writing to tfrecord files in an intuitive way that preserves dtype and data structure\n\n### _Purpose_:
\nThe tfrecord format is a fast and powerful way of feeding data to a tensorflow model; it can automatically batch, randomize and iterate your data across epochs without special instructions. The **problem** with using tfrecord files comes from orchestrating the madness of matching feature structures across the reader, writer and fetcher.\n
\nThe **easy_tfrecords** module contains methods and classes that allow you to write to and read from tfrecord files in a straightforward, extensible manner.\n\n### _Features_:
\n- create tfrecord files\n- read from single or multiple tfrecord files\n- selectively read data from tfrecord files\n- examine the data structure of tfrecord files\n\n### _Usage_:
\n#### **Writing**
\n- Import data into python however you normally would (excel, pandas, csv, matlab, etc.)\n- Reshape each of your arrays of features to `shape=[N, x[, y[, z[, etc.]]]]` where N is the number of features. \n - Add multiple lists of features to the file as key-value pairs\n#### **Reading**
\n- Create a reader class object, specifying your file list (can be length 1), optionally specifying batch size and shuffe spec.\n- pass a list of which inputs to read from the file\n\n#### Example Code:\n```python\nimport numpy as np\nimport tensorflow as tf\n\nfrom easy_tfrecords import create_tfrecords, easy_tfrecords as records\n\n\n# CREATE SOME TEST DATA\nx = np.array([[0, 0, 0, 0], [0, 0, 0, 0]], np.int32)\ntrainX = np.asarray( [x, x+1, x+2] )\n\ny = np.array([0.25], np.float32)\ntrainY = np.asarray( [y, y+1, y+2] )\n\n\n# CREATE AND SAVE TO A FEW TFRECORDS FILES\ncreate_tfrecords('tfr_1.tf', x=trainX, y=trainY)\ncreate_tfrecords('tfr_2.tf', x=trainX+10, y=trainY+10)\ncreate_tfrecords('tfr_3.tf', x=trainX+100, y=trainY+100, z=trainY+100)\n\n# INSTANTIATE THE RECORDS OBJECT\nrec = records(files=['data_1.tf', 'data_2.tf'],\n shuffle=False,\n batch_size=1, \n keys=['x', 'y'])\n\nnext_factory = rec.get_next_factory()\n\nbatch_x = next_factory['x']\nbatch_y = next_factory['y']\n\nwith tf.Session() as sess:\n\n sess.run(rec.get_initializer())\n\n for n in range(10):\n print('------------')\n print('n => {}\\n'.format(n))\n\n x_eval, y_eval = sess.run( [batch_x, batch_y] )\n print('x_eval=\\n{}\\n'.format(x_eval))\n print('y_eval=\\n{}'.format(y_eval))\n\nsess.close()\n```\n#### Output :\n```\n------------\nn => 0\n\nx_eval=\n[[ 0.25]]\n\ny_eval=\n[[[0 0 0 0]\n [0 0 0 0]]]\n------------\nn => 1\n\nx_eval=\n[[ 1.25]]\n\ny_eval=\n[[[1 1 1 1]\n [1 1 1 1]]]\n------------\nn => 2\n\nx_eval=\n[[ 2.25]]\n\ny_eval=\n[[[2 2 2 2]\n [2 2 2 2]]]\n------------\nn => 3\n\nx_eval=\n[[ 100.25]]\n\ny_eval=\n[[[100 100 100 100]\n [100 100 100 100]]]\n------------\nn => 4\n\nx_eval=\n[[ 101.25]]\n\ny_eval=\n[[[101 101 101 101]\n [101 101 101 101]]]\n------------\nn => 5\n\nx_eval=\n[[ 102.25]]\n\ny_eval=\n[[[102 102 102 102]\n [102 102 102 102]]]\n------------\nn => 6\n\nx_eval=\n[[ 10.25]]\n\ny_eval=\n[[[10 10 10 10]\n [10 10 10 10]]]\n------------\nn => 7\n\nx_eval=\n[[ 11.25]]\n\ny_eval=\n[[[11 11 11 11]\n [11 11 11 11]]]\n------------\nn => 8\n\nx_eval=\n[[ 12.25]]\n\ny_eval=\n[[[12 12 12 12]\n [12 12 12 12]]]\n------------\nn => 9\n\nx_eval=\n[[ 0.25]]\n\ny_eval=\n[[[0 0 0 0]\n [0 0 0 0]]]\n```\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/pckosek/easy_tfrecords",
"keywords": "tensorflow tensor machine learning tfrecord",
"license": "",
"maintainer": "",
"maintainer_email": "",
"name": "easy-tfrecords",
"package_url": "https://pypi.org/project/easy-tfrecords/",
"platform": "",
"project_url": "https://pypi.org/project/easy-tfrecords/",
"project_urls": {
"Homepage": "https://github.com/pckosek/easy_tfrecords"
},
"release_url": "https://pypi.org/project/easy-tfrecords/0.1.0/",
"requires_dist": null,
"requires_python": "",
"summary": "Package to streamline reading and writing data to tfrecord files",
"version": "0.1.0"
},
"last_serial": 4551423,
"releases": {
"0.0.1": [
{
"comment_text": "",
"digests": {
"md5": "b3cf374d3cf8a7323bf0991d0979c083",
"sha256": "c90fb6795dc938a77bf8f5b78fd3da12c4df16d074f0a1306e0b211dd0231049"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b3cf374d3cf8a7323bf0991d0979c083",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5829,
"upload_time": "2018-11-09T02:52:05",
"url": "https://files.pythonhosted.org/packages/d0/87/1847486806d5f9bc10668307fda423224b50845aa45c40ba2a41b0eae8e5/easy_tfrecords-0.0.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "ecd3c128f11f129fe18509c320a0ccfd",
"sha256": "8218426d8b00ddf0726f3e82a9d48c43ef8c1cc7b02afb913c142a7385bc3211"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "ecd3c128f11f129fe18509c320a0ccfd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5148,
"upload_time": "2018-11-09T02:52:08",
"url": "https://files.pythonhosted.org/packages/e0/bd/5020f0e84c200222302aead9681d4ce9949262df409c2c8288fa68ee3c3f/easy_tfrecords-0.0.1.tar.gz"
}
],
"0.0.2": [
{
"comment_text": "",
"digests": {
"md5": "1d58ae124c43321c76f68d5157cccbf8",
"sha256": "737a856b20d0f8c799054d5807cd9d6f00aa894135737c024886f1a84c7f64cf"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1d58ae124c43321c76f68d5157cccbf8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5084,
"upload_time": "2018-11-09T03:04:41",
"url": "https://files.pythonhosted.org/packages/cb/54/3c23414166494ce5a807f740fe2da23ddbcffaeff00cdf02f5fd43f76087/easy_tfrecords-0.0.2-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "b8284ab4b2a60434f60f088be7314f86",
"sha256": "10bed1938ac9a747c497b18aaa0f78354c1614b955163039635ad78e1a7b7ec4"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "b8284ab4b2a60434f60f088be7314f86",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4948,
"upload_time": "2018-11-09T03:04:48",
"url": "https://files.pythonhosted.org/packages/d8/3a/b6eb89e52820a328c2cc66e06427401e42c7659ae65beae2d8c2e99ac418/easy_tfrecords-0.0.2.tar.gz"
}
],
"0.0.3": [
{
"comment_text": "",
"digests": {
"md5": "125d76203a22f685ec1517dbe6db029d",
"sha256": "4e4a74b00e152677878394ffa70f959df27489f3d6b89305959f24faadd1dcc9"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "125d76203a22f685ec1517dbe6db029d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5083,
"upload_time": "2018-11-09T13:16:33",
"url": "https://files.pythonhosted.org/packages/fb/bb/59ace7f7ddad21fcb47566fc3d01e7358940d2e3cf08122936c24f82880a/easy_tfrecords-0.0.3-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "d06878f8990ce92b582e3c6074199a54",
"sha256": "a7134354041cdaec1ee1b4e9ba56342614e1d1954e940232c5b1c00966ab8e9d"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "d06878f8990ce92b582e3c6074199a54",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4947,
"upload_time": "2018-11-09T13:16:34",
"url": "https://files.pythonhosted.org/packages/42/e3/06740691a9037c346c5ad9af2d5e63dc89f8bc471e3fc704374a561ad4f3/easy_tfrecords-0.0.3.tar.gz"
}
],
"0.0.4": [
{
"comment_text": "",
"digests": {
"md5": "e0eed82abc7c57ab6233c04bce3e5597",
"sha256": "3397fbac53a3ab85d68235cd9983f6ac86fc1587bc7a1ebed1ec75dfb24da8b8"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e0eed82abc7c57ab6233c04bce3e5597",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5150,
"upload_time": "2018-11-10T03:41:38",
"url": "https://files.pythonhosted.org/packages/85/e7/e78ef33c4b6d461f51ffbe6929c173ebae8765a032a978448b813e5b8973/easy_tfrecords-0.0.4-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "ff9d0550fca463034f18bb369bc3f884",
"sha256": "467118bb641ab434a941a99cce57ea6f100074a76f364747f8883474d3dc871b"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "ff9d0550fca463034f18bb369bc3f884",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5019,
"upload_time": "2018-11-10T03:41:39",
"url": "https://files.pythonhosted.org/packages/c9/f4/90e81242a5ac3f7fda344d1d01c5f8ca19ce4861406b1585e4d90f24abf6/easy_tfrecords-0.0.4.tar.gz"
}
],
"0.0.5": [
{
"comment_text": "",
"digests": {
"md5": "d77d343ae2f5d6e87ac79e34f4746469",
"sha256": "b39af7b3a2361260f146e602d7b60d3f6a4343489852f11a2fbe6d84e78dee70"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d77d343ae2f5d6e87ac79e34f4746469",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5155,
"upload_time": "2018-11-11T02:52:56",
"url": "https://files.pythonhosted.org/packages/36/8c/581cbaed5131a5a1234548700f91fecd5e037c474f2d898e0efa3ed8cc64/easy_tfrecords-0.0.5-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "b9670ca091d2f7cc999f12cfdb0e2ef6",
"sha256": "ef130be5f8570047ee4ddfe3154ba865bd805914d926b90f104b466e04281b3d"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "b9670ca091d2f7cc999f12cfdb0e2ef6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5008,
"upload_time": "2018-11-11T02:52:58",
"url": "https://files.pythonhosted.org/packages/f8/59/ab1e6f64d77e39dd4fae021415a0f55670877207a1bc5e7c651a6c6a728c/easy_tfrecords-0.0.5.tar.gz"
}
],
"0.0.6": [
{
"comment_text": "",
"digests": {
"md5": "eaaa09e344fa240f6862634ca70b7682",
"sha256": "b8d6bdc22a62dd5b435f8b79c53b1e930025eb0cc3ad4c89357bf0cef494077e"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "eaaa09e344fa240f6862634ca70b7682",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5172,
"upload_time": "2018-11-11T03:15:14",
"url": "https://files.pythonhosted.org/packages/de/46/6b9ed46fee06bc65ca8b3b0fdb5b0c9740c12dde37584a2346fa7bed1402/easy_tfrecords-0.0.6-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "cb66def562743cdbb0e8749a2cca8ba7",
"sha256": "e9e210ca89399a544d052d0f3b0ce4e658d318990351fe2529b6436e135fb426"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "cb66def562743cdbb0e8749a2cca8ba7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5016,
"upload_time": "2018-11-11T03:15:16",
"url": "https://files.pythonhosted.org/packages/f2/93/90d137d1bfa795de4a24e9a097d7a6b121d481ba2e5aecef0bcaeff88861/easy_tfrecords-0.0.6.tar.gz"
}
],
"0.0.7": [
{
"comment_text": "",
"digests": {
"md5": "b9ed0cc2c520ce436f976b48b78088f4",
"sha256": "5283cb0f56cba16e67b069d09cc8c68336bd81ddebd04e3c6f02435008a0f973"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b9ed0cc2c520ce436f976b48b78088f4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5217,
"upload_time": "2018-11-15T01:58:18",
"url": "https://files.pythonhosted.org/packages/f2/9f/8a7c7d4a907c9750e9748fe345a3339c80ae3635dae514042bfdc1ef4f55/easy_tfrecords-0.0.7-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "8ef26d3ae1a95f575e23be9635ba1cfa",
"sha256": "1be6d773d46ee214e26921f154e3bc30c1c1c02e5133fc4e275fed9acb3490fb"
},
"downloads": -1,
"filename": "easy_tfrecords-0.0.7.tar.gz",
"has_sig": false,
"md5_digest": "8ef26d3ae1a95f575e23be9635ba1cfa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5065,
"upload_time": "2018-11-15T01:58:20",
"url": "https://files.pythonhosted.org/packages/cd/66/4d04b52a408bcb1cc395f93e447b86690ce11a0f7d791f199867bce0049c/easy_tfrecords-0.0.7.tar.gz"
}
],
"0.1.0": [
{
"comment_text": "",
"digests": {
"md5": "3a3a78dee57b5e4cd5f1c4dfded0ed84",
"sha256": "e38dc019e8e9c039c1b4b7e78a4b207d48591cb05919148c1a694559a7b9b7c7"
},
"downloads": -1,
"filename": "easy_tfrecords-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3a3a78dee57b5e4cd5f1c4dfded0ed84",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5257,
"upload_time": "2018-12-02T02:26:40",
"url": "https://files.pythonhosted.org/packages/6d/85/fd2285fc471153b46d4bd655f2954432f90bb0f017f2207962f4520fb473/easy_tfrecords-0.1.0-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "8c51ed8e7ee3c5cfff33100190466593",
"sha256": "86098faeee9a9d7214de25386b9fdfb1e589da7e9060fb2fffc21a9aa555f2d0"
},
"downloads": -1,
"filename": "easy_tfrecords-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "8c51ed8e7ee3c5cfff33100190466593",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5119,
"upload_time": "2018-12-02T02:26:42",
"url": "https://files.pythonhosted.org/packages/4d/c3/103ef91a3ca3c103ae9e0047724610349e385c814ba7ce7de70d5fd3a819/easy_tfrecords-0.1.0.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "3a3a78dee57b5e4cd5f1c4dfded0ed84",
"sha256": "e38dc019e8e9c039c1b4b7e78a4b207d48591cb05919148c1a694559a7b9b7c7"
},
"downloads": -1,
"filename": "easy_tfrecords-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3a3a78dee57b5e4cd5f1c4dfded0ed84",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5257,
"upload_time": "2018-12-02T02:26:40",
"url": "https://files.pythonhosted.org/packages/6d/85/fd2285fc471153b46d4bd655f2954432f90bb0f017f2207962f4520fb473/easy_tfrecords-0.1.0-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "8c51ed8e7ee3c5cfff33100190466593",
"sha256": "86098faeee9a9d7214de25386b9fdfb1e589da7e9060fb2fffc21a9aa555f2d0"
},
"downloads": -1,
"filename": "easy_tfrecords-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "8c51ed8e7ee3c5cfff33100190466593",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5119,
"upload_time": "2018-12-02T02:26:42",
"url": "https://files.pythonhosted.org/packages/4d/c3/103ef91a3ca3c103ae9e0047724610349e385c814ba7ce7de70d5fd3a819/easy_tfrecords-0.1.0.tar.gz"
}
]
}