{ "info": { "author": "David Donna", "author_email": "davidadonna@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: ISC License (ISCL)", "Natural Language :: English", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4" ], "description": "Minform\n=======\n\n.. image:: https://img.shields.io/travis/daviddonna/minform.svg\n :target: https://travis-ci.org/daviddonna/minform\n.. image:: http://img.shields.io/pypi/v/minform.svg\n :target: https://pypi.python.org/pypi/minform\n\nThe declarative ease of\n`WTForms `_, the small data\nfootprint of\n`struct `_.\n\nFor more detailed examples, read the full docs at\nhttps://minform.readthedocs.org.\n\nWhy does Minform exist?\n-----------------------\n\nLet's talk about data size.\n\nSay you have a fleet of devices with cell modems that report to a web server.\nHere's an example of a data packet struct you might use:\n\n.. code-block:: c\n\n struct sensor_data {\n char version[8];\n char serial[12];\n int32_t latitude; // fixed-point * 100000\n int32_t longitude; // fixed-point * 100000\n int16_t temperature; // fixed-point * 100\n uint16_t pings;\n uint8_t battery_pct;\n };\n\nLet's say they're reporting their data in JSON format, because that's lighter\nthan XML but still coherent to most server frameworks. Here's a data packet:\n\n.. code-block:: javascript\n\n {\n \"version\": \"1.0\",\n \"serial\": \"DEADBEEF\",\n \"latitude\": 4071270,\n \"longitude\": -7400590,\n \"temperature\": 3200,\n \"pings\": 123,\n \"battery_pct\": 62\n }\n\nIf you take out all the whitespace, that's **144 bytes**. (ASCII encoding).\nMaybe that's all you need, but maybe you need to store billions or trillions\nof these little guys. Worse, maybe you need to pay through the tear ducts for\ncellular data. It would be nice for that data to be smaller, and to be\npredictably sized.\n\nBesides, let's face it: serializing to JSON from C is a pain in the patoot.\nDepending on the library, you could be looking at ten or twenty lines of code\n(or a truly epic ``sprintf``), just to serialize a record.\n\nOn the Python side, we can probably just use a form library like WTForms to\nvalidate incoming data, but we've already paid a price to make that data\nserver-friendly.\n\nWhat can Minform do for me?\n---------------------------\n\nLet's build a Minform form to handle incoming sensor data.\n\n.. code-block:: python\n\n from minform import *\n\n class SensorData(BinaryForm):\n\n order = LITTLE_ENDIAN # let's say our devices are little-endian\n version = BytesField(max_length=8, length=AUTOMATIC)\n serial = BytesField(max_length=12, length=AUTOMATIC)\n latitude = Int32Field()\n longitude = Int32Field()\n temperature = Int16Field()\n pings = UInt16Field()\n battery_pct = UInt8Field()\n\n # We could cut the serialized record off three bytes early, but let's\n # say we want to break on four-byte word boundaries for reasons.\n padding = BlankBytes(3)\n\nHere's the C code that will serialize your structure:\n\n.. code-block:: c\n\n #include \n\n char send_buffer[36];\n struct sensor_data data;\n\n // ...\n\n memcpy(send_buffer, &data, sizeof(data));\n\nAnd here's the Python that will receive it:\n\n.. code-block:: python\n\n form = SensorData.unpack(serialized_data)\n\nThat serialized record is **36 bytes**. 36 on the wire, 36 in a file. You may\nneed to tweak the form definition, depending on your C compiler and the target\narchitecture, but Minforms gives you the tools to cope with padding bytes, and\neven mixed byte ordering.\n\nLet's fill in some gaps\n-----------------------\n\nMinforms are an awful lot like WTForms: you subclass ``minform.BinaryForm``,\nand add ``BinaryField``\\ s as class properties. Here's another quick example:\n\n.. code:: python\n\n import minform\n\n class MyForm(minform.BinaryForm):\n '''\n This is a subclass of wtforms.Form: you can validate data with it,\n construct it from an HTML form, extract the data as a Python dict, etc.\n '''\n first_name = minform.BytesField('First Name', max_length=10)\n last_name = minform.BytesField('Last Name', max_length=10)\n age = minform.UInt8Field('Age')\n\n # first_name (10) last_name (10) age (1)\n packed_data = b'David\\x00\\x00\\x00\\x00\\x00Donna\\x00\\x00\\x00\\x00\\x00\\x18'\n form = MyForm.unpack(packed_data)\n assert form.data == {\n 'first_name': b'David',\n 'last_name': b'Donna',\n 'age': 24,\n }\n\n next_form = MyForm(first_name=b'Foo', last_name=b'Barsson', age=100)\n packed = next_form.pack()\n assert packed == b'Foo\\x00\\x00\\x00\\x00\\x00\\x00\\x00Barsson\\x00\\x00\\x00\\x64'\n\nBecause the library is built on ``struct``, binary serializations of a form's\ndata are in fixed-length buffers. This makes them easier to store, and easy to\nmap onto relatively naive serializations of C structs. It also allows for\nclear documentation of the binary format, because the data maps predictably\nonto different positions in a packed buffer.\n\nCompound BinaryFields allow you to create nested structures that still\nserialize into flat buffers.\n\n.. code:: python\n\n class MyBigBadForm(minform.BinaryForm):\n \"\"\"\n This is taking a turn for campy criminality.\n \"\"\"\n riches = minform.Int16Field()\n goons = minform.BinaryFieldList(Person, max_entries=4, length=minform.EXPLICIT)\n\n squad = MyBigBadForm(riches=55223, goons=[\n {'first_name': 'Joey', 'last_name': 'Schmoey', 'age': 32},\n {'first_name': 'Manny', 'last_name': 'The Man', 'age': 40},\n {'first_name': 'Gerta', 'last_name': 'Goethe', 'age': 52},\n ])\n assert squad.pack() == (b'\\xd7\\xb7' + # riches\n b'\\x03' + # goons prefix\n b'Joey\\0\\0\\0\\0\\0\\0Schmoey\\0\\0\\0\\x20' + # goons[0]\n b'Manny\\0\\0\\0\\0\\0The Man\\0\\0\\0\\x28' + # goons[1]\n b'Gerta\\0\\0\\0\\0\\0Goethe\\0\\0\\0\\0\\x34' + # goons[2]\n b'\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0') # goons[3]\n\nEven with an entire set of blank bytes for ``goons[3]``, that's 87 bytes, vs\n185 for the JSON representation.\n\n\n\n\nHistory\n-------\n\n0.1.0 (10/28/2015)\n\n Initial release\n\n0.2.1 (11/22/2015)\n\n - Add pack_into and unpack_from methods to BinaryForm and BinaryItem.\n - Rename all 'buf' arguments to 'buffer'.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/daviddonna/minform", "keywords": "minform wtforms struct binary", "license": "ISCL", "maintainer": null, "maintainer_email": null, "name": "minform", "package_url": "https://pypi.org/project/minform/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/minform/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/daviddonna/minform" }, "release_url": "https://pypi.org/project/minform/0.2.1/", "requires_dist": null, "requires_python": null, "summary": "WTForms/struct integration to validate and serialize to packed buffers of binary data.", "version": "0.2.1" }, "last_serial": 1828569, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "fe621934ed6d32bcb5c871bf919e508c", "sha256": "9731b533ea395101a815bd21b4cc4d67e4a30f0af7b98759eefba18ea28fca3c" }, "downloads": -1, "filename": "minform-0.1.0.tar.gz", "has_sig": false, "md5_digest": "fe621934ed6d32bcb5c871bf919e508c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24269, "upload_time": "2015-10-29T04:22:16", "url": "https://files.pythonhosted.org/packages/c1/78/80d76b5c3dbb0296d8e3910b8c6e96acbe2423fba47eda6192606d2ce838/minform-0.1.0.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "d6cd1637809c8d841d6811a9355bed13", "sha256": "be9d54c35c6ad6db2c9c305eb9329a076777d1ca6427552d869a835bd3a73016" }, "downloads": -1, "filename": "minform-0.1.2.tar.gz", "has_sig": false, "md5_digest": "d6cd1637809c8d841d6811a9355bed13", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 25658, "upload_time": "2015-11-21T09:11:02", "url": "https://files.pythonhosted.org/packages/ec/52/b3522adb3963acc4d96f04191bd8465b65e258db3d862b0702ea1440c977/minform-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "801397191e864e6cce00d144f1583c27", "sha256": "bb38b59e66fd2dd6fe973dc1dcd95a8a62ff2b073e2de57ebc61243b0af4493b" }, "downloads": -1, "filename": "minform-0.1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "801397191e864e6cce00d144f1583c27", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 15465, "upload_time": "2015-11-22T01:56:43", "url": "https://files.pythonhosted.org/packages/2f/1a/37d68eb34ee977e042b026d2a39620c428cccb29afa538263f98f436e337/minform-0.1.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "22a8bafb5ee562fc5c6a0092e200b833", "sha256": "89672dfc43d1353b70b5ac1b075cdc357e757c1d2a2dc196bba02441f5fc6dae" }, "downloads": -1, "filename": "minform-0.1.3.tar.gz", "has_sig": false, "md5_digest": "22a8bafb5ee562fc5c6a0092e200b833", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26377, "upload_time": "2015-11-22T01:56:30", "url": "https://files.pythonhosted.org/packages/63/b4/5d72ee0e4a19b69216835ae44ce257ff3c86ca2fd285936963dd35c24001/minform-0.1.3.tar.gz" } ], "0.2.0": [], "0.2.1": [ { "comment_text": "", "digests": { "md5": "b0bb03506282da3bed884dfffaca6552", "sha256": "694a23555cad023b334bac8afc1471149c246230c300ecf8acb49dc90b1a34d9" }, "downloads": -1, "filename": "minform-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b0bb03506282da3bed884dfffaca6552", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 16255, "upload_time": "2015-11-22T22:02:25", "url": "https://files.pythonhosted.org/packages/be/b1/1419f6fabbf884fa552f997996f1914fdef5e96c0dd4054021c2adf7ff2d/minform-0.2.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a98da02b1c5efa9730826483419b4f52", "sha256": "713644aee229459519cbcf1bbf8254731098302537e02f98bd43ce3688c76e5c" }, "downloads": -1, "filename": "minform-0.2.1.tar.gz", "has_sig": false, "md5_digest": "a98da02b1c5efa9730826483419b4f52", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 27688, "upload_time": "2015-11-22T22:01:28", "url": "https://files.pythonhosted.org/packages/b3/49/bbb2c727d0d9dfca48ea6b8e60de93d04f390cbf0fc2faeb8fea66041198/minform-0.2.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b0bb03506282da3bed884dfffaca6552", "sha256": "694a23555cad023b334bac8afc1471149c246230c300ecf8acb49dc90b1a34d9" }, "downloads": -1, "filename": "minform-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b0bb03506282da3bed884dfffaca6552", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 16255, "upload_time": "2015-11-22T22:02:25", "url": "https://files.pythonhosted.org/packages/be/b1/1419f6fabbf884fa552f997996f1914fdef5e96c0dd4054021c2adf7ff2d/minform-0.2.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a98da02b1c5efa9730826483419b4f52", "sha256": "713644aee229459519cbcf1bbf8254731098302537e02f98bd43ce3688c76e5c" }, "downloads": -1, "filename": "minform-0.2.1.tar.gz", "has_sig": false, "md5_digest": "a98da02b1c5efa9730826483419b4f52", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 27688, "upload_time": "2015-11-22T22:01:28", "url": "https://files.pythonhosted.org/packages/b3/49/bbb2c727d0d9dfca48ea6b8e60de93d04f390cbf0fc2faeb8fea66041198/minform-0.2.1.tar.gz" } ] }