{ "info": { "author": "Andrea Zoppi", "author_email": "texzk@email.it", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: Scientific/Engineering :: Interface Engine/Protocol Translator", "Topic :: Software Development", "Topic :: Software Development :: Embedded Systems", "Topic :: Utilities" ], "description": "********\nOverview\n********\n\n\n\nLibrary to handle hexadecimal record files\n\n* Free software: BSD 2-Clause License\n\n\nIntroduction\n============\n\nThe purpose of this library is to provide simple but useful methods to load,\nedit, and save hexadecimal record files.\n\nIn the field of embedded systems, hexadecimal record files are the most common\nway to share binary data to be written to the target non-volatile memory, such\nas a EEPROM or microcontroller code flash.\nSuch binary data can contain compiled executable code, configuration data,\nvolatile memory dumps, etc.\n\nThe most common file formats for hexadecimal record files are *Intel HEX*\n(.hex) and *Motorola S-record* (.srec).\nOther common formats for binary data exhange for embedded systems include the\n*Executable and Linkable Format* (.elf), hex dumps (by *hexdump* or *xxd*),\nand raw binary files (.bin).\n\nA good thing about hexadecimal record files is that they are almost *de-facto*,\nso every time a supplier has to give away its binary data it is either in HEX\nor SREC, although ELF is arguably the most common for debuggable executables.\n\nA bad thing is that their support in embedded software toolsets is sometimes\nflawed or only one of the formats is supported, while the supplier provides its\nbinary data in the other format.\n\nAnother feature is that binary data is split into text record lines (thus their\nname) protected by some kind of checksum. This is good for data exchange and\nline-by-line writing to the target memory (in the old days), but this makes\nin-place editing by humans rather tedious as data should be split, and the\nchecksum and other metadata have to be updated.\n\nAll of the above led to the development of this library, which allows to,\nfor example:\n\n* convert between hexadecimal record formats;\n* merge/patch multiple hexadecimal record files of different formats;\n* access every single record of a hexadecimal record file;\n* build records through handy methods;\n* edit sparse data in a virtual memory behaving like a ``bytearray``;\n* extract or update only some parts of the binary data.\n\n\nDocumentation\n=============\n\nFor the full documentation, please refer to:\n\nhttps://hexrec.readthedocs.io/\n\n\nArchitecture\n============\n\nAs the core of this library are record files, the ``hexrec.records`` is the\nfirst module a user should look up.\nIt provides high-level functions to deal with record files, as well as classes\nholding record data.\n\nHowever, the ``hexrec.records`` module is actually an user-friendly interface\nover ``hexrec.blocks``, which manages sparse blocks of data.\nIt also provides a handy wrapper to work with sparse byte chunks with an API\nakin to ``bytearray``.\n\nThe ``hexrec.utils`` module provides some miscellaneous utility stuff.\n\n``hexrec.xxd`` is an emulation of the ``xxd`` command line utility delivered\nby ``vim``.\n\nThe package can also be run as a command line tool, by running the `hexrec`\npackage itself (``python -m hexrec``), providing some record file utilities.\nYou can also create your own standalone executable, or download a precompiled\none from the ``pyinstaller`` folder.\n\nThe codebase is written in a simple fashion, to be easily readable and\nmaintainable, following some naive pythonic *K.I.S.S.* approach by choice.\n\nThis is mainly a library to create and manage sparse blocks of binary data,\nnot made to edit binary data chunks directly.\nPlease consider faster native pythonic ways to create and edit your binary data\nchunks (``bytes``, ``bytearray``, ``struct``, ...).\nAlgorithms can be very slow if misused (this is Python anyway), but they are\nfast enough for the vast majority of operations made on the memory of a\nmicrocontroller-based embedded system.\n\n\n.. image:: _static/architecture.svg\n :alt: Architecture\n\n\nExamples\n========\n\nTo have a glimpse of the features provided by this library, some simple but\ncommon examples are shown in the following.\n\n\nConvert format\n--------------\n\nIt happens that some software tool only supports some hexadecimal record file\nformats, or the format given to you is not handled properly, or simply you\nprefer a format against another (*e.g.* SREC has *linear* addressing, while HEX\nis in a *segment:offset* fashion).\n\nIn this example, a HEX file is converted to SREC.\n\n>>> import hexrec.records as hr\n>>> hr.convert_file('data.hex', 'data.srec')\n\nThis can also be done by running the `hexrec` package as a command line tool:\n\n.. code-block:: sh\n\n $ python -m hexrec convert data.hex data.srec\n\n\nMerge files\n-----------\n\nIt is very common that the board factory prefers to receive a single file to\nprogram the microcontroller, because a single file is simpler to manage for\nthem, and might be faster for their workers or machine, where every second\ncounts.\n\nThis example shows how to merge a bootloader, an executable, and some\nconfiguration data into a single file, in the order they are listed.\n\n>>> import hexrec.records as hr\n>>> input_files = ['bootloader.hex', 'executable.mot', 'configuration.s19']\n>>> hr.merge_files(input_files, 'merged.srec')\n\nThis can also be done by running the `hexrec` package as a command line tool:\n\n.. code-block:: sh\n\n $ python -m hexrec merge bootloader.hex executable.mot configuration.s19 merged.srec\n\n\nDataset generator\n-----------------\n\nLet us suppose we are early in the development of the embedded system and we\nneed to test the current executable with some data stored in EEPROM.\nWe lack the software tool to generate such data, and even worse we need to test\n100 configurations.\nFor the sake of simplicity, the data structure consists of 4096 random values\n(0 to 1) of ``float`` type, stored in little-endian at address ``0xDA7A0000``.\n\n>>> import struct, random\n>>> import hexrec.records as hr\n>>> for index in range(100):\n>>> values = [random.random() for _ in range(4096)]\n>>> data = struct.pack('<4096f', *values)\n>>> hr.save_chunk('dataset_%02d.mot' % index, data, 0xDA7A0000)\n\n\nWrite a CRC\n-----------\n\nUsually, the executable or the configuration data of an embedded system are\nprotected by a CRC, so that their integrity can be self-checked.\n\nLet us suppose that for some reason the compiler does not calculate such CRC\nthe expected way, and we prefer to do it with a script.\n\nThis example shows how to load a HEX file, compute a CRC32 from the address\n``0x1000`` to ``0x3FFB`` (``0x3FFC`` exclusive), and write the calculated CRC\nto ``0x3FFC`` in big-endian as a SREC file.\nThe rest of the data is left untouched.\n\n>>> import binascii, struct\n>>> import hexrec.records as hr\n>>> import hexrec.blocks as hb\n>>> blocks = hr.load_blocks('data_original.hex')\n>>> data = hb.read(blocks, 0x1000, 0x3FFC)\n>>> crc = binascii.crc32(data) & 0xFFFFFFFF # remove sign\n>>> blocks = hb.write(blocks, 0x3FFC, struct.pack('>L', crc))\n>>> hr.save_blocks('data_crc.srec', blocks)\n\nThe same example as above, this time using ``hexrec.blocks.SparseData`` as\na virtual memory behaving almost like ``bytearray``.\n\n>>> import binascii, struct\n>>> import hexrec.records as hr\n>>> memory = hr.load_memory('data.srec')\n>>> crc = binascii.crc32(memory[0x1000:0x3FFC]) & 0xFFFFFFFF\n>>> memory.write(0x3FFC, struct.pack('>L', crc))\n>>> hr.save_memory('data_crc.srec', memory)\n\n\nTrim for bootloader\n-------------------\n\nWhen using a bootloader, it is very important that the application being\nwritten does not overlap with the bootloader. Sometimes the compiler still\ngenerates stuff like a default interrupt table which should reside in the\nbootloader, and we need to get rid of it, as well as everything outside the\naddress range allocated for the application itself.\n\nThis example shows how to trim the application executable record file to the\nallocated address range ``0x8000``-``0x1FFFF``. Being written to a flash\nmemory, unused memory byte cells default to ``0xFF``.\n\n>>> import hexrec.records as hr\n>>> memory = hr.load_memory('app_original.hex')\n>>> data = memory[0x8000:0x20000:b'\\xFF']\n>>> hr.save_chunk('app_trimmed.srec', data, 0x8000)\n\nThis can also be done by running the `hexrec` package as a command line tool:\n\n.. code-block:: sh\n\n $ python -m hexrec cut -s 0x8000 -e 0x20000 -v 0xFF app_original.hex app_trimmed.srec\n\nBy contrast, we need to fill the application range within the bootloader image\nwith ``0xFF``, so that no existing application will be available again.\nAlso, we need to preserve the address range ``0x3F800``-``0x3FFFF`` because it\nalready contains some important data.\n\n>>> import hexrec.records as hr\n>>> memory = hr.load_memory('boot_original.hex')\n>>> memory.fill(0x8000, 0x20000, b'\\xFF')\n>>> memory.clear(0x3F800, 0x40000)\n>>> hr.save_memory('boot_fixed.srec', memory)\n\nWith the command line interface, it can be done via a two-pass processing,\nfirst to fill the application range, then to clear the reserved range.\nPlease note that the first command is chained to the second one via standard\noutput/input buffering (the virtual ``-`` file path, in ``intel`` format as\nper ``boot_original.hex``).\n\n.. code-block:: sh\n\n $ python -m hexrec fill -s 0x8000 -e 0x20000 -v 0xFF boot_original.hex - | \\\n python -m hexrec clear -s 0x3F800 -e 0x40000 -i intel - boot_fixed.srec\n\n(newline continuation is backslash ``\\`` for a *Unix-like* shell, caret ``^``\nfor a *DOS* prompt).\n\n\nInstallation\n============\n\nFrom PIP:\n\n.. code-block:: sh\n\n $ pip install hexrec\n\nFrom source:\n\n.. code-block:: sh\n\n $ python setup.py install\n\n\nDevelopment\n===========\n\nTo run the all tests run:\n\n.. code-block:: sh\n\n $ tox\n\n\nNote, to combine the coverage data from all the tox environments run:\n\n.. list-table::\n :widths: 10 90\n :stub-columns: 1\n\n - - Windows\n - .. code-block:: sh\n\n $ set PYTEST_ADDOPTS=--cov-append\n $ tox\n\n - - Other\n - .. code-block:: sh\n\n $ PYTEST_ADDOPTS=--cov-append tox\n\nChangelog\n=========\n\n0.1.0 (2019-08-13)\n------------------\n\n* Added support for Python 3.7\n\n\n0.0.4 (2018-12-22)\n------------------\n\n* New command line interface made with Click.\n* More testing and fixing.\n* Some refactoring.\n* More documentation.\n\n\n0.0.3 (2018-12-04)\n------------------\n\n* Much testing and fixing.\n* Some refactoring.\n* More documentation.\n\n\n0.0.2 (2018-08-29)\n------------------\n\n* Major refactoring.\n* Added most of the documentation.\n* Added first drafts to manage blocks of data.\n* Added first test suites.\n\n\n0.0.1 (2018-06-27)\n------------------\n\n* First release on PyPI.\n* Added frist drafts to manage record files.\n* Added first emulation of xxd.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/TexZK/hexrec", "keywords": "", "license": "BSD 2-Clause License", "maintainer": "", "maintainer_email": "", "name": "hexrec", "package_url": "https://pypi.org/project/hexrec/", "platform": "", "project_url": "https://pypi.org/project/hexrec/", "project_urls": { "Homepage": "https://github.com/TexZK/hexrec" }, "release_url": "https://pypi.org/project/hexrec/0.1.0/", "requires_dist": [ "click", "enum34", "pathlib", "six" ], "requires_python": "", "summary": "Library to handle hexadecimal record files", "version": "0.1.0" }, "last_serial": 5671896, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "15b35b5b77d55b7ab60ee13d50cc8f22", "sha256": "495742bcef1891af1d55324c8658c8d06fd3000169f7fdb28583303e340f27df" }, "downloads": -1, "filename": "hexrec-0.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "15b35b5b77d55b7ab60ee13d50cc8f22", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18343, "upload_time": "2018-07-03T22:36:28", "url": "https://files.pythonhosted.org/packages/42/8b/0f85bf712b4c8d7e4b972397b68bbc4c1ea6fc7b771f75de313923a11fba/hexrec-0.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "34dfae2c219613d43d5577c07dbdb86e", "sha256": "bacd4ebdcfdef0d0c984eeac510979779bdeb83cd9d0974f5a2c9ad3ed607792" }, "downloads": -1, "filename": "hexrec-0.0.1.tar.gz", "has_sig": false, "md5_digest": "34dfae2c219613d43d5577c07dbdb86e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 25800, "upload_time": "2018-07-03T22:36:29", "url": "https://files.pythonhosted.org/packages/79/4b/c41363f4cf790a91ae8b44cea2d16e5d11b22c7a46d69114eb2827bc6801/hexrec-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "d02cffa4a3a7f2229c7e7c1f72684d6c", "sha256": "f54de69c33d4455b0e31893f7659143a152dc99956da284c2a531fdf18cfa37c" }, "downloads": -1, "filename": "hexrec-0.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d02cffa4a3a7f2229c7e7c1f72684d6c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 35182, "upload_time": "2018-08-29T20:37:05", "url": "https://files.pythonhosted.org/packages/48/0b/aa18b314dae17b7d51221333d97ba036c0ff564c956aece782b21388b50f/hexrec-0.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9eb717963ba5d15c5ab1e0e4916a0301", "sha256": "1b6878281d0248309da3861f8a97f378355a88c26433d68f9e6b582e8e9f2d5e" }, "downloads": -1, "filename": "hexrec-0.0.2.tar.gz", "has_sig": false, "md5_digest": "9eb717963ba5d15c5ab1e0e4916a0301", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 53892, "upload_time": "2018-08-29T20:37:06", "url": "https://files.pythonhosted.org/packages/e6/4b/59f9b0e06f47553e7b52ee340604b530186b98fcb34c9c221c69795b6f42/hexrec-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "e0d410d74e23336fdcaa22b49b48a16d", "sha256": "7114ba4955d3a30c4304195918e79f982f515187f15ad4c0b62f5b45eb799cc3" }, "downloads": -1, "filename": "hexrec-0.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e0d410d74e23336fdcaa22b49b48a16d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 43232, "upload_time": "2018-12-03T23:31:03", "url": "https://files.pythonhosted.org/packages/ee/c3/adcfd16413f483f885ffd8c07f0a7b1f1b186a4454483a04ba01a42237d1/hexrec-0.0.3-py2.py3-none-any.whl" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "4537f96c21c6f8779c96b9367efa6b7c", "sha256": "7a768f78ed38096ade3603695f5d9824dd8b5fa33b9968093395814a23736f72" }, "downloads": -1, "filename": "hexrec-0.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "4537f96c21c6f8779c96b9367efa6b7c", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 46515, "upload_time": "2018-12-22T20:18:26", "url": "https://files.pythonhosted.org/packages/16/dd/ffcf4962f8d04a383b51831524bc68f81b770a72da12d1baea377855729e/hexrec-0.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "40c449ccd1324010dd708bdf3d520501", "sha256": "571d74c834e7f11b91004ab673140b05588a373eca36c1d35916fdf9df7abdc8" }, "downloads": -1, "filename": "hexrec-0.0.4.tar.gz", "has_sig": false, "md5_digest": "40c449ccd1324010dd708bdf3d520501", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16357876, "upload_time": "2018-12-22T20:18:21", "url": "https://files.pythonhosted.org/packages/93/73/33017f52e653be8a18688c99802a4bfaa953c3301b09019864c9f385d526/hexrec-0.0.4.tar.gz" } ], "0.1.0": [ { "comment_text": "", "digests": { "md5": "193b677b2bcb3034471b24d580f3763d", "sha256": "bfc3422c3a6f21b24780234a386a878c89f39714abded90715a3ec0580c2e5ad" }, "downloads": -1, "filename": "hexrec-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "193b677b2bcb3034471b24d580f3763d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 48112, "upload_time": "2019-08-13T13:33:57", "url": "https://files.pythonhosted.org/packages/7d/e6/f90a6c3095a614087815884fd912dd77228e70192e42e0cd1610949fbfac/hexrec-0.1.0-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "193b677b2bcb3034471b24d580f3763d", "sha256": "bfc3422c3a6f21b24780234a386a878c89f39714abded90715a3ec0580c2e5ad" }, "downloads": -1, "filename": "hexrec-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "193b677b2bcb3034471b24d580f3763d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 48112, "upload_time": "2019-08-13T13:33:57", "url": "https://files.pythonhosted.org/packages/7d/e6/f90a6c3095a614087815884fd912dd77228e70192e42e0cd1610949fbfac/hexrec-0.1.0-py2.py3-none-any.whl" } ] }