{
    "info": {
        "author": "Michael Tiller",
        "author_email": "michael.tiller@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "# Recon File Format\n\nThe goal of the `recon` file format (or more precisely, suite of file\nformats) is to provide a compact format for storing (typically\ntime-series oriented) simulation data in a network friendly way.\nAlong the way, we've added support for embedding metadata at the file,\ntable, object and signal level as well.\n\nObviously compactness is important to keep the size of files down.\nDisk space is less an issue these days than it has been in the past.\nNevertheless, one our requirements was to be competitive in terms of\nhow much disk space is required to store signals.\n\nIn trying to develop a network friendly format, we quickly found\nourselves facing two conflicting goals.  On the one hand, we wanted\nthe ability to stream data from simulations to files.  This meant,\nessentially, having a format that was \"append friendly\".  On the other\nhand, we wanted a format where data for an individual signal could be\naccomplished with a \"single\" read (i.e. single request over the\nnetwork).  Note, our definition of \"single\" has some caveats.\n\nWe recognized that we could not accomplish this goal with a single\nformat (at least not without making the implementation extremely\ncomplicated and involving lots of extra writes).  So we opted to\nspecify to file formats that could easily be converted.\n\n## Dependencies\n\nThis package requires an implementation of `msgpack`.  However, there\nare several implementation in Python.  During development, the\n`msgpack-python` package was used.  It is possible that other packages\nmight work as well.\n\n## The \"Wall\" Format\n\nThe wall format is the \"append friendly\" format.  You can think of the\nwall format as a series of \"bricks\" being laid down.  Each \"brick\"\nrepresents some data being added to the file.  One of the nice\nadvantages of this format is that it allows concurrent writing to\nmultiple tables of data.  So in a simulation where different results\nare reported at different intervals, this file format can be used to\nappend different results to different tables.  In other words, it\nsupports appending to multiple tables, not just one.\n\n## The \"Meld\" Format\n\nThe meld format is mainly an archival format.  It rearranges the data\nso that individual signals can be easily extracted.  This is what\nenables data to be extracted with a minimal number of \"reads\" from the\ndata source.  The key issue with reads is the case where the data is\nbeing read over a network.\n\nAs simulation moves to cloud based systems, it will be come\nincreasingly cumbersome to move entire files back and forth between\nthe cloud and the desktop/browser.  Having a format that supports\n\"pulling\" just the information that is required on demand facilitates\nutilizing cloud/remote storage solutions which will lead to more\nresponsive interfaces and better data management practices and\ncapacity.  The meld format is designed for this use case.",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://github.com/xogeny/recon/",
        "keywords": null,
        "license": "MIT",
        "maintainer": null,
        "maintainer_email": null,
        "name": "pyrecon",
        "package_url": "https://pypi.org/project/pyrecon/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/pyrecon/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "http://github.com/xogeny/recon/"
        },
        "release_url": "https://pypi.org/project/pyrecon/0.3.0/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Python implementation of Recon file format and utilities",
        "version": "0.3.0"
    },
    "last_serial": 1018911,
    "releases": {
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e5ea34593a41a86704e7883e34da4727",
                    "sha256": "a784d90dfbbec071c5ac5c93820b52d200e8946b13f053121bda211b45aaa55d"
                },
                "downloads": -1,
                "filename": "pyrecon-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "e5ea34593a41a86704e7883e34da4727",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14075,
                "upload_time": "2014-03-04T16:29:05",
                "url": "https://files.pythonhosted.org/packages/0e/84/349e3da5b24f2ddfcbeb57e2ed5078f3748e4c766571a8b54b4d618d6a3d/pyrecon-0.3.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "e5ea34593a41a86704e7883e34da4727",
                "sha256": "a784d90dfbbec071c5ac5c93820b52d200e8946b13f053121bda211b45aaa55d"
            },
            "downloads": -1,
            "filename": "pyrecon-0.3.0.tar.gz",
            "has_sig": false,
            "md5_digest": "e5ea34593a41a86704e7883e34da4727",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 14075,
            "upload_time": "2014-03-04T16:29:05",
            "url": "https://files.pythonhosted.org/packages/0e/84/349e3da5b24f2ddfcbeb57e2ed5078f3748e4c766571a8b54b4d618d6a3d/pyrecon-0.3.0.tar.gz"
        }
    ]
}