{
    "info": {
        "author": "Henri Lefebvre",
        "author_email": "henri.pasdecalais@yahoo.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# ColumnFile : column oriented file storage\n\nColumnFile is a simple column oriented file storage solution for analyzing all kinds of data (csv, json, ...) using local files.\n\nThis solution allows efficient CRUD operations by key. Details concerning the implementation are available here : [see implementation details](https://operations-research.blog/2018/07/08/building-a-local-column-oriented-database-in-python/). It produces a file organization allowing fast access to data as follow : \n\n![How data is organized by column-file](src/structure.png)\n\n## Installation\n\n**With Pip :**\n``` pip3 install ColumnFile ```\n\n**Usage :**\n\n``` from ColumnFile import ColumnFile as db ```\n\n## Documentation\n\n### Create and open operations\n- `create(dbname, schema)` : creates the storage architecture and sets the db schema. The schema parameter must be a python dict containing two attributes : 'hash' and 'sort'. Both are lists of couples '(attribute_name, attribute_types)'. Allowed types are in ['string', 'int', 'float'].\n- `open(dbname)` : opens an existing database\n\n### Reading operations\n\n- `get(key, report_error = False)` : returns the item associated to the key passed as parameter. If the 'report_error' parameter is set to False, empty objetcs are returned when no item matches the key. If True, the function raises an exception when nothin is found.\n- `scan(sub_key, row_filter = lambda _: True)` : returns an iterator on all the values matching the sub_key. Values are sorted in ascending order with respected with the sort key. No particular order is imposed for hash keys. The row_filter function which can be passed as parameter allows you to filter the iterated values. Note that even if a filter is provided, all values are scanned. However, only those fullfilling the filter will be returned. \n\n### Writing operations\n\n- `put(key, column_values)` : the put operation can be use to update a row by replacing all its columns by those passed in the column_values parameter. If no row matches the given key, a new row is created. \n- `merge(key, column_values)` : the merge operation can be use to update a row by merging the existing columns with the columns passed as parameter in column_values. If no row matches the given key, a new row is created. \n- `delete(key)` : deletes a row\n- `commit()` : commits the changes to the database. Note that if no commit is performed after write operations, the database will not take them into account. \n\n## Example codes\n\n```python\nfrom ColumnFile import ColumnFile\nfrom random import randint\nimport numpy as np\n\ndb = ColumnFile(verbose = True)\n\n# create the database\ndb.create(\"demo-db\", {\n    \"hash\": [ (\"year\", \"integer\") ],\n    \"sort\": [ (\"month\", \"integer\"), (\"day\", \"integer\") ]\n})\n\n# open the existing database\ndb.open(\"demo-db\")\n\n# let's add some data\nfor year in range(2012, 2018):\n    for month in range(1, 13):\n        for day in range(1, 32):\n            operation_type = ['sell', 'buy'][randint(0,1)]\n            amount = randint(1, 500)\n            db.put((year, month, day), { \"operation_type\": operation_type, \"amount\": amount })\n\n# change a specific column value\ndb.merge((2016, 6, 23), { \"operation_amount\": 250 })\n\n# do NOT forget to commit\ndb.commit()\n\n# retrieve one specific row and print its operation type\nkey, values = db.get((2016, 6, 23))\nprint(values['operation_amount'])\n\n# compute average sell amount in 2016\nresults = db.scan(2016, row_filter = lambda row: row[1]['operation_type'] == 'sell')\nsum_amount, n = sum(map(lambda row : np.array([row[1]['amount'], 1]), results))\naverage = sum_amount / n\nprint('Average = ', average)\n\n```\n\n## Example projects\n\n- [COMING] [Modeling railway networks with the time expanded model](http://operations-research.blog)\n\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/hlefebvr/column-file",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "ColumnFile",
        "package_url": "https://pypi.org/project/ColumnFile/",
        "platform": "",
        "project_url": "https://pypi.org/project/ColumnFile/",
        "project_urls": {
            "Homepage": "https://github.com/hlefebvr/column-file"
        },
        "release_url": "https://pypi.org/project/ColumnFile/0.0.5/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Simple library to manage local file storage within an application based on the concept of local partition and sort keys",
        "version": "0.0.5"
    },
    "last_serial": 4041132,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1b02c8417725f90bb1483d92ac0749f3",
                    "sha256": "0c31a39c05f35b6e58dda44ab0fffc26ed512100c83d1a1056f9d7d4084db9be"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1b02c8417725f90bb1483d92ac0749f3",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9148,
                "upload_time": "2018-07-08T16:00:39",
                "url": "https://files.pythonhosted.org/packages/ba/20/2d7def4ce5567526d4ee0196931e51237324c7e6e7167d30aee09d597aa8/ColumnFile-0.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "496976fe56e67ee55c2f7ba191f3cc2b",
                    "sha256": "4ffbc053377436607cab23d7cb276baf51b976b94451a9170a834bcec02b83f2"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "496976fe56e67ee55c2f7ba191f3cc2b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7822,
                "upload_time": "2018-07-08T16:00:42",
                "url": "https://files.pythonhosted.org/packages/fc/ec/8233813897f3fdb239ebd3ec3fa1eeab8502ff5c7cfc27ac15d49ff4ccc1/ColumnFile-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a4c6f28f36a44232eaad18ecf31c4c66",
                    "sha256": "8a752c88a4e7ef9777f84996929501348c78f1ed4cf0776f78e3e41a5af31c55"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a4c6f28f36a44232eaad18ecf31c4c66",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9149,
                "upload_time": "2018-07-08T16:14:09",
                "url": "https://files.pythonhosted.org/packages/b0/58/c5d4fd0ec018a92a2e10109ff10f7b584493da75a5936bc240640062ab02/ColumnFile-0.0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "b7f7e57bdc141deeb310bedd5f335be4",
                    "sha256": "61bff820d20a212d1658c189cdea7b1cd4c106fd5fcc40ed1d18c9505db0b538"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "b7f7e57bdc141deeb310bedd5f335be4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7824,
                "upload_time": "2018-07-08T16:14:10",
                "url": "https://files.pythonhosted.org/packages/da/da/7a1b6495043426c2b1fa0c8659d09bbb27b446d92a141af08e1576435500/ColumnFile-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ff1ecb2e3b8e3a772ad22e0e311994ba",
                    "sha256": "d979756f02a70a0690802568134cad2a57a871b1f85eb38f68453afc37331aa0"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ff1ecb2e3b8e3a772ad22e0e311994ba",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9150,
                "upload_time": "2018-07-08T16:29:59",
                "url": "https://files.pythonhosted.org/packages/41/14/803ff1bbdeb05f59b8f6229aaaf08cc1bcc609dc3cc9f28efb9807c44930/ColumnFile-0.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "cc30a7dc4c316262f07a47ae231f41ef",
                    "sha256": "6065a56691f2057e27d931f672c43f79eda1a4a7a73f7281d98c797d685739d0"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "cc30a7dc4c316262f07a47ae231f41ef",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7821,
                "upload_time": "2018-07-08T16:30:00",
                "url": "https://files.pythonhosted.org/packages/45/45/7e6c869486f080a8d8acc8f9d923da3e75c34c6e90774b372753298bea14/ColumnFile-0.0.3.tar.gz"
            }
        ],
        "0.0.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "02a590ffbfe3d10060884460bc13be48",
                    "sha256": "0a50e83804a702a658121162d7902557dbf3cd2b7dfa20331dd00aab150b0724"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.3.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "02a590ffbfe3d10060884460bc13be48",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 11334,
                "upload_time": "2018-07-08T17:05:21",
                "url": "https://files.pythonhosted.org/packages/18/25/6f89bf0b6d7b625dec86349510b6302fa9b4a717e6686a1493bf54e0e37d/ColumnFile-0.0.3.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "6359dc0bafa97c58925857a08bdf0206",
                    "sha256": "850c6aeb975da30b2d8d353b4304826f3efa07a1964801647231f2594a33f4b0"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "6359dc0bafa97c58925857a08bdf0206",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10806,
                "upload_time": "2018-07-08T17:05:22",
                "url": "https://files.pythonhosted.org/packages/ab/5a/7d5e6161f83fc6dbc78014a79d1f770d5698397177b6c1ddaf4ca0b054dd/ColumnFile-0.0.3.1.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f5c829a06d6cde1315295aa7cfbb0a53",
                    "sha256": "2e80a4e356bbf1192c51e0bd833dcefcea52d6a866b543a3cd400a3e80f19168"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f5c829a06d6cde1315295aa7cfbb0a53",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 11314,
                "upload_time": "2018-07-08T17:06:40",
                "url": "https://files.pythonhosted.org/packages/14/6a/23e7e8a969ada5d25e44a9daf92c4232b30764ef571218c36e5d99df901f/ColumnFile-0.0.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "622f664bf58b7de2f2c29e39aac1597b",
                    "sha256": "bee3f7d8b844baeb8a2c8ec165c1ae39b7a6a2420b272cf68b46b6727ba4cd48"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "622f664bf58b7de2f2c29e39aac1597b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10801,
                "upload_time": "2018-07-08T17:06:42",
                "url": "https://files.pythonhosted.org/packages/a7/a8/45d393a88d4d8a8ec6b0273be6ccdb543509fda25e2ac07aecb569ac8250/ColumnFile-0.0.4.tar.gz"
            }
        ],
        "0.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c18fc6441c92d16375fc8e8b6535b846",
                    "sha256": "993685493e3f406bd068919836eee88b539301c7f3105e6958678bd4945a387b"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "c18fc6441c92d16375fc8e8b6535b846",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 11365,
                "upload_time": "2018-07-08T17:30:31",
                "url": "https://files.pythonhosted.org/packages/6f/53/877742af36a1bf2f32ac90551bdfab7a77c2ecc32d5786970d30689cc931/ColumnFile-0.0.5-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7c8236e4e06430981d21c222036e5a18",
                    "sha256": "c2482b726e4433bc674ba3687e46630ca3057d8ab31809353be1915041dbb3c5"
                },
                "downloads": -1,
                "filename": "ColumnFile-0.0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "7c8236e4e06430981d21c222036e5a18",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10945,
                "upload_time": "2018-07-08T17:30:33",
                "url": "https://files.pythonhosted.org/packages/97/9b/98b39db784e38a047527ee2ebae2076baab8e3a64bf8eb0b19b7b23649a6/ColumnFile-0.0.5.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "c18fc6441c92d16375fc8e8b6535b846",
                "sha256": "993685493e3f406bd068919836eee88b539301c7f3105e6958678bd4945a387b"
            },
            "downloads": -1,
            "filename": "ColumnFile-0.0.5-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c18fc6441c92d16375fc8e8b6535b846",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 11365,
            "upload_time": "2018-07-08T17:30:31",
            "url": "https://files.pythonhosted.org/packages/6f/53/877742af36a1bf2f32ac90551bdfab7a77c2ecc32d5786970d30689cc931/ColumnFile-0.0.5-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "7c8236e4e06430981d21c222036e5a18",
                "sha256": "c2482b726e4433bc674ba3687e46630ca3057d8ab31809353be1915041dbb3c5"
            },
            "downloads": -1,
            "filename": "ColumnFile-0.0.5.tar.gz",
            "has_sig": false,
            "md5_digest": "7c8236e4e06430981d21c222036e5a18",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 10945,
            "upload_time": "2018-07-08T17:30:33",
            "url": "https://files.pythonhosted.org/packages/97/9b/98b39db784e38a047527ee2ebae2076baab8e3a64bf8eb0b19b7b23649a6/ColumnFile-0.0.5.tar.gz"
        }
    ]
}