{ "info": { "author": "Ernesto Monroy", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Topic :: Utilities" ], "description": "# JSON Table\n\nA little package to convert a JSON into a table! This project was born out of a need to transform many JSONs mined from APIs to something that Pandas or a relational database could understand. The difference between this package and json path packages is that its designed to create __tables__, not just extract single values.\n\n\n\n\"latest\n\n\n\n## How to install\n\nThe package is available through pypi. So simply go to your command line and:\n\n```bash\npip install jsontable\n```\nYou're also welcome to download the code from github and modify to suit your needs. And if you have time let me know what cool functionality you added and we can improve the project!\n\n## How it works\n\nIt works in a similar manner to JSON parsers\n1. Create a converter object\n2. Give the converter a list of paths you want to explore and how you want to name each column\n3. Give the converter a __decoded__ JSON object you want to read, and it returns a table\n\n## Usage\n\nHere is a quick example to get you going\n\n```python\nimport jsontable as jsontable\n\n#Create a list of paths you want to extract\npaths = [{\"$.id\":\"id\"},\t{\"$.name\":\"name\"}, {\"$.address.city\":\"city\"}]\n#The JSON object you want to explore\nsample = {\"id\":\"1\",\"name\":\"Foo\",\"address\":{\"city\":\"Bar\"}}\n\n#Create an instance of a converter\nconverter = jsontable.converter()\n#Set the paths you want to extract\nconverter.set_paths(paths)\n#Input a JSON to be interpreted\nconverter.convert_json(sample)\n```\n\nIn this case, you will get a table with two columns and two rows (header and first row of data) like these:\n\n```python\n[['id', 'name', 'city'], ['1', 'Foo', 'Bar']]\n```\n\nFor more examples, refer to the [tests folder](/tests)\n\n## How it works\n\n#### JSON Paths\n\nEach path you specify is a column in your final table. Each path that is setup is expanded according to the [standard JSON Path functionality](https://goessner.net/articles/JsonPath/). This is, for each path, the converter starts at the root of the JSON object and navigates each step (a.k.a node) of the path in order. When it reaches the final step in the path (a.k.a leaf), it outputs the resulting element of the JSON into the cell.\n\nThe __final cell value__ is converted based on the [standard JSON values](https://www.json.org/) as follows:\n\n|JSON Value|Conversion|Sample Output|\n|--|--|--|\n|object|stringified object |`'{\"city\":\"Bar\"}'`|\n|array|stringified array |`'[1,2,3]'`|\n|string|string|'Foo'|\n|number|number|4.7|\n|boolean|stringidied boolean|'False'|\n|null|None|None|\n|_missing value_ (i.e. the path did not find an element)|None|None|\n\nThe intention behind stringifying the object, array and boolean is to be able to pass the output to other data libraries (e.g [Pandas](https://pandas.pydata.org/)) or to a relational database. \n\n#### Array Expansion\n\n__With the exception of the final node__, array elements are automatically expanded into rows. So for example a path `'$.a.b'` applied to a JSON `{\"a\":[{\"b\":1},{\"b\":2}]}` would result into two rows `[[1],[2]]`. The array expansion functionality can be applied to the final node by explicitly using the `*` operator as a final step (e.g. `$.a.*`)\n\nExample:\n```python\npaths = [{\"$.name\":\"Name\"},{\"$.telephones.type\":\"Telephone Type\"},{\"$.telephones.number\":\"Telephone Number\"}]\nsample = {\n\t\t\t\"name\":\"Foo\",\n\t\t\t\"telephones\":[\n\t\t\t\t{\"type\":\"mobile\", \"number\":\"0000\"},\n\t\t\t\t{\"type\":\"home\", \"number\":\"1111\"}\n\t\t\t]\n\t\t}\nconverter = jsontable.converter()\nconverter.set_paths(paths)\nconverter.convert_json(sample)\n```\nResult:\n```\n[['Name', 'Telephone Number', 'Telephone Type'], ['Foo', '0000', 'mobile'], ['Foo', '1111', 'home']]\n```\n\nThe reverse of this functionality (not expand arrays if they are encountered before the end) is not implemented only due to the lack of need.\n\n#### Joining Columns\n\nSince a path may result in multiple rows, there is the need to be able to combine the result of each column into the same table. The joining mechanism is similar to an SQL join, where each cell (row-cell combination) is \"matched\" to a row in the result using a \"matching value\". The matching value in this case is the last common element of the paths. \n\nThis is best illustrated with an example, the following table shows the transformations applied to the sample JSON.\n\n```python\nsample = {\n\t\"contacts\":[\n\t\t{\n\t\t\t\"name\":\"Foo\",\n\t\t\t\"telephones\":[\n\t\t\t\t{\"type\":\"mobile\", \"number\":\"0000\"},\n\t\t\t\t{\"type\":\"home\", \"number\":\"1111\"}\n\t\t\t],\n\t\t\t\"emails\":[\n\t\t\t\t{\"type\":\"work\", \"email\":\"foo@w.com\"},\n\t\t\t\t{\"type\":\"personal\", \"email\":\"foo@p.com\"}\n\t\t\t]\n\t\t},\n\t\t{\n\t\t\t\"name\":\"Bar\",\n\t\t\t\"telephones\":[\n\t\t\t\t{\"type\":\"mobile\", \"number\":\"2222\"},\n\t\t\t\t{\"type\":\"home\", \"number\":\"3333\"}\n\t\t\t],\n\t\t\t\"emails\":[\n\t\t\t\t{\"type\":\"work\", \"email\":\"bar@w.com\"},\n\t\t\t\t{\"type\":\"personal\", \"email\":\"bar@p.com\"}\n\t\t\t]\n\t\t}\n\t]\n}\n```\n\n\n\t\n\t\n\t\t\n\t\t\n\t\n\t\t\n\t\t\n\t\t\n\t\n
PathsResult
\n\t\t\t
\n[\n\t{\"$.contacts.name\":\"Name\"},\n\t{\"$.contacts.telephones.type\":\"Type\"},\n\t{\"$.contacts.telephones.number\":\"Number\"}\n]\n\t\t\t
\n\t\t
\n\t\t\t
\n[\n\t['Name', 'Type', 'Number'], \n\t['Foo', 'mobile', '0000'], \n\t['Foo', 'home', '1111'], \n\t['Bar', 'mobile', '2222'], \n\t['Bar', 'home', '3333']\n]\n\t\t\t
\n\t\t
\n\t\t\t
\n[\n\t{\"$.contacts.name\":\"Name\"},\n\t{\"$.contacts.telephones.number\":\"Number\"},\n\t{\"$.contacts.emails.email\":\"Email\"}\n]\n\t\t\t
\n\t\t
\n\t\t\t
\n[\n\t['Name', 'Number', 'Email'], \n\t['Foo', '0000', 'foo@w.com'], \n\t['Foo', '1111', 'foo@w.com'],\n\t['Foo', '0000', 'foo@p.com'], \n\t['Foo', '1111', 'foo@p.com'],  \n\t['Bar', '0000', 'bar@w.com'], \n\t['Bar', '1111', 'bar@w.com'],\n\t['Bar', '0000', 'bar@p.com'], \n\t['Bar', '1111', 'bar@p.com'],  \n]\n\t\t\t
\n\t\t
\n\nIn the first case, the `type` and `number` have a common path `telephone` and therefore the columns are combined for the same telephone element. If we then look at the `name` path it has a common path `contacts` with the rest of the columns, and therefore, the value is repeated across the rows.\n\nIn the second case the `email` and `number` only have a common path `contacts` and since each path results in two rows, the only possible way to match these is to combine all the values, resulting in 4 rows per contact (total 8 rows since there are 2 contacts). \n\n## Operators\n\nCurrently there are two operators supported: * and ~\n\n|Syntax|Description|\n|--|--|\n|`*`| Returns __all values__ of the current element. If its an array, it will return one row per array value. If its an object (dictionary in Python) it will return one row per __value__. If its a value (string, number, boolean, null), it returns the same value|\n|`~`| Return __all indices__ of the current element. If its an array, it returns an ascending numbered sequence starting with 0 (e.g. [1,2] would return [[0],[1]]) . If its an object, it will return the keys (e.g. {\"a\":1,\"b\":2} would return [['a'],['b']]). If its a value it returns 0|\n\nMore operators will be implemented in later releases.\n\n## New in this version\n\n - A bug that was preventing list expansions at different depths (e.g. $.a as well as $.b.c) has been fixed.\n - Implementation of the * and ~ operators\n\nBoth these changes were made possible by changing the search method from depth first to breadth first, as well as recursing through a tree rather than iterating through one column at a time. \n\n## Coming up\n\nIn the wishlist we have:\n - Filtering\n - List indexing\n - More functions (basic arithmetics, string concatenation and expansion)\n - Square bracket notation ($[a][b] for $.a.b)\n - Stringify objects as an option\n - Option to output pandas style named array\n - Method to set paths and convert at the same time\n - CSV Output/Input\n\n## References\n\nI want to mention that whilst I inted to expand the functionality of this package, at the moment it can only take a simple sequence of keys to navigate a path. This is, the full functionality proposed by Stefan Gossner in his [jsonpath](https://goessner.net/articles/JsonPath/) is not yet implemented.... but we will get there.\n\nIf you are looking for a package that simply extracts a single value from a JSON by using more complex paths (and its functions), I recommend you look at [jsonpath-rw](https://github.com/kennknowles/python-jsonpath-rw) by Kenn Knowles [jsonpath-ng](https://pypi.org/project/jsonpath-ng/) by Tomas Aparicio or [jsonpath2](https://pypi.org/project/jsonpath2/) by Mark Borkum. \n\n## Final disclaimer\n\nI will continue to look for improvements in the package and hopefully add some useful functionality. Given the current popularity of the package, the maintenance is in a best effort manner. However if you have issues or bugs to report let me know [here](/issues) and I will try my best to help. \n\nYou can use this package as you wish, but unfortunatelly, I cannot take responsibility of how this code is used, or the results it provides. It is up to you to test this does what you want it to!\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ernestomonroy/jsontable", "keywords": "json mining etl extract transform etltools data parsing parse mapper relational table", "license": "", "maintainer": "", "maintainer_email": "", "name": "jsontable", "package_url": "https://pypi.org/project/jsontable/", "platform": "", "project_url": "https://pypi.org/project/jsontable/", "project_urls": { "Bug Reports": "https://github.com/ernestomonroy/jsontable/issues", "Homepage": "https://github.com/ernestomonroy/jsontable", "Source": "https://github.com/ernestomonroy/jsontable" }, "release_url": "https://pypi.org/project/jsontable/0.1.1/", "requires_dist": null, "requires_python": ">=3.5", "summary": "Convert a JSON to a table", "version": "0.1.1" }, "last_serial": 5907322, "releases": { "0.0.2": [ { "comment_text": "", "digests": { "md5": "4ee0bd5a3c6a8fd0df6c768a8d974a86", "sha256": "8f24d809df9d49d12c57fbd6093e21a8929afe94e6a5eb24f6d7c0ff72b86177" }, "downloads": -1, "filename": "jsontable-0.0.2.tar.gz", "has_sig": false, "md5_digest": "4ee0bd5a3c6a8fd0df6c768a8d974a86", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 3417, "upload_time": "2019-06-27T10:16:01", "url": "https://files.pythonhosted.org/packages/82/67/6ef1b32637b63d2ddce7e3d78b47ea1bf5afcf14395af8b3c25ddfbe9994/jsontable-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "6299860f50093f9ab7abe93b82957f0f", "sha256": "d0f1821bd8305ca87225ded776d8d07c75167164d017b61af1afb8721db49d9e" }, "downloads": -1, "filename": "jsontable-0.0.3.tar.gz", "has_sig": false, "md5_digest": "6299860f50093f9ab7abe93b82957f0f", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 4351, "upload_time": "2019-06-27T13:18:42", "url": "https://files.pythonhosted.org/packages/b0/f7/c1a2bc5cb29a4cce24761a6febb07371c16970e5bfa2151cab6bea612afc/jsontable-0.0.3.tar.gz" } ], "0.1.0": [ { "comment_text": "", "digests": { "md5": "3cef34bfef73106c0dac7dd625a424e1", "sha256": "e84dd5644d6ec6a371cd962e94feea4d1107f1f08722c9b60172484b45ac8bc7" }, "downloads": -1, "filename": "jsontable-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "3cef34bfef73106c0dac7dd625a424e1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 6175, "upload_time": "2019-08-09T13:31:26", "url": "https://files.pythonhosted.org/packages/fc/80/1c09920f3da03f05b7159ec06e10c723da35991f747c551adc14dfac2149/jsontable-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b4aafe7153d0285473d3093f85365a1e", "sha256": "9dd7b50eab27abb422fffd8e066d1a7eaf79230ce631e92428f79c6a31b37813" }, "downloads": -1, "filename": "jsontable-0.1.0.tar.gz", "has_sig": false, "md5_digest": "b4aafe7153d0285473d3093f85365a1e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 5694, "upload_time": "2019-08-09T13:31:27", "url": "https://files.pythonhosted.org/packages/b5/97/0e3802813829295a957aaa50fc2763670dbb5926499a38abaa44ee1151b0/jsontable-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "273ca750508f2017d955250ace4d383c", "sha256": "d7f166e6f89bddcc18cebb86742297e1e613287e618d4b2ff9245866d9e3bce7" }, "downloads": -1, "filename": "jsontable-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "273ca750508f2017d955250ace4d383c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 7782, "upload_time": "2019-09-30T15:03:33", "url": "https://files.pythonhosted.org/packages/19/80/6e8cb772822a4668cd27ace838b836807e6a538fbd36cc4152c9eeb8e47c/jsontable-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "17de1faef67ea7492e03d3bdd7b2a8f8", "sha256": "10505ad871db98a509d04fd408c34b032402c9a410f224ea46d9b9249703b819" }, "downloads": -1, "filename": "jsontable-0.1.1.tar.gz", "has_sig": false, "md5_digest": "17de1faef67ea7492e03d3bdd7b2a8f8", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 8529, "upload_time": "2019-09-30T15:03:34", "url": "https://files.pythonhosted.org/packages/a6/ab/c367ce6f85cf0623ace14b178e507853e0b27d506421af58c0707ea90621/jsontable-0.1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "273ca750508f2017d955250ace4d383c", "sha256": "d7f166e6f89bddcc18cebb86742297e1e613287e618d4b2ff9245866d9e3bce7" }, "downloads": -1, "filename": "jsontable-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "273ca750508f2017d955250ace4d383c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 7782, "upload_time": "2019-09-30T15:03:33", "url": "https://files.pythonhosted.org/packages/19/80/6e8cb772822a4668cd27ace838b836807e6a538fbd36cc4152c9eeb8e47c/jsontable-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "17de1faef67ea7492e03d3bdd7b2a8f8", "sha256": "10505ad871db98a509d04fd408c34b032402c9a410f224ea46d9b9249703b819" }, "downloads": -1, "filename": "jsontable-0.1.1.tar.gz", "has_sig": false, "md5_digest": "17de1faef67ea7492e03d3bdd7b2a8f8", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 8529, "upload_time": "2019-09-30T15:03:34", "url": "https://files.pythonhosted.org/packages/a6/ab/c367ce6f85cf0623ace14b178e507853e0b27d506421af58c0707ea90621/jsontable-0.1.1.tar.gz" } ] }