{ "info": { "author": "Mirko M\u00e4licke", "author_email": "mirko.maelicke@kit.edu", "bugtrack_url": null, "classifiers": [], "description": "# metacatalog\n\nManagement tool for the V-FOR-WaTer metadata database application.\n\n## Install\n\nThere are two different use-cases. Either you want to connect to an \nexisting instance of the database, or you need to install the database itself as well.\nIn both cases you need to install the python package.\n\nA) Install from PyPI\n\n```bash\npip install metacatalog\n```\n\nB) Install from Github\n\nRight now, the version on Github is newer, as the package is under heavy development. \nTherefore, until a stable version 1.0 is released, it is recommended to install the package via \nGithub.\n\n```bash\ngit clone https://github.com/vforwater/metacatalog.git\ncd metacatalog\npython setup.py install\n```\n\nIn either case, you can use the cli to save a default connection into a file in your home folder.\n**Be aware that any password saved along with the default connection is saved in clear-text!!**\n\n```bash\nmetacatalog connection --save postgresql://postgres:@localhost:5432/metacatalog\n```\n\nIn may be not the best idea to save the postgres user password in cleartext. You can of course\ncreate a new user with limited rights and save this information. Alternatively, the connection can \nbe specified on each call of the cli. This way there is no password saving.\n\nNow, you are ready to use the cli. In case you need to install the database itself as well, follow the \ninstructions below:\n\n## Install the database\n\nFirst you need to install PostgreSQL and the PostGIS extension. There are preinstalled binaries \nfor windows. \nOn Linux the commands might look similar to:\n\n```bash\nsudo apt install postgresql postgis\n```\n\nPostGIS will in many cases be a rather outdated version. This is up to now not a big issue, as \nmetacatalog uses only a limited amount of spatial functions. Anything > v2.0 should be fine.\n\nNext, you need to install the database and create the extension. The database name should fit \nthe one specified in the connection string above (or change the string). You can open a SQL\nconsole to postgresql or use psql:\n\n```SQL\ncreate database metacatalog with encoding='UTF8';\ncreate extension postgis;\n```\n\nNow, you are ready to go and let the CLI create all neccessary tables.\n\n## Create Tables and load Defaults\n\nThe `create`command can be used to create all needed tables in the connected database.\nYou can specify the connection string using the `--connection` flag. If not supplied, the\nCLI will search for a saved 'default' connection string. In case there is none, the CLI will \nerror.\n\n```bash\nmetacatalog create --connection postgresql://postgres:@localhost:5432/metacatalog\n```\n\nwill output:\n\n```\nCreating Tables.\nDone.\n```\n\nThe next step is to populate the tables with some useful default data. \nThis step is optional, but recommended. As of this writing, the `populate`\ncommand will load records into the `units`, `variables`, `licenses` and `keywords`.\n\n```bash\nmetacatalog populate\n```\n\nwill output:\n\n```bash\nPopulating datasource_types\nFinished datasource_types\nPopulating units\nFinished units\nPopulating variables\nFinished variables\nPopulating licenses\nFinished licenses\nPopulating person_roles\nFinished person_roles\nPopulating keywords\nFinished keywords\n```\n\nThe `--ignore` flag can be used to omit one or many tables from population.\nInstead of using `create` and `populate`, the `init` will run both in only \none step. The `init` command will accept the same flags.\nCreating a new instance in a test database can be initialized like:\n\n```bash\nmetacatalog init -C postgresql://postgres:@localhost:5432/test --ignore units variables\n```\n\nThis will create the same structure and data in the `test` database. The `variables` and `units` table will, however, be empty.\n\n## Find data\n\nBefore storing your actual metadata into the database, it makes sense to \nlearn how auxiliary information can be found in the database. The CLI exposes an `find` command to find records on exact matches. In a future release, a `search` endpoint will be added as well. \n\nNote: As of this writing the `find` command can not operate on all tables.\n\nWe can find all stored licenses by `find`ing them without any filter:\n\n```bash\nmetacatalog find licenses\n```\n```\nOpen Data Commons Open Database License \nOpen Data Commons Attribution License v1.0 \nOpen Data Commons Public Domain Dedication and License \n```\n\nA filter can be added using the `--by` flag. This flag expects two values, the column to match and the actual value. We can use this to find all keywords that include `SOIL TEMPERATURE` on any level. The keywords are \nimplemented self-referential and hold the keyword name in an attribute called `value`.\n\n```bash\nmetacatalog find keywords --by value \"SOIL TEMPERATURE\"\n```\n```\nEARTH SCIENCE > AGRICULTURE > SOILS > SOIL TEMPERATURE\nEARTH SCIENCE > CLIMATE INDICATORS > LAND SURFACE/AGRICULTURE INDICATORS > SOIL TEMPERATURE\nEARTH SCIENCE > CRYOSPHERE > FROZEN GROUND > SOIL TEMPERATURE\nEARTH SCIENCE > LAND SURFACE > FROZEN GROUND > SOIL TEMPERATURE\nEARTH SCIENCE > LAND SURFACE > SOILS > SOIL TEMPERATURE\n```\n\nAny of these keywords might be suitable to append them to your metadata to make your soil temperature data set findable on that keyword.\n\n## Add data\n\nThe CLI and API can also be used to add new data into the database. This does not work for all entity types yet. If the API is used, \nthe objects need to be created as Python dictionaries. The CLI accepts three different data origin flags:\n\n```bash\nmetacatalog add -h\n```\n```\nusage: metacatalog add [-h] [--version] [--connection CONNECTION] [--verbose]\n [--csv CSV] [--txt TXT] [--json JSON]\n entity\n\npositional arguments:\n entity Name of the record entity to be added.\n\noptional arguments:\n -h, --help show this help message and exit\n --version, -v Returns the module version\n --connection CONNECTION, -C CONNECTION\n Connection string to the database instance.Follows the\n syntax: driver://user:password@host:port/database\n --verbose, -V Activate extended output.\n --csv CSV Data Origin Flag. Pass a CSV filename or content\n containing the data. Column header have to match the\n ADD API keywords.\n --txt TXT Data Origin Flag. Pass a text filename or content\n containing whitespace separated key=value pairs where\n key has to match the ADD API keywords. If used\n directly remember to quote accordingly.\n --json JSON Data Origin Flag. Pass a JSON filename or content\n containing the data. Must contain a list of objects\n matchin the ADD API keywords.\n```\n\nTo add a new unit, we can use the `show` cli action to read the available attributes of the `units` table:\n\n```bash\nmetacatalog show attributes --table units --names-only\n```\n```\nAttributes of units\n-------------------\nid\nname\nsymbol\nsi\n```\n\nThe `si` attribute is optional. Let's create a few new units, using a csv syntax. \nFor the `---csv` flag, we can specify a file name or give the data directly:\n\n```bash\nmetacatalog add unit --csv 'name,symbol\\nnfoo,F\\nbar,B'\n```\n```\nAdded 2 unit records.\nDone.\n```", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "GPL v3", "maintainer": "", "maintainer_email": "", "name": "metacatalog", "package_url": "https://pypi.org/project/metacatalog/", "platform": "", "project_url": "https://pypi.org/project/metacatalog/", "project_urls": null, "release_url": "https://pypi.org/project/metacatalog/0.1.2/", "requires_dist": null, "requires_python": "", "summary": "Metadata model management module.", "version": "0.1.2" }, "last_serial": 5819825, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "45e87ea8b7b86c3bb9a3b4988adb94bf", "sha256": "b2dbaec328559510204d7a21ed54d10969bf9dd4356c7f5ad7c0043bcce2f544" }, "downloads": -1, "filename": "metacatalog-0.1.tar.gz", "has_sig": false, "md5_digest": "45e87ea8b7b86c3bb9a3b4988adb94bf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17370, "upload_time": "2019-08-20T18:56:05", "url": "https://files.pythonhosted.org/packages/a3/70/9e1d5f6905cb331c079741df5b4a6ad3419f49d2e1495715940adbabb1fe/metacatalog-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "01cd9ab979cb1b4119de2d7a0ad41225", "sha256": "c7ee85ea3e3902729c7cb08890d9d56549b9f12abf70b4a53583851e2d7bed5d" }, "downloads": -1, "filename": "metacatalog-0.1.1.tar.gz", "has_sig": false, "md5_digest": "01cd9ab979cb1b4119de2d7a0ad41225", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 27415, "upload_time": "2019-09-05T10:39:39", "url": "https://files.pythonhosted.org/packages/b1/d8/6422e64acd61865184cbf2fdf1c0a4709cfbefdba1e1dfe56d993935bf6c/metacatalog-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "ee58d7bc5a63ba48a7904627413b140d", "sha256": "b80d383ecc5a3ef9f1b44658aaa154c51d28e5e3329d260fe85e2995c31f4d01" }, "downloads": -1, "filename": "metacatalog-0.1.2.tar.gz", "has_sig": false, "md5_digest": "ee58d7bc5a63ba48a7904627413b140d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 31918, "upload_time": "2019-09-12T11:03:31", "url": "https://files.pythonhosted.org/packages/0f/ce/d9b143c87b5185b4a1478600345d0edb7c7c151247d25a98aa4cef26d794/metacatalog-0.1.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ee58d7bc5a63ba48a7904627413b140d", "sha256": "b80d383ecc5a3ef9f1b44658aaa154c51d28e5e3329d260fe85e2995c31f4d01" }, "downloads": -1, "filename": "metacatalog-0.1.2.tar.gz", "has_sig": false, "md5_digest": "ee58d7bc5a63ba48a7904627413b140d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 31918, "upload_time": "2019-09-12T11:03:31", "url": "https://files.pythonhosted.org/packages/0f/ce/d9b143c87b5185b4a1478600345d0edb7c7c151247d25a98aa4cef26d794/metacatalog-0.1.2.tar.gz" } ] }