{
    "info": {
        "author": "",
        "author_email": "",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "# intake-dcat\n\n[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/CityOfLosAngeles/intake-dcat/master?urlpath=lab%2Ftree%2Fexamples%2Fdemo.ipynb)\n\nThis is an [intake](https://intake.readthedocs.io/en/latest)\ndata source for [DCAT](https://www.w3.org/TR/vocab-dcat) catalogs.\n\nThese catalogs are a standardized format for describing metadata and access information\nfor public datasets, as described [here](https://project-open-data.cio.gov/v1.1/schema).\nMany Socrata and ESRI data portals publish `data.json` files in this format describing their catalogs.\nTwo examples of thes can be found at\n\nhttps://data.lacity.org/data.json\n\nhttp://geohub.lacity.org/data.json\n\nThis project provides an opinionated way for users to load datasets from these catalogs into the scientific Python ecosystem.\nAt the moment it loads CSVs into Pandas dataframes and GeoJSON files into GeoDataFrames, and ESRI Shapefiles into GeoDataFrames.\nFuture formats could include plain JSON and Parquet.\n\n## Requirements\n```\nintake >= 0.4.4\nintake_geopandas >= 0.2.2\ngeopandas >= 0.5.0\n```\n## Installation\n\n`intake-dcat` is published on PyPI.\nYou can install it by running the following in your terminal:\n```bash\npip install intake-dcat\n```\n\nYou can test the functionality by opening the example notebooks in the `examples/` directory\n\n### Usage\n\nThe package can be imported using\n```python\nfrom intake_dcat import DCATCatalog\n```\n\n### Loading a catalog\n\nYou can load data from a DCAT catalog by providing the URL to the `data.json` file:\n```python\ncatalog = DCATCatalog('http://geohub.lacity.org/data.json', name='geohub')\nlen(list(catalog))\n```\n\nYou can display the items in the catalog\n```python\nfor entry_id, entry in catalog.items():\n    display(entry)\n```\n\nIf the catalog has too many entries to comfortably print all at once,\nyou can narrow it by searching for a term (e.g. 'district'):\n```python\nfor entry_id, entry in catalog.search('district').items():\n  display(entry)\n```\n\n### Loading a dataset\nOnce you have identified a dataset, you can load it into a dataframe using `read()`:\n\n```python\ndf = entry.read()\n```\n\nThis will automatically load that dataset into a Pandas dataframe, or a GeoDataFrame, depending on the source format.\n\n### Specifying catalogs\n\nYou can read a `DCATCatalog` directly in Python using a URL, as done above,\nbut it is also possible to write a catalog file that itself contains `DCATCatalog` entries.\nThis allows you to more easily specify DCAT catalogs for use in distribution and version control.\n\nFor instance, this YAML file creates entries for two open data catalogs:\n\n```yaml\nmetadata:\n  version: 1\nsources:\n  # Here we have two data sources for this catalog, which are themselves\n  # DCAT catalogs, one for LA open data, and the other for LA GeoHub\n  la_open_data:\n    # We identify them as being loaded with the DCAT driver\n    driver: dcat\n    # Here we specify the args used to load the catalog\n    args:\n      # The URL to the catalog\n      url: https://data.lacity.org/data.json\n      # An optional name for the catalog.\n      name: la-open-data\n  la_geohub:\n    driver: dcat\n    args:\n      url: http://geohub.lacity.org/data.json\n      name: la_geohub\n      # We can also specify a subset of the datasets in the catalog using an \"items\"\n      # dictionary. If these are specified, only these datasets will be available in\n      # the resulting catalog. They will be available under the more human-readable\n      # name specified as the key.\n      items:\n        # So, this dataset will be available as \"bikeways\"\n        bikeways: http://geohub.lacity.org/datasets/2602345a7a8549518e8e3c873368c1d9_0\n        city_boundary: http://geohub.lacity.org/datasets/09f503229d37414a8e67a7b6ceb9ec43_7\n        bike_racks: http://geohub.lacity.org/datasets/3b022cced9704108af157d3d5eedb268_2\n```\n\n### Command Line Interface\n\n`intake-dcat` provides a small command line interface for some common operations.\nThese are invoked using `intake-dcat <subcommand> <options>`\n\n#### The `mirror` command\n\nThis command loads a manifest file that lists a set of DCAT entries,\nuploads them to a specified s3 bucket, and outputs a new catalog with identical entries\npointing to the bucket.\n\nAn example manifest is given by\n```yml\n# Name of the LA open data portal\nla-open-data:\n  # URL to the open data portal catalog\n  url: https://data.lacity.org/data.json\n  # The s3 bucket to upload the data to\n  bucket_uri: s3://my-bucket\n  # A list of data resources to mirror\n  items:\n    lapd_metrics: https://data.lacity.org/api/views/t6kt-2yic\n# Name of the LA GeoHub data portal\nla-geohub:\n  # URL to the open data portal catalog\n  url: http://geohub.lacity.org/data.json\n  # The s3 bucket to upload the data to\n  bucket_uri: s3://my-bucket\n  # A list of data resources to mirror\n  items:\n    bikeways: http://geohub.lacity.org/datasets/2602345a7a8549518e8e3c873368c1d9_0 \n    city_boundary: http://geohub.lacity.org/datasets/09f503229d37414a8e67a7b6ceb9ec43_7\n```\n\nThis can be mirrored using the command\n\n```bash\nintake-dcat mirror manifest.yml > new-catalog.yml\n```\n\nThis command uses the `boto3` library and assumes it can find AWS credentials.\nFor more information see [this documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html).\n\n#### The `create` command\n\nThis command creates a new intake catalog from a DCAT catalog, and outputs it to standard out.\nAn example command is given by\n\n```bash\nintake-dcat create data.lacity.org/data.json > catalog.yml\n```\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/CityOfLosAngeles/intake-dcat",
        "keywords": "intake dcat",
        "license": "Apache-2.0 license",
        "maintainer": "Ian Rose",
        "maintainer_email": "ian.rose@lacity.org",
        "name": "intake-dcat",
        "package_url": "https://pypi.org/project/intake-dcat/",
        "platform": "",
        "project_url": "https://pypi.org/project/intake-dcat/",
        "project_urls": {
            "Homepage": "https://github.com/CityOfLosAngeles/intake-dcat"
        },
        "release_url": "https://pypi.org/project/intake-dcat/0.3.1/",
        "requires_dist": [
            "geopandas (>=0.5)",
            "intake (>=0.5)",
            "intake-geopandas (>=0.2.2)",
            "pyyaml",
            "requests",
            "s3fs"
        ],
        "requires_python": "",
        "summary": "DCAT to Intake Catalog translation layer",
        "version": "0.3.1"
    },
    "last_serial": 5961797,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "007cf3c39b7c39260de30771b280dfe9",
                    "sha256": "f5d1d491b4ff190ed8eff4a3c993a2660a7f1d3c172209101c60144f12cc9a02"
                },
                "downloads": -1,
                "filename": "intake_dcat-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "007cf3c39b7c39260de30771b280dfe9",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12254,
                "upload_time": "2019-04-25T16:19:27",
                "url": "https://files.pythonhosted.org/packages/17/8c/ba1db2f88c9e5b5d426278dddf11ede22e5207400dadb7b32ef8ca3f44e4/intake_dcat-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "769413ca15719fa6771a96f2e6d9fc1e",
                    "sha256": "cd68807eef5327dba1b1c3c3de8a916e223155692b70417a5585e0879509cc01"
                },
                "downloads": -1,
                "filename": "intake-dcat-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "769413ca15719fa6771a96f2e6d9fc1e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10887,
                "upload_time": "2019-04-25T16:19:36",
                "url": "https://files.pythonhosted.org/packages/ed/63/e47a15da4c88d54eda1e16d8c956f21f98461784801c53cfa1b5685567a9/intake-dcat-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "55c5a40361591b96ff23d81aa16e32da",
                    "sha256": "050c7d9f7b468701baf83de4acd1fedf5a3a9e100d561e565a7c52a16bdfcb74"
                },
                "downloads": -1,
                "filename": "intake_dcat-0.2.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "55c5a40361591b96ff23d81aa16e32da",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12476,
                "upload_time": "2019-04-30T20:08:40",
                "url": "https://files.pythonhosted.org/packages/a8/c1/bcce606bc501bc2ece7f37ddc0b4396031a489e10f0b6d87b2e97aa5f102/intake_dcat-0.2.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8b42160adf3ad106c3435bf3c0ce3879",
                    "sha256": "8b7840bc59308a40b7c3e10ed1f97d5cb7b6d5edbcbb6967fe996d935e4b6a96"
                },
                "downloads": -1,
                "filename": "intake-dcat-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8b42160adf3ad106c3435bf3c0ce3879",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10960,
                "upload_time": "2019-04-30T20:08:43",
                "url": "https://files.pythonhosted.org/packages/5b/95/9d95d7bf4a282b95cfe6c14f433e25996307d30bf41c64780bd516bc9fe6/intake-dcat-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f836a061c741a963ebf827a178fcfc96",
                    "sha256": "d89dc8cba7ccd9c44fb4ba202f2faf41790b92094b8f857bb31b39c4b72fbdf2"
                },
                "downloads": -1,
                "filename": "intake_dcat-0.2.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f836a061c741a963ebf827a178fcfc96",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12473,
                "upload_time": "2019-06-19T21:48:35",
                "url": "https://files.pythonhosted.org/packages/19/63/b847892adf685365062700cf34c2f31553839ec3ff63fe8a01b920b36ebe/intake_dcat-0.2.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7aabee72faae14f33e68bd2eb9fb9e66",
                    "sha256": "84a78b5ae11a94d990644f813b694a143bd014a2ccabe069e30e7e48f43cbc1e"
                },
                "downloads": -1,
                "filename": "intake-dcat-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "7aabee72faae14f33e68bd2eb9fb9e66",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11106,
                "upload_time": "2019-06-19T21:48:37",
                "url": "https://files.pythonhosted.org/packages/7e/23/0b3d8a08e9532f42161475204001f46b982ae4041804d0a6717045271db4/intake-dcat-0.2.1.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7a658ab6cae08060c709c59c1b04a600",
                    "sha256": "1f23896a4c8849390facfeb6cf89b61a1140546e2733f7b5860a575d6379da88"
                },
                "downloads": -1,
                "filename": "intake_dcat-0.2.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7a658ab6cae08060c709c59c1b04a600",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12523,
                "upload_time": "2019-07-16T18:31:38",
                "url": "https://files.pythonhosted.org/packages/c7/c1/6bb65bbd91836cb19ff2fff0c7d69f251beaf8b43b0f552138a1cf6ba78b/intake_dcat-0.2.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "148151d636816616a2ceff555e6188ff",
                    "sha256": "1cad6b5bad787e6ed296346c06f802097c99f30f5ab3d739b08dc20bd1950bf7"
                },
                "downloads": -1,
                "filename": "intake-dcat-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "148151d636816616a2ceff555e6188ff",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11147,
                "upload_time": "2019-07-16T18:31:39",
                "url": "https://files.pythonhosted.org/packages/c9/ec/cb340bc0483a6821b8205f0301de6e4d590b3540cc677051f5cced27da40/intake-dcat-0.2.2.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9f9243a3f3c9f64b12d46482cca1016a",
                    "sha256": "9fbe229b5e33228107b2049d725884fecce5c55913179bb19899afe5ac7bea3a"
                },
                "downloads": -1,
                "filename": "intake_dcat-0.2.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9f9243a3f3c9f64b12d46482cca1016a",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 13174,
                "upload_time": "2019-07-17T20:57:31",
                "url": "https://files.pythonhosted.org/packages/a9/a7/2ac91601a3822ff3559f282de16d66c9a043c7d0eb3a1a3aaabf7a37e35c/intake_dcat-0.2.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ac847074cee6205a49ce17cdb07d28a8",
                    "sha256": "386be34f0357cdaae5a24bfd0628720397118a56036b5bde1a7ee4520f50934f"
                },
                "downloads": -1,
                "filename": "intake-dcat-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "ac847074cee6205a49ce17cdb07d28a8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11702,
                "upload_time": "2019-07-17T20:57:33",
                "url": "https://files.pythonhosted.org/packages/aa/0c/694a5ec18866617735c94dad7a96eaddd14d4ad04d37eb54e9ee4db1966a/intake-dcat-0.2.3.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "522b75b91e735c1a680efd3bb9828f6d",
                    "sha256": "d7c4adf5a7e53ca44b737a4ae4de23b60ac653d0019a35c14652c4ffc6d14f75"
                },
                "downloads": -1,
                "filename": "intake_dcat-0.3.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "522b75b91e735c1a680efd3bb9828f6d",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 13738,
                "upload_time": "2019-09-04T16:26:01",
                "url": "https://files.pythonhosted.org/packages/c2/a9/94ce6dd6a8602d046f50be4fd0e95868a78b5db9f0bfbd21504386e6a778/intake_dcat-0.3.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ff17c2aff9ea9d9a6a22da4678fe446e",
                    "sha256": "adf5a7ffce12aa7a3241d6038c0b44b3f1f77ad7b06ea10ffe161a8cf6b73f4b"
                },
                "downloads": -1,
                "filename": "intake-dcat-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ff17c2aff9ea9d9a6a22da4678fe446e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12891,
                "upload_time": "2019-09-04T16:26:02",
                "url": "https://files.pythonhosted.org/packages/c8/15/b14b31a39008797ccce36cce22f7e9a87f4ca1255569822b6907572033d9/intake-dcat-0.3.0.tar.gz"
            }
        ],
        "0.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1458202787a9bb2d7a1bfdd8c5dede83",
                    "sha256": "c4a1431c3c46ad0fee3384da1548451c39b83dec119f9dcdcb5cab4ea67b1f14"
                },
                "downloads": -1,
                "filename": "intake_dcat-0.3.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1458202787a9bb2d7a1bfdd8c5dede83",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 13767,
                "upload_time": "2019-10-11T19:45:18",
                "url": "https://files.pythonhosted.org/packages/d5/e4/f4046fa5add14da03761d647a725d30c5e30c31c3f142bcdcf1367fdf717/intake_dcat-0.3.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3873495dc92c9ab534927cd0b4526d2f",
                    "sha256": "28b02ddfb4d736f80dab99848cc2b4b3e2f1ef0c008ab1c7cd2c3b0efb2fa90b"
                },
                "downloads": -1,
                "filename": "intake-dcat-0.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "3873495dc92c9ab534927cd0b4526d2f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12347,
                "upload_time": "2019-10-11T19:45:20",
                "url": "https://files.pythonhosted.org/packages/65/be/7f99d1b65d805196444b5f348dc8c2984e70c1badb6e26411be84366ef1c/intake-dcat-0.3.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "1458202787a9bb2d7a1bfdd8c5dede83",
                "sha256": "c4a1431c3c46ad0fee3384da1548451c39b83dec119f9dcdcb5cab4ea67b1f14"
            },
            "downloads": -1,
            "filename": "intake_dcat-0.3.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "1458202787a9bb2d7a1bfdd8c5dede83",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 13767,
            "upload_time": "2019-10-11T19:45:18",
            "url": "https://files.pythonhosted.org/packages/d5/e4/f4046fa5add14da03761d647a725d30c5e30c31c3f142bcdcf1367fdf717/intake_dcat-0.3.1-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "3873495dc92c9ab534927cd0b4526d2f",
                "sha256": "28b02ddfb4d736f80dab99848cc2b4b3e2f1ef0c008ab1c7cd2c3b0efb2fa90b"
            },
            "downloads": -1,
            "filename": "intake-dcat-0.3.1.tar.gz",
            "has_sig": false,
            "md5_digest": "3873495dc92c9ab534927cd0b4526d2f",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 12347,
            "upload_time": "2019-10-11T19:45:20",
            "url": "https://files.pythonhosted.org/packages/65/be/7f99d1b65d805196444b5f348dc8c2984e70c1badb6e26411be84366ef1c/intake-dcat-0.3.1.tar.gz"
        }
    ]
}