{
    "info": {
        "author": "Felix Lohmeier",
        "author_email": "felix.lohmeier@opencultureconsulting.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Developers",
            "Intended Audience :: System Administrators",
            "License :: OSI Approved :: GNU General Public License (GPL)",
            "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Topic :: Software Development :: Libraries :: Python Modules",
            "Topic :: Text Processing"
        ],
        "description": "# OpenRefine Python Client with extended command line interface\n\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/33129bd15cdc4ece88c8012caab8d347)](https://www.codacy.com/app/felixlohmeier/openrefine-client?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=opencultureconsulting/openrefine-client&amp;utm_campaign=Badge_Grade) [![Docker](https://img.shields.io/microbadger/image-size/felixlohmeier/openrefine-client?label=docker)](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [![PyPI](https://img.shields.io/pypi/v/openrefine-client)](https://pypi.org/project/openrefine-client/) [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master)\n\nThe [OpenRefine Python Client from PaulMakepeace](https://github.com/PaulMakepeace/refine-client-py) provides a library for communicating with an [OpenRefine](http://openrefine.org) server.\nThis fork extends the command line interface (CLI) and is distributed as a convenient one-file-executable (Windows, Linux, macOS).\nIt is also available via Docker Hub, PyPI and Binder.\n\n## Download\n\nOne-file-executables:\n\n- Windows: [openrefine-client_0-3-8_windows.exe](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_windows.exe) (~5 MB)\n- macOS: [openrefine-client_0-3-8_macos](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_macos) (~5 MB)\n- Linux: [openrefine-client_0-3-8_linux](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_linux) (~5 MB)\n\nFor [Docker](#docker) containers, native [Python](#python) installation and free [Binder](#binder) on-demand server see the corresponding chapters below.\n\n## Peek\n\nA short video loop that demonstrates the basic features (list, create, apply, export):\n\n![video loop that demonstrates basic features](openrefine-client-peek.gif)\n\n## Usage\n\nEnsure you have [OpenRefine](http://openrefine.org) running (i.e. available at http://localhost:3333 or [another URL](#change-url)).\n\nTo use the client:\n\n1. Open a terminal pointing to the folder where you have [downloaded](#download) the one-file-executable (e.g. Downloads in your home directory).\n\n   - Windows: Open PowerShell and enter following command\n\n     ```sh\n     cd ~\\Downloads\n     ```\n\n   - macOS: Open Terminal (Finder > Applications > Utilities > Terminal) and enter following command\n\n     ```sh\n     cd ~/Downloads\n     ```\n\n   - Linux: Open terminal app (Terminal, Konsole, xterm, ...) and enter following command\n\n     ```sh\n     cd ~/Downloads\n     ```\n\n2. Make the file executable.\n\n   - Windows: not necessary\n\n   - macOS:\n\n     ```sh\n     chmod +x openrefine-client_0-3-8_macos\n     ```\n\n   - Linux:\n\n     ```sh\n     chmod +x openrefine-client_0-3-8_linux\n     ```\n\n3. Execute the file.\n\n   - Windows:\n\n     ```sh\n     .\\openrefine-client_0-3-8_windows.exe\n     ```\n\n   - macOS:\n\n     ```sh\n     ./openrefine-client_0-3-8_macos\n     ```\n\n   - Linux:\n\n     ```sh\n     ./openrefine-client_0-3-8_linux\n     ```\n\nUsing tab completion and command history is highly recommended:\n\n- autocomplete filenames: enter a few characters and press `\u21b9`\n- recall previous command: press `\u2191`\n\n### Basic commands\n\nExecute the client by entering its filename followed by the desired command.\n\nThe following example will download two small files ([duplicates.csv](https://raw.githubusercontent.com/opencultureconsulting/openrefine-client/master/tests/data/duplicates.csv) and [duplicates-deletion.json](https://raw.githubusercontent.com/opencultureconsulting/openrefine-client/master/tests/data/duplicates-deletion.json)) into the current directory and will create a new OpenRefine project from file duplicates.csv.\n\nDownload example data (`--download`) and create project from file (`--create`):\n\n- Windows:\n\n  ```sh\n  .\\openrefine-client_0-3-8_windows.exe --download \"https://git.io/fj5hF\" --output=duplicates.csv\n  .\\openrefine-client_0-3-8_windows.exe --download \"https://git.io/fj5ju\" --output=duplicates-deletion.json\n  .\\openrefine-client_0-3-8_windows.exe --create duplicates.csv\n  ```\n\n- macOS:\n\n  ```sh\n  ./openrefine-client_0-3-8_macos --download \"https://git.io/fj5hF\" --output=duplicates.csv\n  ./openrefine-client_0-3-8_macos --download \"https://git.io/fj5ju\" --output=duplicates-deletion.json\n  ./openrefine-client_0-3-8_macos --create duplicates.csv\n  ```\n\n- Linux:\n\n  ```sh\n  ./openrefine-client_0-3-8_linux --download \"https://git.io/fj5hF\" --output=duplicates.csv\n  ./openrefine-client_0-3-8_linux --download \"https://git.io/fj5ju\" --output=duplicates-deletion.json\n  ./openrefine-client_0-3-8_linux --create duplicates.csv\n  ```\n\nOther commands:\n\n- list all projects: `--list`\n- show project metadata: `--info \"duplicates\"`\n- export project to terminal: `--export \"duplicates\"`\n- apply [rules from json file](http://kb.refinepro.com/2012/06/google-refine-json-and-my-notepad-or.html): `--apply duplicates-deletion.json \"duplicates\"`\n- export project to file: `--export --output=deduped.xls \"duplicates\"`\n- delete project: `--delete \"duplicates\"`\n\n### Getting help\n\nCheck `--help` for further options.\n\nPlease file an [issue](https://github.com/opencultureconsulting/openrefine-client/issues) if you miss some features in the command line interface or if you have tracked a bug.\nAnd you are welcome to ask any questions!\n\n### Change URL\n\nBy default the client connects to the usual URL of OpenRefine [http://localhost:3333](http://localhost:3333).\nIf your OpenRefine server is running somewhere else then you may set hostname and port with additional command line options (e.g. http://example.com):\n\n- set host: `-H example.com`\n- set port: `-P 80`\n\n### Advanced Templating\n\nThe OpenRefine [Templating](https://github.com/OpenRefine/OpenRefine/wiki/Export-As-YAML) supports exporting data in any text format (i.e. to construct JSON or XML).\nThe graphical user interface offers four input fields:\n\n1. prefix\n2. row template\n   - supports [GREL](https://github.com/OpenRefine/OpenRefine/wiki/General-Refine-Expression-Language) inside two curly brackets, e.g. `{{jsonize(cells[\"name\"].value)}}`\n3. row separator\n4. suffix\n\nThis templating functionality is available via the openrefine-client command line interface.\nIt even provides an additional feature for splitting results into multiple files.\n\nTo try out the functionality create another project from the example file above.\n\n```sh\n--create duplicates.csv --projectName=advanced\n```\n\nThe following example code will export...\n\n- the columns \"name\" and \"purchase\" in JSON format\n- from the project \"advanced\"\n- for rows matching the regex text filter `^F$` in column \"gender\"\n\nmacOS/Linux Terminal (multi-line input with `\\` ):\n\n```sh\n\"advanced\" \\\n--prefix='{ \"events\" : [\n' \\\n--template='    { \"name\" : {{jsonize(cells[\"name\"].value)}}, \"purchase\" : {{jsonize(cells[\"purchase\"].value)}} }' \\\n--rowSeparator=',\n' \\\n--suffix='\n] }' \\\n--filterQuery='^F$' \\\n--filterColumn='gender'\n```\n\nWindows PowerShell (multi-line input with `` ` ``; quotes needs to be doubled):\n\n```sh\n\"advanced\" `\n--prefix='{ \"\"events\"\" : [\n' `\n--template='    { \"\"name\"\" : {{jsonize(cells[\"\"name\"\"].value)}}, \"\"purchase\"\" : {{jsonize(cells[\"\"purchase\"\"].value)}} }' `\n--rowSeparator=',\n' `\n--suffix='\n] }' `\n--filterQuery='^F$' `\n--filterColumn='gender'\n```\n\nAdd the following options to the last command (recall with `\u2191`) to store the results in multiple files.\nEach file will contain the prefix, an processed row, and the suffix.\n\n```sh\n--output=advanced.json --splitToFiles=true\n```\n\nFilenames are suffixed with the row number by default (e.g. `advanced_1.json`, `advanced_2.json` etc.).\nThere is another option to use the value in the first column instead:\n\n```sh\n--output=advanced.json --splitToFiles=true --suffixById=true\n```\n\nBecause our project \"advanced\" contains duplicates in the first column \"email\" this command will overwrite files (e.g. `advanced_melanie.white@example2.edu.json`).\nWhen using this option, the first column should contain unique identifiers.\n\n### See also\n\n- Linux Bash script to run OpenRefine in batch mode (import, transform, export): [openrefine-batch](https://github.com/opencultureconsulting/openrefine-batch)\n- [Jupyter notebook demonstrating usage in Linux Bash](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/openrefine-client-bash.ipynb)\n- Use case [HOS-MetadataTransformations](https://github.com/subhh/HOS-MetadataTransformations): Automated workflow for harvesting, transforming and indexing of metadata using metha, OpenRefine and Solr. Part of the Hamburg Open Science \"Schaufenster\" software stack.\n- Use case [Data processing of ILS data to facilitate a new discovery layer for the German Literature Archive (DLA)](https://doi.org/10.5281/zenodo.2678113): Custom data processing pipeline based on Pandas (a Python library) and OpenRefine.\n\n## Docker\n\n[felixlohmeier/openrefine-client](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [![Docker](https://img.shields.io/microbadger/image-size/felixlohmeier/openrefine-client?label=docker)](https://hub.docker.com/r/felixlohmeier/openrefine-client/)\n\n```sh\ndocker pull felixlohmeier/openrefine-client:v0.3.8\n```\n\n### Option 1: Dockerized client\n\nRun client and mount current directory as workspace:\n\n```sh\ndocker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8\n```\n\nThe docker option `--network=host` allows you to connect to a local or remote OpenRefine via the host network:\n\n- list projects on default URL (http://localhost:3333)\n\n  ```sh\n  docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 --list\n  ```\n\n- list projects on a remote server (http://example.com)\n\n  ```sh\n  docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H example.com -P 80 --list\n  ```\n\nUsage: same commands as explained above (see [Basic Commands](#basic-commands) and [Advanced Templating](#advanced-templating))\n\n### Option 2: Dockerized client and dockerized OpenRefine\n\nRun openrefine-client linked to a dockerized OpenRefine ([felixlohmeier/openrefine](https://hub.docker.com/r/felixlohmeier/openrefine/) [![Docker](https://img.shields.io/microbadger/image-size/felixlohmeier/openrefine?label=docker)](https://hub.docker.com/r/felixlohmeier/openrefine)):\n\n1. Create docker network\n\n   ```sh\n   docker network create openrefine\n   ```\n\n2. Run server (will be available at http://localhost:3333)\n\n   ```sh\n   docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.2\n   ```\n\n3. Run client with some [basic commands](#basic-commands): 1. download example files, 2. create project from file, 3. list projects, 4. show metadata, 5. export to terminal, 6. apply transformation rules (deduplication), 7. export again to terminal, 8. export to xls file and 9. delete project\n\n   ```sh\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 --download \"https://git.io/fj5hF\" --output=duplicates.csv\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 --download \"https://git.io/fj5ju\" --output=duplicates-deletion.json\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --create duplicates.csv\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --list\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --info \"duplicates\"\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --export \"duplicates\"\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --apply duplicates-deletion.json \"duplicates\"\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --export \"duplicates\"\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --export --output=deduped.xls \"duplicates\"\n   docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --delete \"duplicates\"\n   ```\n\n4. Stop and delete server:\n\n   ```sh\n   docker stop openrefine-server\n   docker rm openrefine-server\n   ```\n\n5. Delete docker network:\n\n   ```sh\n   docker network rm openrefine\n   ```\n\nCustomize OpenRefine server:\n\n- If you want to add an OpenRefine startup option you need to repeat the default commands (cf. [Dockerfile](https://hub.docker.com/r/felixlohmeier/openrefine/dockerfile))\n  - `-i 0.0.0.0` sets OpenRefine to be accessible from outside the container, i.e. from host\n  - `-d /data` sets OpenRefine workspace\n\n- Example for [allocating more memory](https://github.com/OpenRefine/OpenRefine/wiki/FAQ#out-of-memory-errors---feels-slow---could-not-reserve-enough-space-for-object-heap) to OpenRefine with additional option `-m 4G`\n\n  ```sh\n  docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.2 -i 0.0.0.0 -d /data -m 4G\n  ```\n\n- The OpenRefine version is defined by the docker tag.\n  Check the [DockerHub repository](https://hub.docker.com/r/felixlohmeier/openrefine) for available tags.\n  Example for OpenRefine `2.8` with same options as above:\n\n  ```sh\n  docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:2.8 -i 0.0.0.0 -d /data -m 4G\n  ```\n\n- If you want OpenRefine to read and write persistent data in host directory (i.e. store projects) you can mount the container path `/data`. Example for host directory `/home/felix/refine`:\n\n  ```sh\n  docker run -d -p 3333:3333 -v /home/felix/refine:/data:z --network=openrefine name=openrefine-server felixlohmeier/openrefine:2.8 -i 0.0.0.0 -d /data -m 4G\n  ```\n\nSee also:\n\n- [GitHub Repository](https://github.com/opencultureconsulting/openrefine-docker) for docker container `felixlohmeier/openrefine`\n- Linux Bash script to run OpenRefine in batch mode (import, transform, export) with docker containers: [openrefine-batch-docker.sh](https://github.com/opencultureconsulting/openrefine-batch/#docker)\n\n## Python\n\n[openrefine-client](https://pypi.org/project/openrefine-client/) [![PyPI](https://img.shields.io/pypi/v/openrefine-client)](https://pypi.org/project/openrefine-client/) (requires Python 2.x)\n\n```sh\npython2 -m pip install openrefine-client --user\n```\n\nThis will install the package `openrefine-client` containing modules in `google.refine`.\n\nA command line script `openrefine-client` will also be installed.\n\n### Option 1: command line script\n\n```sh\nopenrefine-client --help\n```\n\nUsage: same commands as explained above (see [Basic Commands](#basic-commands) and [Advanced Templating](#advanced-templating))\n\n### Option 2: using cli functions in Python 2.x environment\n\nImport module cli:\n\n```python\nfrom google.refine import cli\n```\n\nChange URL (if necessary):\n\n```python\ncli.refine.REFINE_HOST = 'localhost'\ncli.refine.REFINE_PORT = '3333'\n```\n\nHelp screen:\n\n```python\nhelp(cli)\n```\n\nCommands:\n\n* download (e.g. example data):\n\n  ```python\n  cli.download('https://git.io/fj5hF','duplicates.csv')\n  cli.download('https://git.io/fj5ju','duplicates-deletion.json')\n  ```\n\n* list projects:\n\n  ```python\n  cli.ls()\n  ```\n\n* create project:\n\n  ```python\n  p1 = cli.create('duplicates.csv')\n  ```\n\n* show metadata:\n\n  ```python\n  cli.info(p1.project_id)\n  ```\n\n* apply rules from file to project:\n\n  ```python\n  cli.apply(p1.project_id, 'duplicates-deletion.json')\n  ```\n\n* export project to terminal:\n\n  ```python\n  cli.export(p1.project_id)\n  ```\n\n* export project to file in xls format:\n\n  ```python\n  cli.export(p1.project_id, 'deduped.xls')\n  ```\n\n* export templating (see [Advanced Templating](#advanced-templating) above):\n\n  ```python\n  cli.templating(\n      p1.project_id,\n      prefix='''{ \"events\" : [\n  ''',template='''    { \"name\" : {{jsonize(cells[\"name\"].value)}}, \"purchase\" : {{jsonize(cells[\"purchase\"].value)}} }''',\n      rowSeparator=''',\n  ''',suffix='''\n  ] }''')\n  ```\n\n* delete project:\n\n  ```python\n  cli.delete(p1.project_id)\n  ```\n\n### Option 3: the upstream way\n\nThis fork can be used in the same way as the upstream [Python client library](https://github.com/PaulMakepeace/refine-client-py/).\n\nSome functions in the python client library are not yet compatible with OpenRefine >=3.0 (cf. [issue #19 in refine-client-py](https://github.com/paulmakepeace/refine-client-py/issues/19)).\n\nImport module refine:\n\n```python\nfrom google.refine import refine\n```\n\nServer Commands:\n\n* set up connection:\n\n  ```python\n  server1 = refine.Refine('http://localhost:3333')\n  ```\n\n- show version:\n\n  ```python\n  server1.server.get_version()\n  server1.server.version\n  ```\n\n- list projects:\n\n  ```python\n  server1.list_projects()\n  ```\n\n  - pretty print the returned dict with json.dumps:\n\n    ```python\n    import json\n    print(json.dumps(server1.list_projects(), indent=1))\n    ```\n\n- create project:\n\n  ```python\n  server1.new_project(project_file='duplicates.csv')\n  ```\n\n  * create and open the returned project in one step:\n\n    ```python\n    project1 = server1.new_project(project_file='duplicates.csv')\n    ```\n\nProject commands:\n\n* open project:\n\n  ```python\n  project1 = server1.open_project('1234567890123')\n  ```\n\n* print full URL to project:\n\n  ```python\n  project1.project_url()\n  ```\n\n* list columns:\n\n  ```python\n  project1.columns\n  ```\n\n* compute text facet on first column (**fails with OpenRefine >=3.2**):\n\n  ```python\n  project1.compute_facets(facet.TextFacet(project1.columns[0]))\n  ```\n\n  * print returned object\n\n    ```python\n    facets = project1.compute_facets(facet.TextFacet(project1.columns[0])).facets[0]\n    for k in sorted(facets.choices, key=lambda k: facets.choices[k].count, reverse=True):\n        print(facets.choices[k].count, k)\n    ```\n\n* compute clusters on first column:\n\n  ```python\n  project1.compute_clusters(project1.columns[0])\n  ```\n\n* apply rules from file to project:\n\n  ```python\n  project1.apply_operations('duplicates-deletion.json')\n  ```\n\n* export project:\n\n  ```python\n  project1.export(export_format='tsv')\n  ```\n\n  * print the returned fileobject:\n\n    ```python\n    print(project1.export(export_format='tsv').read())\n    ```\n\n  * save the returned fileobject to file:\n\n    ```python\n    with open('export.tsv', 'wb') as f:\n        f.write(project1.export(export_format='tsv').read())\n    ```\n\n* templating export (**function was added in this fork**, see [Advanced Templating](#advanced-templating) above):\n\n  ```python\n  data = project1.export_templating(\n      prefix='''{ \"events\" : [\n  ''',template='''    { \"name\" : {{jsonize(cells[\"name\"].value)}}, \"purchase\" : {{jsonize(cells[\"purchase\"].value)}} }''',\n      rowSeparator=''',\n  ''',suffix='''\n  ] }''')\n  print(data.read())\n  ```\n\n* print help screen with available commands (many more!):\n\n  ```python\n  help(project1)\n  ```\n\n* example for custom commands:\n\n  ```python\n  project1.do_json('get-rows')['total']\n  ```\n\n* delete project:\n\n  ```python\n  project1.delete()\n  ```\n\nSee also:\n\n- Jupyter notebook by Trevor Mu\u00f1oz (2013-08-18): [Programmatic Use of Open Refine to Facet and Cluster Names of 'Dishes' from NYPL's What's on the menu?](https://nbviewer.jupyter.org/gist/trevormunoz/6265360)\n- Jupyter notebook by Tony Hirst (2019-01-09) [Notebook demonstrating how to control OpenRefine via a Python client.](https://nbviewer.jupyter.org/github/ouseful-PR/openrefineder/blob/4cef25a4ca6077536c5f49cafb531499fbcad96e/notebooks/OpenRefine%20Demos.ipynb)\n- Unittests [test_refine.py](tests/test_refine.py) and [test_tutorial.py](tests/test_tutorial.py) (both importing [refinetest.py](tests/refinetest.py))\n- [OpenRefine API](https://github.com/OpenRefine/OpenRefine/wiki/OpenRefine-API) in official OpenRefine wiki\n\n## Binder\n\n[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master)\n\n- free to use on-demand server with Jupyter notebook, OpenRefine and Bash\n- no registration needed, will start within a few minutes\n- [restricted](https://mybinder.readthedocs.io/en/latest/faq.html#how-much-memory-am-i-given-when-using-binder) to 2 GB RAM and server will be deleted after 10 minutes of inactivity\n- [bash_kernel demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/openrefine-client-bash.ipynb) for using the openrefine-client in a Linux Bash environment [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/openrefine-client-bash.ipynb)\n- [python2 demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/openrefine-client-python.ipynb) for using the openrefine-client in a Python 2 environment [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/openrefine-client-python.ipynb)\n\n## Development\n\nIf you would like to contribute to the Python client library please consider a pull request to the upstream repository [refine-client-py](https://github.com/PaulMakepeace/refine-client-py/).\n\n### Tests\n\nEnsure you have OpenRefine running (i.e. available at http://localhost:3333). If necessary set the environment variables `OPENREFINE_HOST` and `OPENREFINE_PORT` to change the URL.\n\nThe Python client library includes several unit tests.\n\n- run all tests\n\n  ```sh\n  python setup.py test\n  ```\n\n- run subset test_facet\n\n  ```sh\n  python setup.py --test-suite tests.test_facet\n  ```\n\nThere is also a script that uses docker images to run the unit tests with different versions of OpenRefine.\n\n- run tests on all OpenRefine versions (from 2.0 up to 3.2)\n\n  ```sh\n  ./tests.sh -a\n  ```\n\n- run tests on tag 3.2\n\n  ```sh\n  ./tests.sh -t 3.2\n  ```\n\n- run tests on tag 3.2 interactively (pause before and after tests)\n\n  ```sh\n  ./tests.sh -t 3.2 -i\n  ```\n\n- run tests on tags 3.2 and 2.7\n\n  ```sh\n  ./tests.sh -t 3.2 -t 2.7\n  ```\n\n### Distributing\n\nNote to myself: When releasing a new version...\n\n1. Run tests\n\n   ```sh\n   ./tests.sh -a\n   jupyter notebook tests/cli_python2.ipynb\n   ```\n\n2. Make final changes in Git\n\n   - update versions (e.g. 0.3.7 und 0-3-7) in [README.md](https://github.com/opencultureconsulting/openrefine-client/blob/master/README.md#download)\n   - update version in [setup.py](https://github.com/opencultureconsulting/openrefine-client/blob/master/setup.py)\n   - check if [Dockerfile](https://github.com/opencultureconsulting/openrefine-client/blob/master/docker/Dockerfile) needs to be changed\n\n3. Build executables with PyInstaller\n\n   - Run PyInstaller in Python 2 environments on native Windows, macOS and Linux. Should be \"the oldest version of the OS you need to support\"! Current release is built with: \n\n     - Ubuntu 16.04 LTS (64-bit)\n     - macOS Sierra 10.12\n     - Windows 10\n\n   - One-file-executables will be available in `dist/`.\n\n     ```sh\n     git clone https://github.com/opencultureconsulting/openrefine-client.git\n     cd openrefine-client\n     python -m pip install . --user\n     python -m pip install pyinstaller --user\n     pyinstaller --onefile refine.py --hidden-import google.refine.__main__\n     ```\n\n4. Run test with Linux executable\n\n   ```sh\n   ./tests.sh -a\n   jupyter notebook tests/cli_bash.ipynb\n   ```\n\n5. Create release in GitHub\n\n   - draft [release notes](https://github.com/opencultureconsulting/openrefine-client/releases) and attach one-file-executables\n\n6. Build package and upload to PyPI\n\n   ```sh\n   python3 setup.py sdist bdist_wheel\n   python3 -m twine upload dist/*\n   ```\n\n7. Update Docker container\n\n   - add new autobuild for release version\n   - trigger latest build\n\n8. Bump openrefine-client version in related projects\n\n   - openrefine-batch: [openrefine-batch.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch.sh#L7) and [openrefine-batch-docker.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch-docker.sh)\n\n   - openrefineder: [postBuild](https://github.com/felixlohmeier/openrefineder/blob/master/postBuild)\n\n## Credits\n\n[Paul Makepeace](http://paulm.com), author\n\nDavid Huynh, [initial cut](<http://markmail.org/message/jsxzlcu3gn6drtb7)\n\n[Artfinder](http://www.artfinder.com), inspiration\n\n[Felix Lohmeier](https://felixlohmeier.de), extended the CLI features\n\nSome data used in the test suite has been used from publicly available sources:\n\n- louisiana-elected-officials.csv: from http://www.sos.louisiana.gov/tabid/136/Default.aspx\n\n- us_economic_assistance.csv: [\"The Green Book\"](http://www.data.gov/raw/1554)\n\n- eli-lilly.csv: [ProPublica's \"Docs for Dollars](http://projects.propublica.org/docdollars) leading to a [Lilly Faculty PDF](http://www.lillyfacultyregistry.com/documents/EliLillyFacultyRegistryQ22010.pdf) processed by [David Huynh's ScraperWiki script](http://scraperwiki.com/scrapers/eli-lilly-dollars-for-docs-scraper/edit/)\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/opencultureconsulting/openrefine-client",
        "keywords": "openrefine client batch processing docker etl code4lib",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "openrefine-client",
        "package_url": "https://pypi.org/project/openrefine-client/",
        "platform": "Any",
        "project_url": "https://pypi.org/project/openrefine-client/",
        "project_urls": {
            "Homepage": "https://github.com/opencultureconsulting/openrefine-client"
        },
        "release_url": "https://pypi.org/project/openrefine-client/0.3.8/",
        "requires_dist": [
            "urllib2-file"
        ],
        "requires_python": ">=2.7, !=3.*",
        "summary": "The OpenRefine Python Client Library provides an interface to communicating with an OpenRefine server. This fork extends the command line interface (CLI).",
        "version": "0.3.8"
    },
    "last_serial": 5712641,
    "releases": {
        "0.3.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6f2b57796d2c460440ec9e9fc43d8012",
                    "sha256": "8795c17faebb01eefd48c884819b6e2d5eb5a24b3db809332ee306586618bd70"
                },
                "downloads": -1,
                "filename": "openrefine_client-0.3.7-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6f2b57796d2c460440ec9e9fc43d8012",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=2.7, !=3.*",
                "size": 39377,
                "upload_time": "2019-08-16T12:21:45",
                "url": "https://files.pythonhosted.org/packages/7c/3b/de2ce9a9226dbfed54be276e03c0a03387cce4254570b32bcbfd44093a13/openrefine_client-0.3.7-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "54ccbde824d904556adf1b9142c3fce6",
                    "sha256": "a6eda527b34062d1127ad423ca45742c7d535f3b46f5c1f220092680fb0ddcc1"
                },
                "downloads": -1,
                "filename": "openrefine-client-0.3.7.tar.gz",
                "has_sig": false,
                "md5_digest": "54ccbde824d904556adf1b9142c3fce6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=2.7, !=3.*",
                "size": 570341,
                "upload_time": "2019-08-16T12:21:47",
                "url": "https://files.pythonhosted.org/packages/74/6c/3b3c3a024ba0fa0d0330e99894a037c95bc91f140b18742a4baeb0421ad8/openrefine-client-0.3.7.tar.gz"
            }
        ],
        "0.3.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dd640f118b09f698f559ba0dfbf22338",
                    "sha256": "a405230b400f1875ef9bbfa2f9308ac35fec7ddf4ecaafc445a912739060861c"
                },
                "downloads": -1,
                "filename": "openrefine_client-0.3.8-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "dd640f118b09f698f559ba0dfbf22338",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=2.7, !=3.*",
                "size": 40288,
                "upload_time": "2019-08-22T01:45:15",
                "url": "https://files.pythonhosted.org/packages/8c/58/8141613e02e244878971e8d6e4043fd21aaad939fd023b196ff5d5e4afaf/openrefine_client-0.3.8-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "43c613987165b724a40312f8655a5417",
                    "sha256": "a7902eb86582b7313f13be167da4c6709c4cc8695f1f46610732225b0d288048"
                },
                "downloads": -1,
                "filename": "openrefine-client-0.3.8.tar.gz",
                "has_sig": false,
                "md5_digest": "43c613987165b724a40312f8655a5417",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=2.7, !=3.*",
                "size": 571741,
                "upload_time": "2019-08-22T01:45:18",
                "url": "https://files.pythonhosted.org/packages/d5/98/c7a9cf2e4fd256c6567e8b2ae8a2ad793a77574e61047e252ba4295c5702/openrefine-client-0.3.8.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "dd640f118b09f698f559ba0dfbf22338",
                "sha256": "a405230b400f1875ef9bbfa2f9308ac35fec7ddf4ecaafc445a912739060861c"
            },
            "downloads": -1,
            "filename": "openrefine_client-0.3.8-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "dd640f118b09f698f559ba0dfbf22338",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=2.7, !=3.*",
            "size": 40288,
            "upload_time": "2019-08-22T01:45:15",
            "url": "https://files.pythonhosted.org/packages/8c/58/8141613e02e244878971e8d6e4043fd21aaad939fd023b196ff5d5e4afaf/openrefine_client-0.3.8-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "43c613987165b724a40312f8655a5417",
                "sha256": "a7902eb86582b7313f13be167da4c6709c4cc8695f1f46610732225b0d288048"
            },
            "downloads": -1,
            "filename": "openrefine-client-0.3.8.tar.gz",
            "has_sig": false,
            "md5_digest": "43c613987165b724a40312f8655a5417",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=2.7, !=3.*",
            "size": 571741,
            "upload_time": "2019-08-22T01:45:18",
            "url": "https://files.pythonhosted.org/packages/d5/98/c7a9cf2e4fd256c6567e8b2ae8a2ad793a77574e61047e252ba4295c5702/openrefine-client-0.3.8.tar.gz"
        }
    ]
}