{ "info": { "author": "Francesco de Virgilio, Luca Foppiano", "author_email": "francesco.devirgilio@ubiquitypress.com, luca.foppiano@inria.fr", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.6" ], "description": "HIRMEOS Entity Fishing client\n=============================\n\n.. image:: http://img.shields.io/:license-apache-blue.svg\n :target: http://www.apache.org/licenses/LICENSE-2.0.html\n\n.. image:: https://travis-ci.org/hirmeos/entity-fishing-client-python.svg?branch=master\n :target: https://travis-ci.org/hirmeos/entity-fishing-client-python\n\n\nPython client to query the `Entity Fishing service API`_ developed in the context of the EU H2020 HIRMEOS project (WP3).\nFor more information about entity-fishing, please check the `Entity Fishing Documentation`_. \n\n.. _Entity Fishing service API: http://github.com/kermitt2/nerd\n.. _Entity Fishing Documentation: http://nerd.readthedocs.io\n\n\nInstallation\n------------\n\nThe client can be installed using `pip`:\n\n pip install entity-fishing-client\n\nUsage\n-----\n\nDisambiguation\n##############\n\n.. code-block:: python\n\n from nerd import nerd_client\n client = nerd_client.NerdClient()\n\n\nTo disambiguate text (> 5 words):\n\n.. code-block:: python\n\n client.disambiguate_text(\n \"Linux is a name that broadly denotes a family of free and open-source software operating systems (OS) built around the Linux kernel.\"\n )\n\nTo disambiguate a search query\n\n.. code-block:: python\n\n client.disambiguate_query(\n \"python method acronym concrete\"\n )\n\n\nTo process a PDF:\n\n.. code-block:: python\n\n client.disambiguate_pdf(pdfFile, language)\n\n\nyou can supply the language (iso form of two digits, en, fr, etc..) and the entities (only for text) you already know,\nusing the format:\n\n.. code-block:: python\n\n {\n 'offsetEnd': 5,\n 'offsetStart': 0,\n 'rawName': 'Linux',\n 'wikidataId': 'Q388',\n 'wikipediaExternalRef': 6097297\n }\n\n\nThe response is a tuple where the first element is the response body as a dictionary and the second element the error code.\nHere an example: \n\n.. code-block:: python\n\n (\n {'entities': [\n {\n 'domains': ['Computer_Science'],\n 'nerd_score': 0.3753,\n 'nerd_selection_score': 0.7268,\n 'offsetEnd': 5,\n 'offsetStart': 0,\n 'rawName': 'Linux',\n 'type': 'PERSON',\n 'wikidataId': 'Q388',\n 'wikipediaExternalRef': 6097297\n },\n {\n 'domains': ['Computer_Science'],\n 'nerd_score': 0.7442,\n 'nerd_selection_score': 0.85,\n 'offsetEnd': 78,\n 'offsetStart': 49,\n 'rawName': 'free and open-source software',\n 'wikidataId': 'Q506883',\n 'wikipediaExternalRef': 1721496\n },\n {\n 'domains': ['Electrotechnology', 'Electronics',\n 'Computer_Science'],\n 'nerd_score': 0.7442,\n 'nerd_selection_score': 0.4487,\n 'offsetEnd': 96,\n 'offsetStart': 79,\n 'rawName': 'operating systems',\n 'wikidataId': 'Q9135',\n 'wikipediaExternalRef': 22194\n },\n {\n 'domains': [\n 'Electrotechnology', 'Electronics', 'Computer_Science'\n ],\n 'nerd_score': 0.7442,\n 'nerd_selection_score': 0.4487,\n 'offsetEnd': 100,\n 'offsetStart': 98,\n 'rawName': 'operating systems',\n 'wikidataId': 'Q9135',\n 'wikipediaExternalRef': 22194\n },\n {\n 'domains': ['Electronics', 'Computer_Science'],\n 'nerd_score': 0.743,\n 'nerd_selection_score': 0.8383,\n 'offsetEnd': 131,\n 'offsetStart': 119,\n 'rawName': 'Linux kernel',\n 'wikidataId': 'Q14579',\n 'wikipediaExternalRef': 21347315\n }\n ],\n 'global_categories': [\n {'category': 'Finnish inventions',\n 'page_id': 27421536,\n 'source': 'wikipedia-en',\n 'weight': 0.09684039970133569},\n {'category': 'Free software programmed in C',\n 'page_id': 11241711,\n 'source': 'wikipedia-en',\n 'weight': 0.06433942787438053},\n {'category': 'Unix variants',\n 'page_id': 10429397,\n 'source': 'wikipedia-en',\n 'weight': 0.09684039970133569},\n {'category': 'Operating systems',\n 'page_id': 693664,\n 'source': 'wikipedia-en',\n 'weight': 0.12888888710813473},\n {'category': 'Free software',\n 'page_id': 693287,\n 'source': 'wikipedia-en',\n 'weight': 0.06444444355406737},\n {'category': 'Free system software',\n 'page_id': 6721544,\n 'source': 'wikipedia-en',\n 'weight': 0.06433942787438053},\n {'category': 'Software licenses',\n 'page_id': 703100,\n 'source': 'wikipedia-en',\n 'weight': 0.06444444355406737},\n {'category': 'Linux kernel',\n 'page_id': 13215678,\n 'source': 'wikipedia-en',\n 'weight': 0.06433942787438053},\n {'category': 'Monolithic kernels',\n 'page_id': 10730969,\n 'source': 'wikipedia-en',\n 'weight': 0.06433942787438053},\n {'category': '1991 software',\n 'page_id': 11167446,\n 'source': 'wikipedia-en',\n 'weight': 0.09684039970133569},\n {'category': 'Linus Torvalds',\n 'page_id': 53479567,\n 'source': 'wikipedia-en',\n 'weight': 0.09684039970133569}\n ],\n 'language': {'conf': 0.9999973266294648, 'lang': 'en'},\n 'nbest': False,\n 'onlyNER': False,\n 'runtime': 107,\n 'sentences': [{'offsetEnd': 132, 'offsetStart': 0}],\n 'text': 'Linux is a name that broadly denotes a family of free and open-source software operating systems (OS) built around the Linux kernel.'\n },\n 200\n )\n\nBatch processing\n######################\nThe batch processing is implemented in the class ``NerdBatch``.\nThe class can be instantiated by defining the entity-fishing url in the constructor, else the default one is used.\n\nTo run the processing, the method `process` requires the `input` directory, a callback and the number of threads/processes.\nThere is an already ready implementation in `script/batchSample.py`.\n\nTo run it:\n - under this work branch, prepare two folders: `input` which containing the input Pdf files to be processed and `output` which collecting the processing result\n - we recommend to create a new virtualenv, activate it and install all the requirements needed in this virtual environment using `$ pip install -r /path/of/entity-fishing-client-python/source/requirements.txt`\n - (temporarly, until this branch is not merged) install entity-fishing **multithread branch** in edit mode (`pip install -e /path/of/entity-fishing-client-python/source`)\n - run it with `python runFile.py input output 5`\n\n\nKB access\n#########\n.. code-block:: python\n\n nerd.get_concept(\"Q456\")\n\n\nwith response\n\n.. code-block:: python\n\n (\n {\n 'rawName': 'Lyon',\n 'preferredTerm': 'Lyon',\n 'nerd_score': 0,\n 'nerd_selection_score': 0,\n 'wikipediaExternalRef': 8638634,\n 'wikidataId': 'Q456',\n 'definitions': [\n {\n 'definition': \"'''Lyon''' ( or ;, locally: ; ), also known as ''Lyons'', is a city in east-central [[France]], in the [[Auvergne-Rh\u00f4ne-Alpes]] [[Regions of France|region]], about from [[Paris]], from [[Marseille]] and from [[Saint-\u00c9tienne]]. Inhabitants of the city are called ''Lyonnais''.\",\n 'source': 'wikipedia-en',\n 'lang': 'en'\n }\n ],\n 'domains': [\n 'Geology',\n 'Sociology'\n ],\n 'categories': [\n {\n 'source': 'wikipedia-en',\n 'category': 'World Heritage Sites in France',\n 'page_id': 1178961\n },\n [...]\n ],\n 'multilingual': [\n {\n 'lang': 'de',\n 'term': 'Lyon',\n 'page_id': 13964\n },\n {\n 'lang': 'es',\n 'term': 'Lyon',\n 'page_id': 46490\n },\n {\n 'lang': 'fr',\n 'term': 'Lyon',\n 'page_id': 802627\n },\n {\n 'lang': 'it',\n 'term': 'Lione',\n 'page_id': 41786\n }\n ],\n 'statements': [\n {\n 'conceptId': 'Q456',\n 'propertyId': 'P1082',\n 'propertyName': 'population',\n 'valueType': 'quantity',\n 'value': {\n 'amount': '+500716',\n 'unit': '1',\n 'upperBound': '+500717',\n 'lowerBound': '+500715'\n }\n },\n {\n 'conceptId': 'Q456',\n 'propertyId': 'P1082',\n 'propertyName': 'population',\n 'valueType': 'quantity',\n 'value': {\n 'amount': '+500716',\n 'unit': '1',\n 'upperBound': '+500717',\n 'lowerBound': '+500715'\n }\n },\n {\n 'conceptId': 'Q456',\n 'propertyId': 'P1464',\n 'propertyName': 'category for people born here',\n 'valueType': 'wikibase-item',\n 'value': 'Q8061504'\n },\n {\n 'conceptId': 'Q456',\n 'propertyId': 'P190',\n 'propertyName': 'sister city',\n 'valueType': 'wikibase-item',\n 'value': 'Q5687',\n 'valueName': 'Jericho'\n },\n {\n 'conceptId': 'Q456',\n 'propertyId': 'P190',\n 'propertyName': 'sister city',\n 'valueType': 'wikibase-item',\n 'value': 'Q2079',\n 'valueName': 'Leipzig'\n },\n {\n 'conceptId': 'Q456',\n 'propertyId': 'P190',\n 'propertyName': 'sister city',\n 'valueType': 'wikibase-item',\n 'value': 'Q580',\n 'valueName': '\u0141\u00f3d\u017a'\n },\n [...]\n ]\n },\n 200\n )\n\n\nUtilities\n#########\n\nLanguage detection\n==================\n.. code-block:: python\n\n nerd.get_language(\"This is a sentence. This is a second sentence.\")\n\n\nwith response\n\n.. code-block:: python\n\n (\n {\n 'sentences':\n [\n {'offsetStart': 0, 'offsetEnd': 19},\n {'offsetStart': 19, 'offsetEnd': 46}\n ]\n },\n 200\n )\n\nSegmentation\n============\n.. code-block:: python\n\n nerd.segment(\"This is a sentence. This is a second sentence.\")\n\n\nwith response\n.. code-block:: python\n\n (\n {\n \"lang\": \"en\",\n \"conf\": 0.9\n },\n 200\n )", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Hirmeos/entity-fishing-client-python", "keywords": "named entity recognition,entity matching,entity recognition", "license": "Apache 2.0", "maintainer": "", "maintainer_email": "", "name": "entity-fishing-client", "package_url": "https://pypi.org/project/entity-fishing-client/", "platform": "", "project_url": "https://pypi.org/project/entity-fishing-client/", "project_urls": { "Homepage": "https://github.com/Hirmeos/entity-fishing-client-python" }, "release_url": "https://pypi.org/project/entity-fishing-client/0.7.1/", "requires_dist": null, "requires_python": "", "summary": "A minimal client for entity-fishing service.", "version": "0.7.1" }, "last_serial": 5096341, "releases": { "0.2.1": [ { "comment_text": "", "digests": { "md5": "0b3451dafbb637d5200f29a7b3d6b5f5", "sha256": "494dc38beedb3e411450dc63dd23d5602535a29d9ee0309993d0b299ca00f9b1" }, "downloads": -1, "filename": "entity-fishing-client-0.2.1.tar.gz", "has_sig": false, "md5_digest": "0b3451dafbb637d5200f29a7b3d6b5f5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6097, "upload_time": "2018-03-06T16:47:15", "url": "https://files.pythonhosted.org/packages/be/1a/68a1232cf7dbd08dd6b2b053f2442da7d39d2699ccfff7efbb11795d52bc/entity-fishing-client-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "3256ede11b3039c5cdf93ba18e10b9fe", "sha256": "28d79459cdbb0c9c993153e514b0c1b264e065341476951feb5f69cad7940d98" }, "downloads": -1, "filename": "entity-fishing-client-0.2.2.tar.gz", "has_sig": false, "md5_digest": "3256ede11b3039c5cdf93ba18e10b9fe", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6088, "upload_time": "2018-03-08T13:21:36", "url": "https://files.pythonhosted.org/packages/a0/94/35b63e63e50a7296b508710283a33e34be56cf390a47fab11c155521e3db/entity-fishing-client-0.2.2.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "23e85c97659b958dae69f1b16eb8b392", "sha256": "8aab5d9ba16b405fd4e065ddac07300a0444ba66565355ed52514ea6cfc6a212" }, "downloads": -1, "filename": "entity-fishing-client-0.3.0.tar.gz", "has_sig": false, "md5_digest": "23e85c97659b958dae69f1b16eb8b392", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6055, "upload_time": "2018-04-03T16:01:12", "url": "https://files.pythonhosted.org/packages/09/26/88230ecc933845e9e79595f7f522d515e3e98dcd86a105e5585c080a09dd/entity-fishing-client-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "1b586e5c4a03f279f5cd1581da44cd1e", "sha256": "006abc5af751d8e508ade6398ebb7256c633cda3cb4de14d6d5e87694534754e" }, "downloads": -1, "filename": "entity-fishing-client-0.4.0.tar.gz", "has_sig": false, "md5_digest": "1b586e5c4a03f279f5cd1581da44cd1e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6052, "upload_time": "2018-04-05T13:14:51", "url": "https://files.pythonhosted.org/packages/04/a6/0ab0ef6b3885c0e78aa7126086b5f92b08fd4f815b172aa17ab6f9eeb9ca/entity-fishing-client-0.4.0.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "6186c988707aa7055f9d640d0a2feec8", "sha256": "4a16b2ea6f98a49e4233647c8e155197e0c6d0c8068d672fe9357d6ebd53126e" }, "downloads": -1, "filename": "entity-fishing-client-0.5.1.tar.gz", "has_sig": false, "md5_digest": "6186c988707aa7055f9d640d0a2feec8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7430, "upload_time": "2018-08-10T15:39:29", "url": "https://files.pythonhosted.org/packages/19/c3/00b4cfcdc5ef6e7450ec3543cfe23d866ff7f677728f951324a4f83bc9af/entity-fishing-client-0.5.1.tar.gz" } ], "0.6.0": [ { "comment_text": "", "digests": { "md5": "fe2bc47ac8f21e0de8bf0632479b479e", "sha256": "0fdfc51f99355110bcf926412b19a3d076964a1cdf33e66fc6f3ef3a5b64bc9e" }, "downloads": -1, "filename": "entity-fishing-client-0.6.0.tar.gz", "has_sig": false, "md5_digest": "fe2bc47ac8f21e0de8bf0632479b479e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12300, "upload_time": "2018-11-20T14:02:47", "url": "https://files.pythonhosted.org/packages/7a/a7/2f330077bce042ad144d1d1a739ff1bd5594629c6b43682ebc85e4089a46/entity-fishing-client-0.6.0.tar.gz" } ], "0.7.0": [ { "comment_text": "", "digests": { "md5": "536c4bff068242f666e64bf76a3d256c", "sha256": "687298c479bbbcc0447f61fe7004658c0a025cffa2ee52d989b879e537a483dd" }, "downloads": -1, "filename": "entity-fishing-client-0.7.0.tar.gz", "has_sig": false, "md5_digest": "536c4bff068242f666e64bf76a3d256c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12298, "upload_time": "2018-11-20T14:02:44", "url": "https://files.pythonhosted.org/packages/c2/bd/c11de6270d580ba40edac7a43a294716428538b852a1062a69e46592b31c/entity-fishing-client-0.7.0.tar.gz" } ], "0.7.1": [ { "comment_text": "", "digests": { "md5": "2964d688c14afad1efd055a402de0aad", "sha256": "7bf097e384a396666e7de4487641ba746c9634988c430c1212bf770364da3385" }, "downloads": -1, "filename": "entity-fishing-client-0.7.1.tar.gz", "has_sig": false, "md5_digest": "2964d688c14afad1efd055a402de0aad", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12304, "upload_time": "2019-04-04T10:03:07", "url": "https://files.pythonhosted.org/packages/a8/0c/45c71a2653f56dd1192ea193fb4b4cfb66e494afa5928d6f82a8093afc9b/entity-fishing-client-0.7.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2964d688c14afad1efd055a402de0aad", "sha256": "7bf097e384a396666e7de4487641ba746c9634988c430c1212bf770364da3385" }, "downloads": -1, "filename": "entity-fishing-client-0.7.1.tar.gz", "has_sig": false, "md5_digest": "2964d688c14afad1efd055a402de0aad", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12304, "upload_time": "2019-04-04T10:03:07", "url": "https://files.pythonhosted.org/packages/a8/0c/45c71a2653f56dd1192ea193fb4b4cfb66e494afa5928d6f82a8093afc9b/entity-fishing-client-0.7.1.tar.gz" } ] }