{ "info": { "author": "CERN", "author_email": "info@invenio-software.org", "bugtrack_url": null, "classifiers": [ "Development Status :: 1 - Planning", "Environment :: Web Environment", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v2 (GPLv2)", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Topic :: Internet :: WWW/HTTP :: Dynamic Content", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "..\n This file is part of Invenio.\n Copyright (C) 2015 CERN.\n\n Invenio is free software; you can redistribute it\n and/or modify it under the terms of the GNU General Public License as\n published by the Free Software Foundation; either version 2 of the\n License, or (at your option) any later version.\n\n Invenio is distributed in the hope that it will be\n useful, but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU\n General Public License for more details.\n\n You should have received a copy of the GNU General Public License\n along with Invenio; if not, write to the\n Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,\n MA 02111-1307, USA.\n\n In applying this license, CERN does not\n waive the privileges and immunities granted to it by virtue of its status\n as an Intergovernmental Organization or submit itself to any jurisdiction.\n\n======================\n Invenio-OAIHarvester\n======================\n\n.. image:: https://img.shields.io/travis/inveniosoftware/invenio-oaiharvester.svg\n :target: https://travis-ci.org/inveniosoftware/invenio-oaiharvester\n\n.. image:: https://img.shields.io/coveralls/inveniosoftware/invenio-oaiharvester.svg\n :target: https://coveralls.io/r/inveniosoftware/invenio-oaiharvester\n\n.. image:: https://img.shields.io/github/tag/inveniosoftware/invenio-oaiharvester.svg\n :target: https://github.com/inveniosoftware/invenio-oaiharvester/releases\n\n.. image:: https://img.shields.io/pypi/dm/invenio-oaiharvester.svg\n :target: https://pypi.python.org/pypi/invenio-oaiharvester\n\n.. image:: https://img.shields.io/github/license/inveniosoftware/invenio-oaiharvester.svg\n :target: https://github.com/inveniosoftware/invenio-oaiharvester/blob/master/LICENSE\n\n\nInvenio module for OAI-PMH metadata harvesting between repositories.\n\n* Free software: GPLv2 license\n* Documentation: https://invenio-oaiharvester.readthedocs.org.\n\n*This is an experimental development preview release.*\n\nFeatures\n========\n\nThis module allows you to easily harvest OAI-PMH repositories, thanks to the `Sickle`_ module, and feed the\noutput into your ingestion workflows, or simply to files. You can configure\nyour OAI-PMH sources via a web-interface and run or schedule immediate harvesting jobs\nvia command-line or regularly via `Celery beat`_.\n\n.. _Celery beat: http://celery.readthedocs.org/en/latest/userguide/periodic-tasks.html\n.. _Sickle: http://sickle.readthedocs.org/en/latest/\n\nHarvesting is simple\n====================\n\n.. code-block:: shell\n\n inveniomanage oaiharvester get -u http://export.arxiv.org/oai2 -i oai:arXiv.org:1507.07286 > my_record.xml\n\n\nThis will harvest the repository for a specific record and print the records to stdout - which in this case will save it to a file called ``my_record.xml``.\n\nIf you want to have your harvested records saved in a directory automatically, its easy:\n\n.. code-block:: shell\n\n inveniomanage oaiharvester get -u http://export.arxiv.org/oai2 -i oai:arXiv.org:1507.07286 -o dir\n\n\nNote the output ``-o`` parameter that specifies how to output the harvested records. The three options are:\n\n * Sent to a workflow (E.g. `-o workflow`)\n * Saved files in a folder (E.g. `-o dir`)\n * Printed to stdout (default)\n\n\nHarvesting with workflows\n=========================\n\n.. code-block:: shell\n\n inveniomanage oaiharvester get -u http://export.arxiv.org/oai2 -i oai:arXiv.org:1507.07286 -o workflow\n\nWhen you send an harvested record to a workflow you can process the harvested\nfiles however you'd like and then even upload it automatically into your own repository.\n\nThis module already provides some\n\n\nManaging OAI-PMH sources\n========================\n\nIf you want to store configuration for an OAI repository, you can use the\nadministration interface available via the admin panel. This is useful if you regularly need to query a server.\n\nHere you can add information about the server URL, metadataPrefix to use etc. This information is also available when scheduling and running tasks:\n\n\n.. code-block:: shell\n\n inveniomanage oaiharvester get -n somerepo -i oai:example.org:1234\n\nHere we are using the `-n, --name` parameter to specify which stored OAI-PMH source to query, by name.\n\n\nAPI\n===\n\nIf you need to schedule or run harvests via Python, you can use our API:\n\n.. code-block:: python\n\n from invenio_oaiharvester.api import get_records\n for rec in get_records(identifiers=[\"oai:arXiv.org:1207.7214\"],\n url=\"http://export.arxiv.org/oai2\"):\n print rec.raw\n\n\n..\n This file is part of Invenio.\n Copyright (C) 2015 CERN.\n\n Invenio is free software; you can redistribute it\n and/or modify it under the terms of the GNU General Public License as\n published by the Free Software Foundation; either version 2 of the\n License, or (at your option) any later version.\n\n Invenio is distributed in the hope that it will be\n useful, but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU\n General Public License for more details.\n\n You should have received a copy of the GNU General Public License\n along with Invenio; if not, write to the\n Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,\n MA 02111-1307, USA.\n\n In applying this license, CERN does not\n waive the privileges and immunities granted to it by virtue of its status\n as an Intergovernmental Organization or submit itself to any jurisdiction.\n\nChanges\n=======\n\nVersion 0.1.1 (released 2015-08-25)\n\n- Adds missing `invenio_upgrader` dependency and amends past upgrade\n recipes following its separation into standalone package.\n\n- Fixes imports due to separation of invenio-workflows. (#9)\n\nVersion 0.1.0 (released 2015-08-14)\n\n- Initial public release", "description_content_type": null, "docs_url": "https://pythonhosted.org/invenio-oaiharvester/", "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/inveniosoftware/invenio-oaiharvester", "keywords": "invenio oai-pmh harvest metadata", "license": "GPLv2", "maintainer": null, "maintainer_email": null, "name": "invenio-oaiharvester", "package_url": "https://pypi.org/project/invenio-oaiharvester/", "platform": "any", "project_url": "https://pypi.org/project/invenio-oaiharvester/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/inveniosoftware/invenio-oaiharvester" }, "release_url": "https://pypi.org/project/invenio-oaiharvester/0.1.1/", "requires_dist": null, "requires_python": null, "summary": "Invenio module for OAI-PMH metadata harvesting between repositories.", "version": "0.1.1" }, "last_serial": 3390849, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "65bf34875e6f9c567e48b057e23c7f49", "sha256": "2337c87ba197ecdfe93353dd53df6b76988e54b97e416e872f0a39d0e6ccf2ad" }, "downloads": -1, "filename": "invenio_oaiharvester-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "65bf34875e6f9c567e48b057e23c7f49", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 38434, "upload_time": "2015-08-14T14:36:49", "url": "https://files.pythonhosted.org/packages/ef/8b/9a6058acf06e99871137d054cd57a8b4d448e47ad64fc22701532a7d8ae0/invenio_oaiharvester-0.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2255af5ad24f63d4de54e6169011995b", "sha256": "402a4657eb011f30d1105549f123f3af1a99c73467d74657d56ece2442cfeda7" }, "downloads": -1, "filename": "invenio-oaiharvester-0.1.0.tar.gz", "has_sig": false, "md5_digest": "2255af5ad24f63d4de54e6169011995b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 41840, "upload_time": "2015-08-14T14:36:46", "url": "https://files.pythonhosted.org/packages/11/ff/610ff4056790c8546a35c9709fb2944557fed322d60d9a1458d975fddf0c/invenio-oaiharvester-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "e2211df7ed8aa580de8dba41a62e2e69", "sha256": "405d8a777c498294e83f0fc3fad43b1ca2b23ae81d64b2096b5b02cde4b5e426" }, "downloads": -1, "filename": "invenio-oaiharvester-0.1.1.tar.gz", "has_sig": false, "md5_digest": "e2211df7ed8aa580de8dba41a62e2e69", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 43123, "upload_time": "2015-08-25T20:19:26", "url": "https://files.pythonhosted.org/packages/eb/aa/c58f68afe8a138fd2d3da4e78f958bf4f793f5342279fa427c2ef62d8460/invenio-oaiharvester-0.1.1.tar.gz" } ], "1.0.0a2": [ { "comment_text": "", "digests": { "md5": "9b22f8eb3f6b5770a1b8311d367ee155", "sha256": "a993e87d2951d15cc1d4253e73d3791fb9ec2ebb27410921df57010afc6992d3" }, "downloads": -1, "filename": "invenio_oaiharvester-1.0.0a2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9b22f8eb3f6b5770a1b8311d367ee155", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 21022, "upload_time": "2016-06-15T14:20:33", "url": "https://files.pythonhosted.org/packages/74/af/695947c330f519f1c1d5fd2a579f92625a6112763ce44a19ea9bb9abb0c5/invenio_oaiharvester-1.0.0a2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "63decf0adfda759616ab8b8d7406d224", "sha256": "e8f195a1d10439c5964e45207c67f1d9a545955106a4d45ebe7f2abe007736f5" }, "downloads": -1, "filename": "invenio-oaiharvester-1.0.0a2.tar.gz", "has_sig": false, "md5_digest": "63decf0adfda759616ab8b8d7406d224", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 145442, "upload_time": "2016-06-15T14:20:59", "url": "https://files.pythonhosted.org/packages/81/4a/687ac5ed69eedfe780cfc3b10373af5b1b8c239e8e7cc8bb162b5b23bb47/invenio-oaiharvester-1.0.0a2.tar.gz" } ], "1.0.0a3": [ { "comment_text": "", "digests": { "md5": "28b8d45ad8b8fc239b225d9189131b73", "sha256": "c9985db602ac539cbba2ef7375301e28445edf0be8ea9bbf7770432e6b1ae54e" }, "downloads": -1, "filename": "invenio_oaiharvester-1.0.0a3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "28b8d45ad8b8fc239b225d9189131b73", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 21228, "upload_time": "2017-09-22T16:02:08", "url": "https://files.pythonhosted.org/packages/2f/65/6987cf6551f641993c12afd4aedadc16ef1c51fef9c7cbc7ab06f306940d/invenio_oaiharvester-1.0.0a3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f85e53d2d2e992326b3d8a3e76b48b0d", "sha256": "c25074f27f214700e9ba263ad80b2632e920a1a3ab87f2332f80af88fb6908bd" }, "downloads": -1, "filename": "invenio-oaiharvester-1.0.0a3.tar.gz", "has_sig": false, "md5_digest": "f85e53d2d2e992326b3d8a3e76b48b0d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 144942, "upload_time": "2017-09-22T16:02:11", "url": "https://files.pythonhosted.org/packages/e7/09/518fee4fea57adbdd4d5acdbe71ea59f2f11724c0551fdc85cc509904793/invenio-oaiharvester-1.0.0a3.tar.gz" } ], "1.0.0a4": [ { "comment_text": "", "digests": { "md5": "75adbcff8bfff966f83fa217ec217e42", "sha256": "7dc2d2d9572807120599a798779424df1261fa8bea77c0fbb4e81a41ff1291dc" }, "downloads": -1, "filename": "invenio_oaiharvester-1.0.0a4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "75adbcff8bfff966f83fa217ec217e42", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 21353, "upload_time": "2017-12-05T15:07:09", "url": "https://files.pythonhosted.org/packages/b9/32/28275fbc1f6bddcfaaf84d6e4b660599f4276e4de968a3848bdd355f5676/invenio_oaiharvester-1.0.0a4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6f3fd765e688e5e7f982803e4d4c0722", "sha256": "9be11255e52c8b932135e38060533409db9e5a7812f40ccac6ec04122770bdef" }, "downloads": -1, "filename": "invenio-oaiharvester-1.0.0a4.tar.gz", "has_sig": false, "md5_digest": "6f3fd765e688e5e7f982803e4d4c0722", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 145028, "upload_time": "2017-12-05T15:07:12", "url": "https://files.pythonhosted.org/packages/b2/5c/dd0108ae91f0db4b81d143b5714ae24274685f80662669f23a574fe1bd01/invenio-oaiharvester-1.0.0a4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e2211df7ed8aa580de8dba41a62e2e69", "sha256": "405d8a777c498294e83f0fc3fad43b1ca2b23ae81d64b2096b5b02cde4b5e426" }, "downloads": -1, "filename": "invenio-oaiharvester-0.1.1.tar.gz", "has_sig": false, "md5_digest": "e2211df7ed8aa580de8dba41a62e2e69", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 43123, "upload_time": "2015-08-25T20:19:26", "url": "https://files.pythonhosted.org/packages/eb/aa/c58f68afe8a138fd2d3da4e78f958bf4f793f5342279fa427c2ef62d8460/invenio-oaiharvester-0.1.1.tar.gz" } ] }