{ "info": { "author": "UCL Machine Reading", "author_email": "s.riedel@cs.ucl.ac.uk", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering :: Artificial Intelligence", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# Jack the Reader [![Wercker build badge][wercker_badge]][wercker] [![codecov](https://codecov.io/gh/uclmr/jack/branch/master/graph/badge.svg?token=jbZrj9oSmi)](https://codecov.io/gh/uclmr/jack) [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/jack-the-reader/Lobby?source=orgpage) [![license](https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000)](https://github.com/uclmr/jack/blob/master/LICENSE)\n\n##### A Machine Reading Comprehension framework.\n\n* All work and no play makes Jack a great frame*work*!\n* All work and no play makes Jack a great frame*work*!\n* All work and no play makes Jack a great frame*work*!\n\n[wercker_badge]: https://app.wercker.com/status/8ed61192a5b16769a41dc24c30a3bc6a/s/master\n[wercker]: https://app.wercker.com/project/byKey/8ed61192a5b16769a41dc24c30a3bc6a\n[heres_johnny]: https://upload.wikimedia.org/wikipedia/en/b/bb/The_shining_heres_johnny.jpg\n\n**Jack the Reader** - or **jack**, for short - is a framework for building and using models on a variety of tasks that require *reading comprehension*. For more informations about the overall architecture, we refer to [Jack the Reader \u2013 A Machine Reading Framework](https://arxiv.org/abs/1806.08727) (ACL 2018).\n\n## Installation\nTo install Jack, install requirements and [TensorFlow](http://tensorflow.org/). In case you want to use PyTorch for writing models, please install [PyTorch](http://pytorch.org/) as well.\n\n## Supported ML Backends\n\nWe currently support [TensorFlow](http://tensorflow.org/) and [PyTorch](http://pytorch.org/).\nReaders can be implemented using both. Input and output modules (i.e., pre- and post-processing) are independent of the\nML backend and can thus be reused for model modules that either backend.\nThough most models are implemented in TensorFlow by reusing the cumbersome pre- and post-processing it is easy to\nquickly build new readers in PyTorch as well.\n\n## Pre-trained Models\n\nFind pre-trained models [here](https://www.dropbox.com/sh/vnmc9pq4yrgl1sr/AAD7HVhGdpof2IgIifSZ6PEUa?dl=0).\n\n## Code Structure\n\n* `jack.core` - core abstractions used\n* `jack.readers` - implementations of models\n* `jack.eval` - task evaluation code\n* `jack.util` - utility code that is used throughout the framework, including shared ML code\n* `jack.io` - IO related code, including loading and dataset conversion scripts\n\n\n## Projects\n\n* [Integration of Knowledge into neural NLU systems](/projects/knowledge_integration)\n\n## Quickstart\n\n### Coding Tutorials - Notebooks & CLI\nWe provide ipython notebooks with tutorials on Jack. For the quickest start, you can begin [here][quickstart]. If you're interested in training a model yourself from code, see [this tutorial][model_training] (we recommend the command-line, see below), and if you'd like to implement a new model yourself, [this notebook][implementation] gives you a tutorial that explains this process in more detail.\n\nThere is documentation on our [command-line interface][cli] for actually **training and evaluating models**.\nFor a high-level explanation of the ideas and vision, see [Understanding Jack the Reader][understanding].\n\n[quickstart]: notebooks/quick_start.ipynb\n[model_training]: notebooks/model_training.ipynb\n[implementation]: notebooks/model_implementation.ipynb\n[install]: docs/How_to_install_and_run.md\n[api]: https://uclmr.github.io/jack/\n[notebooks]: notebooks/\n[understanding]: docs/Understanding_Jack_the_Reader.md\n[cli]: docs/CLI.md\n\n### Command-line Training and Usage of a QA System\nTo illustrate how jack works, we will show how to train a question answering\nmodel using our [command-line interface][cli] which is analoguous for other tasks (browse [conf/](./conf/) for existing task-dataset configurations).\nIt is probably best to setup a [virtual environment](https://docs.python.org/3/library/venv.html) to avoid\nclashes with system wide python library versions.\n\nFirst, install the framework:\n\n```bash\n$ python3 -m pip install -e .[tf]\n```\n\nThen, download the SQuAD dataset, and the GloVe word embeddings:\n\n```bash\n$ ./data/SQuAD/download.sh\n$ ./data/GloVe/download.sh\n```\n\nTrain a [FastQA][fastqa] model:\n\n```bash\n$ python3 bin/jack-train.py with train='data/SQuAD/train-v1.1.json' dev='data/SQuAD/dev-v1.1.json' reader='fastqa_reader' \\\n> repr_dim=300 dropout=0.5 batch_size=64 seed=1337 loader='squad' save_dir='./fastqa_reader' epochs=20 \\\n> with_char_embeddings=True embedding_format='memory_map_dir' embedding_file='data/GloVe/glove.840B.300d.memory_map_dir' vocab_from_embeddings=True\n```\n\nor shorter, using our prepared config:\n\n```bash\n$ python3 bin/jack-train.py with config='./conf/qa/squad/fastqa.yaml'\n```\n\nA copy of the model is written into the `save_dir` directory after each\ntraining epoch when performance improves. These can be loaded using the commands below or see e.g.\n[quickstart].\n\nYou want to train another model? No problem, we have a fairly modular QAModel implementation which allows you to stick\ntogether your own model. There are examples in `conf/qa/squad/` (e.g., `bidaf.yaml` or our own creation `jack_qa.yaml`).\nThese models are defined solely in the configs, i.e., there is not implementation in code.\nThis is possible through our `ModularQAModel`.\n\nIf all of that is too cumbersome for you and you just want to play, why not downloading a pretrained model:\n\n```bash\n$ # we still need GloVe in memory mapped format, ignore the next 2 commands if already downloaded and transformed\n$ data/GloVe/download.sh\n$ wget -O fastqa.zip https://www.dropbox.com/s/qb796uljoqj0lvo/fastqa.zip?dl=1\n$ unzip fastqa.zip && mv fastqa fastqa_reader\n```\n\n```python\nfrom jack import readers\nfrom jack.core import QASetting\n\nfastqa_reader = readers.reader_from_file(\"./fastqa_reader\")\n\nsupport = \"\"\"\"It is a replica of the grotto at Lourdes,\nFrance where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858.\nAt the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome),\nis a simple, modern stone statue of Mary.\"\"\"\n\nanswers = fastqa_reader([QASetting(\n question=\"To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?\",\n support=[support]\n)])\n\nprint(answers[0][0].text)\n```\n[fastqa]: https://arxiv.org/abs/1703.04816\n[tf_summaries]: https://www.tensorflow.org/get_started/summaries_and_tensorboard\n[quick_start]: notebooks/quick_start.ipynb\n[cli]: docs/CLI.md\n\n## Support\nWe are thankful for support from:\n\n\n\n\n\n\n\n\n\n\n## Developer guidelines\n\n- [Comply with the PEP 8 Style Guide][pep8]\n- Make sure all your code runs from the top level directory, e.g.:\n\n```shell\n$ pwd\n/home/pasquale/workspace/jack\n$ python3 bin/jack-train.py [..]\n```\n\n[pep8]: https://www.python.org/dev/peps/pep-0008/\n\n## Citing\n\n```\n@InProceedings{weissenborn2018jack,\nauthor = {Dirk Weissenborn, Pasquale Minervini, Tim Dettmers, Isabelle Augenstein, Johannes Welbl, Tim Rockta\u0308schel, Matko Bos\u030cnjak, Jeff Mitchell, Thomas Demeester, Pontus Stenetorp, Sebastian Riedel},\ntitle = {{Jack the Reader \u2013 A Machine Reading Framework}},\nbooktitle = {{Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL) System Demonstrations}},\nMonth = {July},\nyear = {2018},\nurl = {https://arxiv.org/abs/1806.08727}\n}\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/uclmr/jack", "keywords": "tensorflow machine learning natural language processing question answering", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "uclmr-jack", "package_url": "https://pypi.org/project/uclmr-jack/", "platform": "", "project_url": "https://pypi.org/project/uclmr-jack/", "project_urls": { "Homepage": "https://github.com/uclmr/jack" }, "release_url": "https://pypi.org/project/uclmr-jack/0.2.1/", "requires_dist": [ "jsonschema", "numpy", "parse", "scipy", "sklearn", "typing", "sacred (==0.7.2)", "sqlalchemy", "pyyaml", "progressbar2", "spacy (==1.9)", "diskcache", "pytest", "pytest-runner", "pytest-xdist", "pytest-pep8", "pytest-xdist", "pytest-cov", "codecov", "tensorflow (==1.8.0); extra == 'tf'", "tensorflow-gpu (==1.8.0); extra == 'tf_gpu'", "torch; extra == 'torch'" ], "requires_python": "", "summary": "Jack the Reader is a Python framework for Machine Reading", "version": "0.2.1" }, "last_serial": 4097747, "releases": { "0.2.0": [ { "comment_text": "", "digests": { "md5": "4a731d970d253fe6480e7a1b4650a7a4", "sha256": "d9e640dfca8453bd570ac49e28bf1a3c7de1a0d3cdecbea596fdfb025f50da09" }, "downloads": -1, "filename": "uclmr_jack-0.2.0-py3.6.egg", "has_sig": false, "md5_digest": "4a731d970d253fe6480e7a1b4650a7a4", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": null, "size": 125393, "upload_time": "2018-07-24T13:31:41", "url": "https://files.pythonhosted.org/packages/a2/35/159ff9511e34d22996dba3e7530f2ace6826013c63f58a3394eb521e4783/uclmr_jack-0.2.0-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "7f26554aceb194fd3f42a9ce82d18672", "sha256": "43ced8c327d4fc2685d10b92c8e06508bdae1b5864b9da33cfcd22609a9f934d" }, "downloads": -1, "filename": "uclmr_jack-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "7f26554aceb194fd3f42a9ce82d18672", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 129119, "upload_time": "2018-07-24T13:31:38", "url": "https://files.pythonhosted.org/packages/bb/c1/403ac902b815737ce1c5b69e151abad70d2cad67e94c6d88375f9a8523e5/uclmr_jack-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "679898e85d0d005faa8acba93c84cbe2", "sha256": "3d853921838a7e71b5a7d3dd6218713570733a80a919c0b3fbd9146eef47bceb" }, "downloads": -1, "filename": "uclmr-jack-0.2.0.tar.gz", "has_sig": false, "md5_digest": "679898e85d0d005faa8acba93c84cbe2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 92875, "upload_time": "2018-07-24T13:31:40", "url": "https://files.pythonhosted.org/packages/ce/2d/1bdc96292e5ae22f8666a468b2b656f64d58924d62b2bdda82ce4320eb31/uclmr-jack-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "a3889ddb760f67677104ced28e1507c8", "sha256": "d974ec89345a44a6ed3f1e3e5ee1e8d95e008a401c4f1ee20eade6a4f5d18aef" }, "downloads": -1, "filename": "uclmr_jack-0.2.1-py3.6.egg", "has_sig": false, "md5_digest": "a3889ddb760f67677104ced28e1507c8", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": null, "size": 129157, "upload_time": "2018-07-24T17:16:50", "url": "https://files.pythonhosted.org/packages/70/e3/8b86a42bf0928ca59534c2ce1be44c1b1c2e958585f194a7cc6f6a0e5817/uclmr_jack-0.2.1-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "36d2d41ff7ec6a5646bb6f96c6b36c9d", "sha256": "ae268c5e15c05652639a641fd3e43bdcb0e0af59088345e5a55120168e25a08a" }, "downloads": -1, "filename": "uclmr_jack-0.2.1-py3-none-any.whl", "has_sig": false, "md5_digest": "36d2d41ff7ec6a5646bb6f96c6b36c9d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 135262, "upload_time": "2018-07-24T17:16:48", "url": "https://files.pythonhosted.org/packages/5c/a0/3943928dd77b2d622f8d033a99394aecc686485606f771ce7310e1e92978/uclmr_jack-0.2.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9210a888a4fe683ead044e8856e38fa5", "sha256": "4cd2b4efcec044d75f5dc196679604df8dc05c19f2a475115c9cded826365282" }, "downloads": -1, "filename": "uclmr-jack-0.2.1.tar.gz", "has_sig": false, "md5_digest": "9210a888a4fe683ead044e8856e38fa5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 93168, "upload_time": "2018-07-24T17:16:51", "url": "https://files.pythonhosted.org/packages/38/cd/c798267e9e7f9f73d95ac3c7fe2210fc9ccd5807f15c90040891c7e6cbdb/uclmr-jack-0.2.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a3889ddb760f67677104ced28e1507c8", "sha256": "d974ec89345a44a6ed3f1e3e5ee1e8d95e008a401c4f1ee20eade6a4f5d18aef" }, "downloads": -1, "filename": "uclmr_jack-0.2.1-py3.6.egg", "has_sig": false, "md5_digest": "a3889ddb760f67677104ced28e1507c8", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": null, "size": 129157, "upload_time": "2018-07-24T17:16:50", "url": "https://files.pythonhosted.org/packages/70/e3/8b86a42bf0928ca59534c2ce1be44c1b1c2e958585f194a7cc6f6a0e5817/uclmr_jack-0.2.1-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "36d2d41ff7ec6a5646bb6f96c6b36c9d", "sha256": "ae268c5e15c05652639a641fd3e43bdcb0e0af59088345e5a55120168e25a08a" }, "downloads": -1, "filename": "uclmr_jack-0.2.1-py3-none-any.whl", "has_sig": false, "md5_digest": "36d2d41ff7ec6a5646bb6f96c6b36c9d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 135262, "upload_time": "2018-07-24T17:16:48", "url": "https://files.pythonhosted.org/packages/5c/a0/3943928dd77b2d622f8d033a99394aecc686485606f771ce7310e1e92978/uclmr_jack-0.2.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9210a888a4fe683ead044e8856e38fa5", "sha256": "4cd2b4efcec044d75f5dc196679604df8dc05c19f2a475115c9cded826365282" }, "downloads": -1, "filename": "uclmr-jack-0.2.1.tar.gz", "has_sig": false, "md5_digest": "9210a888a4fe683ead044e8856e38fa5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 93168, "upload_time": "2018-07-24T17:16:51", "url": "https://files.pythonhosted.org/packages/38/cd/c798267e9e7f9f73d95ac3c7fe2210fc9ccd5807f15c90040891c7e6cbdb/uclmr-jack-0.2.1.tar.gz" } ] }