{ "info": { "author": "Igor Tokarev", "author_email": "TigorC@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Programming Language :: Python", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3 :: Only", "Topic :: Internet :: WWW/HTTP" ], "description": "[![Build Status](https://travis-ci.org/TigorC/aiounfurl.svg?branch=master)](https://travis-ci.org/TigorC/aiounfurl)\n[![Coverage Status](https://coveralls.io/repos/github/TigorC/aiounfurl/badge.svg?branch=master)](https://coveralls.io/github/TigorC/aiounfurl?branch=master)\n\n## aiounfurl\nUsing this library you can extract meta information from web pages and create site preview.\nThe library uses four sources of information:\n\n1. [oEmbed](http://oembed.com)\n2. [Open Graph](http://ogp.me)\n3. [Twitter Cards](https://dev.twitter.com/cards/overview)\n4. HTML meta tags\n\n## Requirements\n* python 3.5\n* aiohttp\n* beautifulsoup4\n* html5lib\n\n## Installation\n```bash\npip install aiounfurl\n```\n\n## Example of using\n\nTo extract all site data:\n\n```python\nimport asyncio\nimport aiohttp\nfrom pprint import pprint\nfrom aiounfurl.views import get_preview_data, fetch_all\n\n\nasync def get_links_data(links, loop):\n results = []\n async with aiohttp.ClientSession() as session:\n tasks = [fetch_all(session, l, loop) for l in links]\n results = await asyncio.gather(*tasks, loop=loop, return_exceptions=True)\n return [{'link':l, 'data': d} for l, d in zip(links, results)]\n\n\nlinks = [\n 'https://habrahabr.ru/post/314606/',\n 'https://www.youtube.com/watch?v=9EftQMnuhvU',\n 'https://medium.freecodecamp.com/million-requests-per-second-with-python-95c137af319'\n]\nloop = asyncio.get_event_loop()\nresult = loop.run_until_complete(get_links_data(links, loop))\nloop.close()\npprint(result)\n```\n\n## Server example.\nFull example you can find [here](https://github.com/TigorC/aiounfurl/blob/master/example/srv.py).\n\nInstall required packages for running example:\n\n```bash\npip install -r example/requirements.txt\n```\nRun `python srv.py runserver`, then open http://127.0.0.1:8080/\n\n## Running the example in Docker\n\nI added a docker image with the example in http://hub.docker.com/ to run the sample as a separate independent service.\n\nRunning in the background:\n\n```bash\ndocker run --name aiounfurl -p 8080:8080 -d tigorc/aiounfurl\n```\n\nthen you can open our example [http://127.0.0.1:8080/](http://127.0.0.1:8080/).\n\nUsing the list of oEmbed providers (a json file with a list of providers /path_to_file/providers.json has to be preliminarily created):\n\n```bash\ndocker run --name aiounfurl -p 8080:8080 -e \"OEMBED_PROVIDERS_FILE=/srv/app/providers.json\" -v /path_to_file/providers.json:/srv/app/providers.json -d tigorc/aiounfurl\n```\n\n## Tests\nInstall the `tox` package and run command:\n\n```bash\ntox\n```\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/tigorc/aiounfurl", "keywords": "async embed preview", "license": "BSD License", "maintainer": "", "maintainer_email": "", "name": "aiounfurl", "package_url": "https://pypi.org/project/aiounfurl/", "platform": "", "project_url": "https://pypi.org/project/aiounfurl/", "project_urls": { "Homepage": "https://github.com/tigorc/aiounfurl" }, "release_url": "https://pypi.org/project/aiounfurl/0.2.4/", "requires_dist": null, "requires_python": "", "summary": "Making site preview", "version": "0.2.4" }, "last_serial": 2636293, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "b201c304094789b58bc645d0b09cb861", "sha256": "f7c7efcd7d08bad83bbc63925e94de31632d97d5c9f35ff458e961b90cc0fbce" }, "downloads": -1, "filename": "aiounfurl-0.1.tar.gz", "has_sig": false, "md5_digest": "b201c304094789b58bc645d0b09cb861", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4533, "upload_time": "2016-12-26T16:39:22", "url": "https://files.pythonhosted.org/packages/96/43/c32aaedc98f452b50193c438d6b413bccc1353982b0ed6242caf64a73bca/aiounfurl-0.1.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "ce6883d28d3cdd022a85d5511c24379c", "sha256": "d95a21faac5009c76172007a7ee63e479d31058857779c0bd8a6f517643fc924" }, "downloads": -1, "filename": "aiounfurl-0.2.0.tar.gz", "has_sig": false, "md5_digest": "ce6883d28d3cdd022a85d5511c24379c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5318, "upload_time": "2017-01-14T16:34:26", "url": "https://files.pythonhosted.org/packages/ee/a7/cd43e04ff6019b105521d5be902ca33d3b7e9375f481006a069aa31142a3/aiounfurl-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "7e8c515e7a87b60be5d00f8fda026f47", "sha256": "8ad7b55b06e6b7b2f18c5b04a6de85228fda5d99b4cfcabf25c43acf5983194b" }, "downloads": -1, "filename": "aiounfurl-0.2.1.tar.gz", "has_sig": false, "md5_digest": "7e8c515e7a87b60be5d00f8fda026f47", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6175, "upload_time": "2017-01-28T13:00:34", "url": "https://files.pythonhosted.org/packages/d9/9b/69f64791b548495ab97fc794e24b4a0b5046f6de36568cd0174975872ec9/aiounfurl-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "b6e4a207835bc8cb362b4c0a2a393605", "sha256": "769bbcb178a686a3a9207a87d3fae552c5298905a6541705a9e6a5ba56c7c560" }, "downloads": -1, "filename": "aiounfurl-0.2.2.tar.gz", "has_sig": false, "md5_digest": "b6e4a207835bc8cb362b4c0a2a393605", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6181, "upload_time": "2017-01-28T13:33:47", "url": "https://files.pythonhosted.org/packages/33/e2/77da1c0654c1c99f6e1ba9bbd693b723ca8aafa3f053310a9c65c149f9da/aiounfurl-0.2.2.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "3812ce8e95dfb8adc17a3cc9493553d5", "sha256": "0b546faeddaa681bc969d7370b2758bb45408ce7fb8d622a356ef70a16b9bce4" }, "downloads": -1, "filename": "aiounfurl-0.2.4.tar.gz", "has_sig": false, "md5_digest": "3812ce8e95dfb8adc17a3cc9493553d5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7930, "upload_time": "2017-02-12T06:22:41", "url": "https://files.pythonhosted.org/packages/92/e8/5c3bf2ac08a35d924c92bbc2f9e2a4dfcae69757d09b70002283faf4ce0d/aiounfurl-0.2.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3812ce8e95dfb8adc17a3cc9493553d5", "sha256": "0b546faeddaa681bc969d7370b2758bb45408ce7fb8d622a356ef70a16b9bce4" }, "downloads": -1, "filename": "aiounfurl-0.2.4.tar.gz", "has_sig": false, "md5_digest": "3812ce8e95dfb8adc17a3cc9493553d5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7930, "upload_time": "2017-02-12T06:22:41", "url": "https://files.pythonhosted.org/packages/92/e8/5c3bf2ac08a35d924c92bbc2f9e2a4dfcae69757d09b70002283faf4ce0d/aiounfurl-0.2.4.tar.gz" } ] }