{ "info": { "author": "Joshua Carp", "author_email": "jm.carp@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Natural Language :: English", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4" ], "description": "RoboBrowser: Your friendly neighborhood web scraper\n===============================================\n\n.. image:: https://badge.fury.io/py/robobrowser.png\n :target: http://badge.fury.io/py/robobrowser\n\n.. image:: https://travis-ci.org/jmcarp/robobrowser.png?branch=master\n :target: https://travis-ci.org/jmcarp/robobrowser\n\n.. image:: https://coveralls.io/repos/jmcarp/robobrowser/badge.png?branch=master\n :target: https://coveralls.io/r/jmcarp/robobrowser\n\nHomepage: `http://robobrowser.readthedocs.org/ `_\n\nRoboBrowser is a simple, Pythonic library for browsing the web without a standalone web browser. RoboBrowser\ncan fetch a page, click on links and buttons, and fill out and submit forms. If you need to interact with web services\nthat don't have APIs, RoboBrowser can help.\n\n.. code-block:: python\n\n import re\n from robobrowser import RoboBrowser\n\n # Browse to Genius\n browser = RoboBrowser(history=True)\n browser.open('http://genius.com/')\n\n # Search for Porcupine Tree\n form = browser.get_form(action='/search')\n form # \n form['q'].value = 'porcupine tree'\n browser.submit_form(form)\n\n # Look up the first song\n songs = browser.select('.song_link')\n browser.follow_link(songs[0])\n lyrics = browser.select('.lyrics')\n lyrics[0].text # \\nHear the sound of music ...\n\n # Back to results page\n browser.back()\n\n # Look up my favorite song\n song_link = browser.get_link('trains')\n browser.follow_link(song_link)\n\n # Can also search HTML using regex patterns\n lyrics = browser.find(class_=re.compile(r'\\blyrics\\b'))\n lyrics.text # \\nTrain set and match spied under the blind...\n\nRoboBrowser combines the best of two excellent Python libraries:\n`Requests `_ and\n`BeautifulSoup `_.\nRoboBrowser represents browser sessions using Requests and HTML responses\nusing BeautifulSoup, transparently exposing methods of both libraries:\n\n.. code-block:: python\n\n import re\n from robobrowser import RoboBrowser\n\n browser = RoboBrowser(user_agent='a python robot')\n browser.open('https://github.com/')\n\n # Inspect the browser session\n browser.session.cookies['_gh_sess'] # BAh7Bzo...\n browser.session.headers['User-Agent'] # a python robot\n\n # Search the parsed HTML\n browser.select('div.teaser-icon') # [
\n # \n #
,\n # ...\n browser.find(class_=re.compile(r'column', re.I)) #
\n #
\n # \n # ...\n\nYou can also pass a custom `Session` instance for lower-level configuration:\n\n.. code-block:: python\n\n from requests import Session\n from robobrowser import RoboBrowser\n\n session = Session()\n session.verify = False # Skip SSL verification\n session.proxies = {'http': 'http://custom.proxy.com/'} # Set default proxies\n browser = RoboBrowser(session=session)\n\nRoboBrowser also includes tools for working with forms, inspired by\n`WebTest `_ and `Mechanize `_.\n\n.. code-block:: python\n\n from robobrowser import RoboBrowser\n\n browser = RoboBrowser()\n browser.open('http://twitter.com')\n\n # Get the signup form\n signup_form = browser.get_form(class_='signup')\n signup_form # \n form['vehicle'] # \n\n # Checked values can be get and set like lists\n form['vehicle'].options # [u'Bike', u'Car']\n form['vehicle'].value # []\n form['vehicle'].value = ['Bike']\n form['vehicle'].value = ['Bike', 'Car']\n\n # Values can also be set using input labels\n form['vehicle'].labels # [u'I have a bike', u'I have a car \\r\\n']\n form['vehicle'].value = ['I have a bike']\n form['vehicle'].value # [u'Bike']\n\n # Only values that correspond to checkbox values or labels can be set;\n # this will raise a `ValueError`\n form['vehicle'].value = ['Hot Dogs']\n\nUploading files:\n\n.. code-block:: python\n\n from robobrowser import RoboBrowser\n\n # Browse to a page with an upload form\n browser = RoboBrowser()\n browser.open('http://cgi-lib.berkeley.edu/ex/fup.html')\n\n # Find the form\n upload_form = browser.get_form()\n upload_form # \n\n # Choose a file to upload\n upload_form['upfile'] # \n upload_form['upfile'].value = open('path/to/file.txt', 'r')\n\n # Submit\n browser.submit(upload_form)\n\nBy default, creating a browser instantiates a new requests `Session`. \n\nRequirements\n------------\n\n- Python >= 2.6 or >= 3.3\n\nLicense\n-------\n\nMIT licensed. See the bundled `LICENSE `_ file for more details.\n\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jmcarp/robobrowser", "keywords": "robobrowser", "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "robobrowser", "package_url": "https://pypi.org/project/robobrowser/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/robobrowser/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/jmcarp/robobrowser" }, "release_url": "https://pypi.org/project/robobrowser/0.5.3/", "requires_dist": null, "requires_python": null, "summary": "Your friendly neighborhood web scraper", "version": "0.5.3" }, "last_serial": 1582259, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "1cdba35236cbd7c9fcbb9bfe12d12197", "sha256": "cf542b0703ff23f00d872844b9c29c0180f7d29ae2e547d80d8cf1bda321474c" }, "downloads": -1, "filename": "robobrowser-0.1.0.tar.gz", "has_sig": false, "md5_digest": "1cdba35236cbd7c9fcbb9bfe12d12197", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10355, "upload_time": "2014-02-08T23:25:28", "url": "https://files.pythonhosted.org/packages/3a/2a/f509c684f1b172fa2a0a4e8ddfb94af50e1698c01f8a696d4605c31b99e9/robobrowser-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "51142589ed60361b8ff2b8be225db67a", "sha256": "13810a168c42ffb6959d0edbc0ad3653234ea7dcea11ef736ae19186c6345859" }, "downloads": -1, "filename": "robobrowser-0.1.1.tar.gz", "has_sig": false, "md5_digest": "51142589ed60361b8ff2b8be225db67a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16177, "upload_time": "2014-02-11T21:23:17", "url": "https://files.pythonhosted.org/packages/3e/96/69c70e62c33c21a9cd5091d9fb677827fef2f29f759a7ed879106cfbe9e0/robobrowser-0.1.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "df889c2a842b7186d85207eee565cbd1", "sha256": "3597c8eb7e6ed65467bc522f0087ac943aa662e046472a837751770762fae21a" }, "downloads": -1, "filename": "robobrowser-0.2.tar.gz", "has_sig": false, "md5_digest": "df889c2a842b7186d85207eee565cbd1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18036, "upload_time": "2014-03-10T01:19:45", "url": "https://files.pythonhosted.org/packages/d5/9e/3901685b95c49a20145c254b73f11940b90198946d4d8f510226aafb1958/robobrowser-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "67e848cba02bbca6f567d7fcd6208a06", "sha256": "c86d06d80e6efaa942f556db895a44226d1491c7b1abc04a3ae399f0efeb038f" }, "downloads": -1, "filename": "robobrowser-0.3.tar.gz", "has_sig": false, "md5_digest": "67e848cba02bbca6f567d7fcd6208a06", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18943, "upload_time": "2014-05-04T01:06:30", "url": "https://files.pythonhosted.org/packages/88/53/d7772603eabb2c1afad593531abfccb272a9df1371530b78b1b011840946/robobrowser-0.3.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "e5fa952bb9b347ee4b0dd1bb4aba2cc4", "sha256": "638a7c59c4776ca0da07c2cdd36cd67b744575970c4d31cde5f44a64ca1985b6" }, "downloads": -1, "filename": "robobrowser-0.3.1.tar.gz", "has_sig": false, "md5_digest": "e5fa952bb9b347ee4b0dd1bb4aba2cc4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19203, "upload_time": "2014-07-05T21:12:33", "url": "https://files.pythonhosted.org/packages/47/47/c6013b0a1e2d67627a7aee897076792108e5a1da613a3cb7c53452577a22/robobrowser-0.3.1.tar.gz" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "8c82cfc6b7844c2a8585308aac580d33", "sha256": "f6859aa0ce85cf9879fb274f28fdcd3570fdf5f73c78122bdfef821e5d51ce7a" }, "downloads": -1, "filename": "robobrowser-0.3.2.tar.gz", "has_sig": false, "md5_digest": "8c82cfc6b7844c2a8585308aac580d33", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20186, "upload_time": "2014-07-20T01:26:22", "url": "https://files.pythonhosted.org/packages/96/24/f0697194b7c36d1f03bbc3ef18eb25d852a89bc67dcb9111db1749aafb44/robobrowser-0.3.2.tar.gz" } ], "0.3.3": [ { "comment_text": "", "digests": { "md5": "ab437cc6ea7dc56bc91b8b441f8c2b6b", "sha256": "a1136269c5c6a0af549aa7531a797479996d30dc07b371854bf589e49005e9cc" }, "downloads": -1, "filename": "robobrowser-0.3.3.tar.gz", "has_sig": false, "md5_digest": "ab437cc6ea7dc56bc91b8b441f8c2b6b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20287, "upload_time": "2014-07-22T01:50:04", "url": "https://files.pythonhosted.org/packages/ca/e2/3a28934357b5d400517bcbbe71b33780340cffaa3cf47ef5dc3431bd049e/robobrowser-0.3.3.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "f441ea1db1b66be3544d11d549e1b677", "sha256": "018504af8d9e4206ad78c0c7fd885d740705ca86f9e06d0aa5c55797e97810a6" }, "downloads": -1, "filename": "robobrowser-0.4.0.tar.gz", "has_sig": false, "md5_digest": "f441ea1db1b66be3544d11d549e1b677", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21218, "upload_time": "2014-07-24T19:42:12", "url": "https://files.pythonhosted.org/packages/71/e3/a3f0cb763730f95200792ef2c9469ba2b779015d3219beb80b5e5a3306c8/robobrowser-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "ed17c054f3c4a728602a49b583cf9fda", "sha256": "706475f199c35a172cc9047d9105f58382e46d8c0ab9bc7c41975c4d38880b79" }, "downloads": -1, "filename": "robobrowser-0.4.1.tar.gz", "has_sig": false, "md5_digest": "ed17c054f3c4a728602a49b583cf9fda", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21197, "upload_time": "2014-07-27T20:56:06", "url": "https://files.pythonhosted.org/packages/bf/1c/50ace9f16b59c19b645f7baa3d74076da95b483064027b3c96f83ff8411b/robobrowser-0.4.1.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "3c8ed147515609a561c6c3b7ba85a4f3", "sha256": "e76ba6900970ce1facfc86f0f9e4b1c4c10d9f56015e8f4c5b4edd77c230fe60" }, "downloads": -1, "filename": "robobrowser-0.5.0.tar.gz", "has_sig": false, "md5_digest": "3c8ed147515609a561c6c3b7ba85a4f3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21396, "upload_time": "2014-07-28T23:21:37", "url": "https://files.pythonhosted.org/packages/b3/b2/bc52d26e6d74b5a957626f97f4110d315a2fc949532f033771a8b61b175a/robobrowser-0.5.0.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "83a63db75efa904b8a40502b14918c34", "sha256": "07e83255103769f6e371ea61e248c118cae4071bb4461bbc5b2eb6c8fd5fa793" }, "downloads": -1, "filename": "robobrowser-0.5.1.tar.gz", "has_sig": false, "md5_digest": "83a63db75efa904b8a40502b14918c34", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21635, "upload_time": "2014-09-15T02:38:27", "url": "https://files.pythonhosted.org/packages/1f/0c/b10dad80f9bae15b8b90c80b72205319092dfc785f251f8788fa60648973/robobrowser-0.5.1.tar.gz" } ], "0.5.2": [ { "comment_text": "", "digests": { "md5": "7d8b49461b77e5f3e77c1dabdf175cfe", "sha256": "11a42b8dc28bb7e8c338325d6ca95e04f4f06362fbb8250b335669730bb3ca42" }, "downloads": -1, "filename": "robobrowser-0.5.2.tar.gz", "has_sig": false, "md5_digest": "7d8b49461b77e5f3e77c1dabdf175cfe", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21789, "upload_time": "2015-04-18T20:34:28", "url": "https://files.pythonhosted.org/packages/b6/cf/54d4244de9b40e4dce8cca001eadda2af16e9ac19432b307d55591c64f20/robobrowser-0.5.2.tar.gz" } ], "0.5.3": [ { "comment_text": "", "digests": { "md5": "333ad401f4a0b320fa873c78bc5fb64d", "sha256": "31219acab41ca68adce928e5c1e04acebba4ceabeb447b9c5e408d7b30fee983" }, "downloads": -1, "filename": "robobrowser-0.5.3.tar.gz", "has_sig": false, "md5_digest": "333ad401f4a0b320fa873c78bc5fb64d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22040, "upload_time": "2015-06-07T19:47:05", "url": "https://files.pythonhosted.org/packages/f3/53/fd527e78fe8bdf7ec24b3b821c88c2dcbbe63d120c838f80f12ed0a0fac6/robobrowser-0.5.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "333ad401f4a0b320fa873c78bc5fb64d", "sha256": "31219acab41ca68adce928e5c1e04acebba4ceabeb447b9c5e408d7b30fee983" }, "downloads": -1, "filename": "robobrowser-0.5.3.tar.gz", "has_sig": false, "md5_digest": "333ad401f4a0b320fa873c78bc5fb64d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22040, "upload_time": "2015-06-07T19:47:05", "url": "https://files.pythonhosted.org/packages/f3/53/fd527e78fe8bdf7ec24b3b821c88c2dcbbe63d120c838f80f12ed0a0fac6/robobrowser-0.5.3.tar.gz" } ] }