{ "info": { "author": "Mathieu Pasquet", "author_email": "kiorky@cryptelium.net", "bugtrack_url": null, "classifiers": [], "description": "Introduction\n=============\nThis Yet-Another-Mechanize implementation aims to give the developper those new\nfeatures:\n\n - It can be proxified\n - It does proxy balancing\n - It fakes user agent ``by default``\n - It does not handle robots ``by default``\n - There is a 'real\" modification which uses an underlying moz repl server to\n control a distance firefox instance\n\nIt uses sys.prefix/etc/config.ini with a part [collective.anonymousbrowser] for its settings::\n\n [collective.anonymousbrowser]\n proxies=\n ; for a mozrepl server\n host = localhost\n port = 4242\n firefox = /path/To/Firefox\n ff-profile = /path/to/FFprofile\n\nThis file is generated at the first run without proxies. It s your own to feed it with some open proxies.\n\nOf course, it can take another configuration file, please see the __init__ method.\n\nCredits\n======================================\n|makinacom|_\n\n* `Planet Makina Corpus `_\n* `Contact us `_\n\n.. |makinacom| image:: http://depot.makina-corpus.org/public/logo.gif\n.. _makinacom: http://www.makina-corpus.com\n\n\n\nTODO\n=====\n- lxml integration, maybe steal z3c.etestbrowser\n\n\nTests and Handbook\n======================\n\nFirst, we need to instantiate the sources where we come from::\n\n >>> from collective.anonymousbrowser.browser import Browser, FF2_USERAGENT\n\n \nUser Agent\n------------\nOh, my god, we have a brand new user agent by default::\n\n >>> br = Browser()\n >>> br.open('http://localhost:45678')\n >>> FF2_USERAGENT in br.contents\n True \n >>> br2 = Browser('http://localhost:45678')\n >>> FF2_USERAGENT in br2.contents\n True\n\n\nProxy mode\n------------\nBut, we want to be anonymous, and we ll set a proxy\nTo define those proxies, just just a config.ini file like::\n\n [collective.anonymousbrowser]\n proxies =\n host1:port\n host2:port\n\nWhen the browser has many proxies defined, it will circly through those ones.\nBut, it will not use the same host indefinitivly, just set the ``proxy_max_use`` argument::\n\n >>> from StringIO import StringIO\n >>> from tempfile import mkstemp\n >>> __, config = mkstemp()\n >>> open(config, 'w').write(\"\"\"[collective.anonymousbrowser]\n ... proxies =\n ... 127.0.0.1:45675\n ... 127.0.0.1:45676\n ... 127.0.0.1:45677\n ... 127.0.0.1:45678\n ... 127.0.0.1:45679\n ... \"\"\")\n >>> b = Browser(config=config, proxy_max_use=3)\n >>> b._config._sections\n {'collective.anonymousbrowser': {'__name__': 'collective.anonymousbrowser', 'proxies': '\\n127.0.0.1:45675\\n127.0.0.1:45676\\n127.0.0.1:45677\\n127.0.0.1:45678\\n127.0.0.1:45679'}}\n >>> b.proxies\n ['127.0.0.1:45675', '127.0.0.1:45676', '127.0.0.1:45677', '127.0.0.1:45678', '127.0.0.1:45679']\n >>> b.proxified\n True\n >>> b.open('http://localhost:45678')\n >>> 'Host: localhost:45678' in b.contents\n True\n >>> b._lastproxy['count'] == 1 and b._lastproxy['proxy'] in [0,1,2,3,4]\n True\n\nWe can have a normal unproxified browser too ::\n\n >>> b1 = Browser(proxify=False)\n >>> b1.proxified\n False\n\nNext thing to verify is that we have our pseudo-random loop running\nFirst thing is we will choose 2 times the 2nd proxy, then the third\nAnd of course, we will set the mocker to change the proxy at each row.::\n\n >>> import mocker\n >>> import random\n >>> mocked = mocker.Mocker()\n >>> custom_random_int = mocked.replace('random.randint')\n >>> custom_random_int(0, 4)\n >> mocked.result(2)\n >>> custom_random_int(0,1)\n >> mocked.result(0)\n >>> custom_random_int(0, 4)\n >> mocked.result(2)\n >>> custom_random_int(0,1)\n >> mocked.result(0)\n >>> custom_random_int(0, 4)\n >> mocked.result(2)\n >>> custom_random_int(0,1)\n >> mocked.result(0)\n >>> custom_random_int(0, 4)\n >> mocked.result(3)\n >>> custom_random_int(0,1)\n >> mocked.result(0)\n >>> custom_random_int(0, 4)\n >> mocked.result(4)\n >>> custom_random_int(0,1)\n >> mocked.result(0)\n >>> custom_random_int(0, 4)\n >> mocked.result(2)\n >>> custom_random_int(0,1)\n >> mocked.result(0)\n >>> custom_random_int(0, 4)\n >> mocked.result(1)\n >>> custom_random_int(0,1)\n >> mocked.result(0)\n >>> mocked.replay()\n >>> b = Browser('http://localhost:45678', config=config, proxy_max_use=3)\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 1, 'proxy': 2}\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 2, 'proxy': 2}\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 3, 'proxy': 2}\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 1, 'proxy': 0}\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 1, 'proxy': 3}\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 1, 'proxy': 4}\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 1, 'proxy': 2}\n >>> b.open('http://localhost:45678')\n >>> b._lastproxy\n {'count': 1, 'proxy': 1}\n >>> mocked.restore()\n\nIf the proxies are dead, we remove them from the list::\n\n >>> __, config = mkstemp()\n >>> open(config, 'w').write(\"\"\"[collective.anonymousbrowser]\n ... proxies =\n ... 127.0.0.1:35675\n ... 127.0.0.1:35676\n ... 127.0.0.1:35677\n ... 127.0.0.1:45678\n ... \"\"\")\n >>> mybrowser = Browser(config=config, proxy_max_use=3)\n >>> mybrowser.proxies\n ['127.0.0.1:35675', '127.0.0.1:35676', '127.0.0.1:35677', '127.0.0.1:45678']\n >>> mybrowser.open('http://localhost:45678')\n >>> mybrowser.proxies\n ['127.0.0.1:45678']\n >>> mybrowser.proxies = ['127.0.0.1:34785']\n >>> mybrowser.open('http://localhost:45678')\n Traceback (most recent call last):\n ...\n Exception: There are no valid proxies left\n\n\nThe loop is recursion protected. If we return always the same host, so the chooser cannot choose anything else.\nIt will loop until it crashes or it handle the recursion::\n\n >>> def randomint(a,b):\n ... return 2\n >>> import random; random.randint = randomint\n >>> b2 = Browser(config=config, proxy_max_use=3)\n >>> b2.proxy_max_use\n 3\n >>> b2._lastproxy['count']\n 0\n >>> b2.chooseProxy()\n '...\n >>> b2._lastproxy['count']\n 1\n >>> b2.chooseProxy()\n '...\n >>> b2._lastproxy['count']\n 2\n >>> b2.chooseProxy()\n '...\n >>> b2._lastproxy['count']\n 3\n >>> b2.chooseProxy()\n '...\n >>> b2.chooseProxy()\n Ho, seems we got the max wills to choose, something has gone wrong\n '127.0.0.1:35675'\n\n\n\n.. vim:set ft=doctest:\n\nReal Browser implementation throught mozrepl\n=============================================\n\nTODO:\n\n - Handle configuration with mozrunner for:\n\n - user agent faking\n - proxies management\n\nFirst, we need to instantiate the sources where we come from::\n\n >>> from collective.anonymousbrowser.real import *\n\nIn the section [collective.anonymousbrowser] of your configuration file you\ncan add those parameters:\n\n - ``host`` : host of firefox mozrepl instance\n - ``port`` : port of firefox mozrepl instance\n - ``firefox`` : path to the firefox binary\n - ``firefox-profile`` : path to the firefox profile to use\n\n\nStart to use it on our little http server::\n\n >>> b = Browser('http://localhost:45675')\n >>> b.contents\n '...
...localhost:45675...
...'\n\n >>> b.open('http://localhost:45675')\n >>> b.contents\n '...
...localhost:45675...
...'\n\nKill any launched firefox from the browser instance with its configuration\nsettings::\n\n >>> b.stop_ff()\n >>> b.start_ff()\n \n >>> b.restart_ff()\n \n\nCleanup::\n\n >>> b.stop_ff()\n\n.. vim:set ft=doctest:\n\n\nHISTORY\n========\n0.10-<0.11\n-----------\n- bugfix for 0.9\n\n0.9\n----\n- Fix binary distributions, now with a sample decorator, mozrunner executes\n its commands in firefox directories\n\n0.8\n-----\n- bugfix for js execution\n\n0.7\n-----\n- bugfix: firefox is started when you call open... its better.\n \n0.6\n-----\n- doc + bugfixes\n- use of testrunner to handle firefox instance\n- robustify the proxy code\n- add tests\n\n0.4\n-----\n- doc + bugfixes\n\n0.3\n-----\n- adding error message\n\n0.2\n-----\n- Adding proxy fallback facility\n\n0.1\n-----\n- Initial release", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://svn.plone.org/svn/collective/collective.anonymousbrowser/trunk", "keywords": null, "license": "GPL", "maintainer": null, "maintainer_email": null, "name": "collective.anonymousbrowser", "package_url": "https://pypi.org/project/collective.anonymousbrowser/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/collective.anonymousbrowser/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://svn.plone.org/svn/collective/collective.anonymousbrowser/trunk" }, "release_url": "https://pypi.org/project/collective.anonymousbrowser/0.11/", "requires_dist": null, "requires_python": null, "summary": "A zope.testbrowser extension with useragent faking and proxy abilities by Makina Corpus", "version": "0.11" }, "last_serial": 787642, "releases": { "0.11": [ { "comment_text": "", "digests": { "md5": "25796a3db8c6032add7ce309019801a9", "sha256": "7bdb3e830e579bae31abf235b718fbd509dde5282c205001eeb0b857f55e7dd4" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.11.zip", "has_sig": false, "md5_digest": "25796a3db8c6032add7ce309019801a9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21392, "upload_time": "2010-04-21T14:10:20", "url": "https://files.pythonhosted.org/packages/43/81/8219cef6ca8ec60d7b463e0ec8409bc10fe3bfbf3e1b3bf871ffe226ebd2/collective.anonymousbrowser-0.11.zip" } ], "0.11dev-r115879": [], "0.11dev-r115880": [], "0.11dev-r82630": [ { "comment_text": "", "digests": { "md5": "bc68a054a80170939eb6575e13cd7972", "sha256": "bd9f7c40ff612af554ba256a65ec1e8a78881a20d5968b4b2535413fddf9d948" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.11dev-r82630.tar.gz", "has_sig": false, "md5_digest": "bc68a054a80170939eb6575e13cd7972", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12954, "upload_time": "2009-03-17T21:25:27", "url": "https://files.pythonhosted.org/packages/46/3c/62f50e42ceace867bf3287debeff2f477a6e8aa4c95b5c6b2fedcd6f1552/collective.anonymousbrowser-0.11dev-r82630.tar.gz" } ], "0.1dev-r73299": [], "0.1dev-r73324": [ { "comment_text": "", "digests": { "md5": "601d122eb43584833f6f208d4b868fb5", "sha256": "2aa4ef434103e3c73c238867830c7675c7e935fc3ed87424ae3a3270ee01f8bc" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.1dev-r73324.zip", "has_sig": false, "md5_digest": "601d122eb43584833f6f208d4b868fb5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13835, "upload_time": "2008-10-11T18:28:04", "url": "https://files.pythonhosted.org/packages/ae/6c/bba4030a3d04107cc135d45fb687e555c9f0f19eac86b336e7b816779160/collective.anonymousbrowser-0.1dev-r73324.zip" } ], "0.2dev-r73407": [ { "comment_text": "", "digests": { "md5": "df4577d68e3803b0be00041f3561ca9b", "sha256": "3c6b360f27e827e04bcb52236049788a94cde046ca4edf449ade1713e2500921" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.2dev-r73407.zip", "has_sig": false, "md5_digest": "df4577d68e3803b0be00041f3561ca9b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14349, "upload_time": "2008-10-12T00:07:18", "url": "https://files.pythonhosted.org/packages/04/05/a0032a262c7f369611974d0015d085f6d3554375102cf07fcc1adb70f97d/collective.anonymousbrowser-0.2dev-r73407.zip" } ], "0.2dev-r73410": [ { "comment_text": "", "digests": { "md5": "434db4d03a17679c8ef6ae57ec2c4357", "sha256": "9e7a577afe98222303012712457e007abaae18b25975955dc6425c318381fbcb" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.2dev-r73410.zip", "has_sig": false, "md5_digest": "434db4d03a17679c8ef6ae57ec2c4357", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14625, "upload_time": "2008-10-12T00:13:32", "url": "https://files.pythonhosted.org/packages/fa/14/485c5bc35c90de3e9b7e4b4d50fe67abcce274a84e356bf77a775cdae3ed/collective.anonymousbrowser-0.2dev-r73410.zip" } ], "0.3dev-r73429": [ { "comment_text": "", "digests": { "md5": "1d18824a8c9386bad4b5b8765f4512f8", "sha256": "ca5b2829655d5b87628c032a68dbf59de3415434d91acde502ea1bfef1595994" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.3dev-r73429.zip", "has_sig": false, "md5_digest": "1d18824a8c9386bad4b5b8765f4512f8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14623, "upload_time": "2008-10-12T12:05:26", "url": "https://files.pythonhosted.org/packages/29/3f/f1cac2949b13f91c5f2ee251a33e1c7bc2b96d0f7cbe220eff414f1293ba/collective.anonymousbrowser-0.3dev-r73429.zip" } ], "0.4dev-r73660": [ { "comment_text": "", "digests": { "md5": "19dbc406e2f924e6eee4ab9b2afab4b0", "sha256": "c8b89743b79aafc37a7f4fe67707b5679069589d71dc624cea16d17d657cefda" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.4dev-r73660.zip", "has_sig": false, "md5_digest": "19dbc406e2f924e6eee4ab9b2afab4b0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15058, "upload_time": "2008-10-14T23:40:47", "url": "https://files.pythonhosted.org/packages/9a/dc/6e075306184d6d1e2c3a52dd39afbae59987ed231af467a23915040c570c/collective.anonymousbrowser-0.4dev-r73660.zip" } ], "0.5dev-r79038": [ { "comment_text": "", "digests": { "md5": "3df8f38041eb3445137e521ee61d535b", "sha256": "68f2ca3e9fbaf12b3106bda24b94ac0faa1676c0247917f5fe3a55fc0b6ebbf1" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.5dev-r79038.tar.gz", "has_sig": false, "md5_digest": "3df8f38041eb3445137e521ee61d535b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8851, "upload_time": "2009-01-15T10:07:20", "url": "https://files.pythonhosted.org/packages/4a/53/fcf51a40de4eae3e3248ff5174717764c187c85df30b87908d0cc79156f2/collective.anonymousbrowser-0.5dev-r79038.tar.gz" } ], "0.6dev-r82290": [ { "comment_text": "", "digests": { "md5": "3c0300f19c795d3dcfa25fe1d48e1f3f", "sha256": "ec7281e49a783661ea5b30c18fa1f403fa304ff6b803586601e6e8dba6a79076" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.6dev-r82290.tar.gz", "has_sig": false, "md5_digest": "3c0300f19c795d3dcfa25fe1d48e1f3f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12027, "upload_time": "2009-03-13T23:10:12", "url": "https://files.pythonhosted.org/packages/0b/46/01a7a5296e121dd0544ed226e7bcd515371bd70164331b6803824ed54609/collective.anonymousbrowser-0.6dev-r82290.tar.gz" } ], "0.6dev-r82291": [], "0.7dev-r82296": [ { "comment_text": "", "digests": { "md5": "ab5a66b14ff07bcf6d17959b1b7a1b97", "sha256": "1c11b0c268f3fec51cd08b92daac1c580c40f375043f93800b507fe8add18f32" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.7dev-r82296.tar.gz", "has_sig": false, "md5_digest": "ab5a66b14ff07bcf6d17959b1b7a1b97", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12538, "upload_time": "2009-03-14T01:48:06", "url": "https://files.pythonhosted.org/packages/fd/08/7e00267f5e8a227f20b3c0761d5f25ee0fb5dbd038f300d1c16a54d13d53/collective.anonymousbrowser-0.7dev-r82296.tar.gz" } ], "0.8dev-r82578": [ { "comment_text": "", "digests": { "md5": "7627e8649473d11f2e161f1a84feba5b", "sha256": "408f48b01e137ea71ee808445daeb40bc5bc4d746e7c745be1570861bebd3db2" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.8dev-r82578.tar.gz", "has_sig": false, "md5_digest": "7627e8649473d11f2e161f1a84feba5b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12566, "upload_time": "2009-03-17T14:23:37", "url": "https://files.pythonhosted.org/packages/7e/11/51d09b7518658d02b2904301a3ce831a4bc2ddee8975ea320c44c28fa1ed/collective.anonymousbrowser-0.8dev-r82578.tar.gz" } ], "0.9dev-r82608": [ { "comment_text": "", "digests": { "md5": "f637df2f45bfdc362b25f67310d99552", "sha256": "bf12851050ea019c6b1c61aac89adfa9c2fd4cc60cdf121a33d7c9deddf3566e" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.9dev-r82608.tar.gz", "has_sig": false, "md5_digest": "f637df2f45bfdc362b25f67310d99552", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12804, "upload_time": "2009-03-17T17:15:01", "url": "https://files.pythonhosted.org/packages/29/be/c8a88d8f13e23de561f0dfe53b409229fd016288bad2234f58ceeaef1a10/collective.anonymousbrowser-0.9dev-r82608.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "25796a3db8c6032add7ce309019801a9", "sha256": "7bdb3e830e579bae31abf235b718fbd509dde5282c205001eeb0b857f55e7dd4" }, "downloads": -1, "filename": "collective.anonymousbrowser-0.11.zip", "has_sig": false, "md5_digest": "25796a3db8c6032add7ce309019801a9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21392, "upload_time": "2010-04-21T14:10:20", "url": "https://files.pythonhosted.org/packages/43/81/8219cef6ca8ec60d7b463e0ec8409bc10fe3bfbf3e1b3bf871ffe226ebd2/collective.anonymousbrowser-0.11.zip" } ] }