{ "info": { "author": "Mikhail Korobov", "author_email": "kmike84@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5" ], "description": "adblockparser\n=============\n\n.. image:: https://img.shields.io/pypi/v/adblockparser.svg\n :target: https://pypi.python.org/pypi/adblockparser\n :alt: PyPI Version\n\n.. image:: https://img.shields.io/pypi/l/adblockparser.svg\n :target: https://github.com/scrapinghub/adblockparser/blob/master/LICENSE.txt\n :alt: License\n\n.. image:: https://img.shields.io/travis/scrapinghub/adblockparser/master.svg\n :target: https://travis-ci.org/scrapinghub/adblockparser\n :alt: Build Status\n\n.. image:: http://codecov.io/github/scrapinghub/adblockparser/coverage.svg?branch=master\n :target: http://codecov.io/github/scrapinghub/adblockparser?branch=master\n :alt: Code Coverage\n\n\n``adblockparser`` is a package for working with `Adblock Plus`_ filter rules.\nIt can parse Adblock Plus filters and match URLs against them.\n\n.. _Adblock Plus: https://adblockplus.org\n\nInstallation\n------------\n\n::\n\n pip install adblockparser\n\nPython 2.7 and Python 3.3+ are supported.\n\nIf you plan to use this library with a large number of filters\ninstalling pyre2_ library is highly recommended: the speedup\nfor a list of default EasyList_ filters can be greater than 1000x.\n\n pip install 're2 >= 0.2.21'\n\nNote that pyre2 library requires C++ re2_ library installed.\nOn OS X you can get it using homebrew (``brew install re2``).\n\n.. _re2: https://github.com/google/re2\n.. _pyre2: https://github.com/axiak/pyre2\n.. _EasyList: https://easylist.to/\n\nUsage\n-----\n\nTo learn about Adblock Plus filter syntax check these links:\n\n* https://adblockplus.org/en/filter-cheatsheet\n* https://adblockplus.org/en/filters\n\n\n1. Get filter rules somewhere: write them manually, read lines from a file\n downloaded from EasyList_, etc.::\n\n >>> raw_rules = [\n ... \"||ads.example.com^\",\n ... \"@@||ads.example.com/notbanner^$~script\",\n ... ]\n\n2. Create ``AdblockRules`` instance from rule strings::\n\n >>> from adblockparser import AdblockRules\n >>> rules = AdblockRules(raw_rules)\n\n3. Use this instance to check if an URL should be blocked or not::\n\n >>> rules.should_block(\"http://ads.example.com\")\n True\n\n Rules with options are ignored unless you pass a dict with options values::\n\n >>> rules.should_block(\"http://ads.example.com/notbanner\")\n True\n >>> rules.should_block(\"http://ads.example.com/notbanner\", {'script': False})\n False\n >>> rules.should_block(\"http://ads.example.com/notbanner\", {'script': True})\n True\n\nConsult with Adblock Plus `docs `__\nfor options description. These options allow to write filters that depend\non some external information not available in URL itself.\n\nPerformance\n-----------\n\nRegex engines\n^^^^^^^^^^^^^\n\n``AdblockRules`` class creates a huge regex to match filters that\ndon't use options. pyre2_ library works better than stdlib's re\nwith such regexes. If you have pyre2_ installed then ``AdblockRules``\nshould work faster, and the speedup can be dramatic - more than 1000x\nin some cases.\n\nSometimes pyre2 prints something like\n``re2/dfa.cc:459: DFA out of memory: prog size 270515 mem 1713850`` to stderr.\nGive re2 library more memory to fix that::\n\n >>> rules = AdblockRules(raw_rules, use_re2=True, max_mem=512*1024*1024) # doctest: +SKIP\n\nMake sure you are using re2 0.2.20 installed from PyPI, it doesn't work.\n\nParsing rules with options\n^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nRules that have options are currently matched in a loop, one-by-one.\nAlso, they are checked for compatibility with options passed by user:\nfor example, if user didn't pass 'script' option (with a ``True`` or ``False``\nvalue), all rules involving ``script`` are discarded.\n\nThis is slow if you have thousands of such rules. To make it work faster,\nexplicitly list all options you want to support in ``AdblockRules`` constructor,\ndisable skipping of unsupported rules, and always pass a dict with all options\nto ``should_block`` method::\n\n >>> rules = AdblockRules(\n ... raw_rules,\n ... supported_options=['script', 'domain'],\n ... skip_unsupported_rules=False\n ... )\n >>> options = {'script': False, 'domain': 'www.mystartpage.com'}\n >>> rules.should_block(\"http://ads.example.com/notbanner\", options)\n False\n\nThis way rules with unsupported options will be filtered once, when\n``AdblockRules`` instance is created.\n\nLimitations\n-----------\n\nThere are some known limitations of the current implementation:\n\n* element hiding rules are ignored;\n* matching URLs against a large number of filters can be slow-ish,\n especially if pyre2_ is not installed and many filter options are enabled;\n* ``match-case`` filter option is not properly supported (it is ignored);\n* ``document`` filter option is not properly supported;\n* rules are not validated *before* parsing, so invalid rules may raise\n inconsistent exceptions or silently work incorrectly.\n\nIt is possible to remove all these limitations. Pull requests are welcome\nif you want to make it happen sooner!\n\nContributing\n------------\n\n* source code: https://github.com/scrapinghub/adblockparser\n* issue tracker: https://github.com/scrapinghub/adblockparser/issues\n\nIn order to run tests, install `tox `_ and type\n\n::\n\n tox\n\nfrom the source checkout.\n\nThe license is MIT.\n\n\nChanges\n=======\n\n0.7 (2016-10-17)\n----------------\n\n* Fixed parsing issue with recent easylist.txt;\n* fixed a link to easylist (thanks https://github.com/limonte).\n\n0.6 (2016-09-10)\n----------------\n\n* Added support for regex rules (thanks https://github.com/mlyko).\n\n0.5 (2016-03-04)\n----------------\n\n* Fixed an issue with blank lines in filter files\n (thanks https://github.com/skrypka);\n* fixed an issue with applying rules with 'domain' option\n when domain doesn't have a dot (e.g. 'localhost');\n* Python 2.6 and Python 3.2 support is dropped;\n adblockparser likely still work in these interpreters,\n but this is no longer checked by tests.\n\n0.4 (2015-03-29)\n----------------\n\n* AdblockRule now caches the compiled regexes (thanks\n https://github.com/mozbugbox);\n* Fixed an issue with \"domain\" option handling\n (thanks https://github.com/nbraem for the bug report and a test case);\n* cleanups and test improvements.\n\n0.3 (2014-07-11)\n----------------\n\n* Switch to setuptools;\n* better ``__repr__`` for ``AdblockRule``;\n* Python 3.4 support is confirmed;\n* testing improvements.\n\n0.2 (2014-03-20)\n----------------\n\nThis release provides much faster `AdblockRules.should_block()` method\nfor rules without options and rules with 'domain' option.\n\n* better combined regex for option-less rules that makes re2 library\n always use DFA without falling back to NFA;\n* an index for rules with domains;\n* ``params`` method arguments are renamed to ``options`` for consistency.\n\n0.1.1 (2014-03-11)\n------------------\n\nBy default ``AdblockRules`` autodetects re2 library and uses\nit if a compatible version is detected.\n\n0.1 (2014-03-03)\n----------------\n\nInitial release.\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/scrapinghub/adblockparser", "keywords": "adblock easylist", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "adblockparser", "package_url": "https://pypi.org/project/adblockparser/", "platform": "", "project_url": "https://pypi.org/project/adblockparser/", "project_urls": { "Homepage": "https://github.com/scrapinghub/adblockparser" }, "release_url": "https://pypi.org/project/adblockparser/0.7/", "requires_dist": null, "requires_python": "", "summary": "Parser for Adblock Plus rules", "version": "0.7" }, "last_serial": 4308994, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "c6779d1621337b824ad70beee2900f3f", "sha256": "77f4ff1d6b81f26b85d3a41722cbd7dee4b88f1a0cc656049fa8c78ad6b649ed" }, "downloads": -1, "filename": "adblockparser-0.1.tar.gz", "has_sig": false, "md5_digest": "c6779d1621337b824ad70beee2900f3f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9515, "upload_time": "2014-02-07T20:36:54", "url": "https://files.pythonhosted.org/packages/83/6e/28e81289be1f6a9a2df024d41a13be40f5cf2d60cd638797677150d5cf7d/adblockparser-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "d2c4876259cc8b2f91065a6e8778d9b5", "sha256": "0adc5d080a22beef95c12e2576ac265ec7a386e52f5d75b2fe3a20b0dcb67ff5" }, "downloads": -1, "filename": "adblockparser-0.1.1.tar.gz", "has_sig": false, "md5_digest": "d2c4876259cc8b2f91065a6e8778d9b5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9946, "upload_time": "2014-02-11T17:24:20", "url": "https://files.pythonhosted.org/packages/81/e0/674b976aa1a4135b5a6a66f1e92d93d2516f8165d8b2871b101cc55ba10b/adblockparser-0.1.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "a5cfbcd288dc9c11b5360b19e730b56e", "sha256": "e3d17fe3da68549f31527cf10ab5d2bf9b522b0b527fbe61be034d14d8d311c4" }, "downloads": -1, "filename": "adblockparser-0.2.tar.gz", "has_sig": false, "md5_digest": "a5cfbcd288dc9c11b5360b19e730b56e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10750, "upload_time": "2014-02-20T01:38:15", "url": "https://files.pythonhosted.org/packages/c0/10/98fe369cc7487d99d9bef25db348bba471858ee4d16df8f8f129ef6da05b/adblockparser-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "a07ffaf5cb282f13a2e3f9b7f7b764e0", "sha256": "b63712134954c8f8b6d8dd6c812d2d93f12f418050009376191154a274a7b693" }, "downloads": -1, "filename": "adblockparser-0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a07ffaf5cb282f13a2e3f9b7f7b764e0", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 11672, "upload_time": "2014-07-11T13:53:04", "url": "https://files.pythonhosted.org/packages/64/f8/b9a667e590b8a185ac962a486668f34637cdc0290faacf67b769a7b7f816/adblockparser-0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "733a92104029777301b7b89592524629", "sha256": "fb64ad236af49cd7a50832e864c9d9bd471568a6758704a7fefad68806e66149" }, "downloads": -1, "filename": "adblockparser-0.3.tar.gz", "has_sig": false, "md5_digest": "733a92104029777301b7b89592524629", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11525, "upload_time": "2014-07-11T13:52:53", "url": "https://files.pythonhosted.org/packages/5d/ed/185ce924ffdbec8e03f67e95cbabecfd7e1a615ccf76aebef5bb40a989c7/adblockparser-0.3.tar.gz" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "6359e1f3d63dbd64ce791ddc33db4592", "sha256": "7157a18d61abf6320545fb873dfb1002ef5ba773c042c82386150fb6b0948130" }, "downloads": -1, "filename": "adblockparser-0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "6359e1f3d63dbd64ce791ddc33db4592", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 12474, "upload_time": "2015-03-28T20:27:19", "url": "https://files.pythonhosted.org/packages/5a/15/a3af99d962f023c995a14caea8a02ceca5e30b1352a947c3539d48ba2309/adblockparser-0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "dca62a7844591711eeb78121a2e99e1f", "sha256": "d1ec5f0530d11a592543184daed3ee9742b1be845f1955569d7645c093de2660" }, "downloads": -1, "filename": "adblockparser-0.4.tar.gz", "has_sig": false, "md5_digest": "dca62a7844591711eeb78121a2e99e1f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12311, "upload_time": "2015-03-28T20:27:10", "url": "https://files.pythonhosted.org/packages/c5/ee/dec8f1a7608e158011b2fc71f5eb96b351216b4bca87fa5f567208dcf568/adblockparser-0.4.tar.gz" } ], "0.5": [ { "comment_text": "", "digests": { "md5": "f7260361014022dc6912fe31c575c296", "sha256": "557f6f1054e2d14217230468830088fa18dee066fd4ae1e4b32f69f8006e6227" }, "downloads": -1, "filename": "adblockparser-0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f7260361014022dc6912fe31c575c296", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 13141, "upload_time": "2016-03-03T22:03:05", "url": "https://files.pythonhosted.org/packages/89/86/2f688317a5f7e1fe9886dd425b2096de8461f458de1441fc01204d4aedd8/adblockparser-0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "469030e3bc0ad2439b087bd11b2ed106", "sha256": "aab8161b8217c46c397f7afa4eb61ecb0007c38a435d0232f2960a7bba25b306" }, "downloads": -1, "filename": "adblockparser-0.5.tar.gz", "has_sig": false, "md5_digest": "469030e3bc0ad2439b087bd11b2ed106", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12708, "upload_time": "2016-03-03T22:02:53", "url": "https://files.pythonhosted.org/packages/99/55/3bf91f615798dc1cdc07da8b8cff86102a160f868ee2b4582e1ccb6d11f8/adblockparser-0.5.tar.gz" } ], "0.6": [ { "comment_text": "", "digests": { "md5": "e0c31b24f664c55926bc0ef9014db6d6", "sha256": "bdcbdcfc4bef13001464f2d532d6d322d9481fe9ce4c48ad6f80ab73cadfbd14" }, "downloads": -1, "filename": "adblockparser-0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e0c31b24f664c55926bc0ef9014db6d6", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 13226, "upload_time": "2016-09-09T22:32:39", "url": "https://files.pythonhosted.org/packages/c9/15/7a995c9a7e5ae32c1d37939977ae79706005a92ef5b1bc203a2195fbd069/adblockparser-0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "eda182718c1c9bf35ddc33d3d9fb3606", "sha256": "9f595b6138d5a684b2f19d2666fa9e13054e8d90e548102765a290e8bb352c46" }, "downloads": -1, "filename": "adblockparser-0.6.tar.gz", "has_sig": false, "md5_digest": "eda182718c1c9bf35ddc33d3d9fb3606", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12793, "upload_time": "2016-09-09T22:32:29", "url": "https://files.pythonhosted.org/packages/dd/fe/235c4757e2c47c414f2f827801358dbff232797a15f4c080fc30b3af76de/adblockparser-0.6.tar.gz" } ], "0.7": [ { "comment_text": "", "digests": { "md5": "d8aeaadd59b56eec0b83a76deece2945", "sha256": "90662a6397d441898e8757553997e0a6cd09c8e56fee4e4ac61b32bda65bad3e" }, "downloads": -1, "filename": "adblockparser-0.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d8aeaadd59b56eec0b83a76deece2945", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 13350, "upload_time": "2016-10-17T08:34:27", "url": "https://files.pythonhosted.org/packages/a8/f3/ad0236ef2872e80b4f890ff3b35401439da0fab0210eb98d8e78e2a40a75/adblockparser-0.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e40e721bcb7a82a2c85794667eec0f01", "sha256": "7a3407ddc31a29e42732bbcb04f3677c6959bffa1ea9d712afd498e0b4d09b22" }, "downloads": -1, "filename": "adblockparser-0.7.tar.gz", "has_sig": false, "md5_digest": "e40e721bcb7a82a2c85794667eec0f01", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12919, "upload_time": "2016-10-17T08:34:17", "url": "https://files.pythonhosted.org/packages/d3/65/76ede5435428f1ffb62092b0d2ced1a94d2084e2777337754767d88feb51/adblockparser-0.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d8aeaadd59b56eec0b83a76deece2945", "sha256": "90662a6397d441898e8757553997e0a6cd09c8e56fee4e4ac61b32bda65bad3e" }, "downloads": -1, "filename": "adblockparser-0.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d8aeaadd59b56eec0b83a76deece2945", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 13350, "upload_time": "2016-10-17T08:34:27", "url": "https://files.pythonhosted.org/packages/a8/f3/ad0236ef2872e80b4f890ff3b35401439da0fab0210eb98d8e78e2a40a75/adblockparser-0.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e40e721bcb7a82a2c85794667eec0f01", "sha256": "7a3407ddc31a29e42732bbcb04f3677c6959bffa1ea9d712afd498e0b4d09b22" }, "downloads": -1, "filename": "adblockparser-0.7.tar.gz", "has_sig": false, "md5_digest": "e40e721bcb7a82a2c85794667eec0f01", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12919, "upload_time": "2016-10-17T08:34:17", "url": "https://files.pythonhosted.org/packages/d3/65/76ede5435428f1ffb62092b0d2ced1a94d2084e2777337754767d88feb51/adblockparser-0.7.tar.gz" } ] }