{ "info": { "author": "eyeo GmbH", "author_email": "info@adblockplus.org", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "python-abp\n==========\n\nThis repository contains a library for working with Adblock Plus filter lists,\na script for rendering diffs between filter lists, and the script that is used\nfor building Adblock Plus filter lists from the form in which they are authored\ninto the format suitable for consumption by the adblocking software (aka\nrendering).\n\n.. contents::\n\n\nInstallation\n------------\n\nPrerequisites:\n\n* Linux, Mac OS X or Windows (any modern Unix should work too),\n* Python (2.7 or 3.5+),\n* pip.\n\nTo install::\n\n $ pip install --upgrade python-abp\n\n\nRendering of filter lists\n-------------------------\n\nThe filter lists are originally authored in relatively smaller parts focused\non particular types of filters, related to a specific topic or relevant for a\nparticular geographical area.\nWe call these parts *filter list fragments* (or just *fragments*) to\ndistinguish them from full filter lists that are consumed by the adblocking\nsoftware such as Adblock Plus.\n\nRendering is a process that combines filter list fragments into a filter list.\nIt starts with one fragment that can include other ones and so forth.\nThe produced filter list is marked with a `version and a timestamp `_.\n\nPython-abp contains a script that can do this called ``flrender``::\n\n $ flrender fragment.txt filterlist.txt\n\n\nThis will take the top level fragment in ``fragment.txt``, render it and save it\ninto ``filterlist.txt``.\n\nThe ``flrender`` script can also be used by only specifying ``fragment.txt``::\n\n $ flrender fragment.txt\n\n\nin which case the rendering result will be sent to ``stdout``. Moreover, when\nit's run with no positional arguments::\n\n $ flrender\n\n\nit will read from ``stdin`` and send the results to ``stdout``.\n\nFragments might reference other fragments that should be included into them.\nThe references come in two forms: http(s) includes and local includes::\n\n %include http://www.server.org/dir/list.txt%\n %include easylist:easylist/easylist_general_block.txt%\n\n\nThe http include contains a URL that will be fetched and inserted at the point\nof reference.\nThe local include contains a path inside the easylist repository.\n``flrender`` needs to be able to find a copy of the repository on the local\nfilesystem. We use ``-i`` option to point it to to the right directory::\n\n $ flrender -i easylist=/home/abc/easylist input.txt output.txt\n\n\nNow the local include referenced above will be resolved to:\n``/home/abc/easylist/easylist/easylist_general_block.txt``\nand the fragment will be loaded from this file.\n\nDirectories that contain filter list fragments that are used during rendering\nare called sources.\nThey are normally working copies of the repositories that contain filter list\nfragments.\nEach source is identified by a name: that's the part that comes before \":\" in\nthe include instruction and it should be the same as what comes before \"=\" in\nthe ``-i`` option.\n\nCommonly used sources have generally accepted names. For example the main\nEasyList repository is referred to as ``easylist``.\nIf you don't know all the source names that are needed to render some list,\njust run ``flrender`` and it will report what it's missing::\n\n $ flrender easylist.txt output/easylist.txt\n Unknown source: 'easylist' when including 'easylist:easylist/easylist_gener\n al_block.txt' from 'easylist.txt'\n\n\nYou can clone the necessary repositories to a local directory and add ``-i``\noptions accordingly.\n\n\nGenerating diffs\n----------------\n\nA diff allows a client running ad blocking software such as Adblock Plus to\nupdate the filter lists incrementally, instead of downloading a new copy of a\nfull list during each update. This is meant to lessen the amount of resources\nused when updating filter lists (e.g. network data, memory usage, battery\nconsumption, etc.), allowing clients to update their lists more frequently\nusing less resources.\n\npython-abp contains a script called ``fldiff`` that will find the diff between\nthe latest filter list, and any number of previous filter lists::\n\n $ fldiff -o diffs/easylist/ easylist.txt archive/*\n\n\nwhere ``-o diffs/easylist/`` is the (optional) output directory where the diffs\nshould be written, ``easylist.txt`` is the most recent version of the filter\nlist, and ``archive/*`` is the directory where all the archived filter lists are.\nWhen called like this, the shell should automatically expand the ``archive/*``\ndirectory, giving the script each of the filenames separately.\n\nIn the above example, the output of each archived ``list[version].txt`` will be\nwritten to ``diffs/diff[version].txt``. If the output argument is omitted, the\ndiffs will be written to the current directory.\n\nThe script produces three types of lines, as specified in the `technical\nspecification `_:\n\n\n* Special comments of the form ``! :[ ]``\n* Added filters of the form ``+ ``\n* Removed filters of the form ``- ``\n\n\nLibrary API\n-----------\n\npython-abp can also be used as a library for parsing filter lists. For example\nto read a filter list (we use Python 3 syntax here but the API is the same):\n\n.. code-block:: python\n\n from abp.filters import parse_filterlist\n\n with open('filterlist.txt') as filterlist:\n for line in parse_filterlist(filterlist):\n print(line)\n\n\nIf ``filterlist.txt`` contains this filter list::\n\n [Adblock Plus 2.0]\n ! Title: Example list\n\n abc.com,cdf.com##div#ad1\n abc.com/ad$image\n @@/abc\\.com/\n\n\nthe output will look something like:\n\n.. code-block:: python\n\n Header(version='Adblock Plus 2.0')\n Metadata(key='Title', value='Example list')\n EmptyLine()\n Filter(text='abc.com,cdf.com##div#ad1', selector={'type': 'css', 'value': 'div#ad1'}, action='hide', options=[('domain', [('abc .com', True), ('cdf.com', True)])])\n Filter(text='abc.com/ad$image', selector={'type': 'url-pattern', 'value': 'abc.com/ad'}, action='block', options=[('image', True)])\n Filter(text='@@/abc\\\\.com/', selector={'type': 'url-regexp', 'value': 'abc\\\\.com'}, action='allow', options=[])\n\n\nThe ``abp.filters`` module also exports a lower-level function for parsing\nindividual lines of a filter list: ``parse_line``. It returns a parsed line\nobject just like the items in the iterator returned by ``parse_filterlist``.\n\nFor further information on the library API use ``help()`` on ``abp.filters`` and\nits contents in an interactive Python session, read the docstrings, or look at\nthe tests for some usage examples.\n\nBlocks of filters\n~~~~~~~~~~~~~~~~~\n\nFurther processing of blocks of filters separated by comments can be performed\nusing ``to_blocks`` function from ``abp.filters.blocks``:\n\n.. code-block:: python\n\n from abp.filters import parse_filterlist\n from abp.filters.blocks import to_blocks\n\n with open(fl_path) as f:\n for block in to_blocks(parse_filterlist(f)):\n print(json.dumps(block.to_dict(), indent=2))\n\nUse ``help()`` on ``abp.filters.blocks`` for more information.\n\nTesting\n-------\n\nUnit tests for ``python-abp`` are located in the ``/tests`` directory. `Pytest `_\nis used for quickly running the tests during development. `Tox `_ is used for\ntesting in different environments (Python 2.7, Python 3.5+ and PyPy) and code\nquality reporting.\n\nUse tox for a comprehensive report of unit tests and test coverage::\n\n $ tox\n\nDevelopment\n-----------\n\nWhen adding new functionality, add tests for it (preferably first). If some\ncode will never be reached on a certain version of Python, it may be exempted\nfrom coverage tests by adding a comment, e.g. ``# pragma: no py2 cover``.\n\nAll public functions, classes and methods should have docstrings compliant with\n`NumPy/SciPy documentation guide `_.\nOne exception is the constructors of classes that the user is not expected to\ninstantiate (such as exceptions).\n\n\nUsing the library with R\n------------------------\nInstallation\n~~~~~~~~~~~~\n``python-abp`` can be installed from PyPI or from the source code, either\ndirectly onto a system or in a virtual environment.\n\nTo install from PyPI::\n\n $ pip install -U python-abp\n\nTo install from a local source, clone the repo and then::\n\n $ pip install -U /path/to/python-abp\n\nTo use the virtual environment, it must first be created. Python 2 and 3 use\ndifferent scripts to create a virtualenv.\n\nIn Python 2::\n\n $ virtualenv env\n\nIn Python 3::\n\n $ python3 -m venv env\n\nThen, use the virtualenv's version of pip to install python-abp, either from\nPyPI or from source (as shown above)::\n\n $ env/bin/pip install -U python-abp\n\nFor more information about virtualenv, please see the `User Guide`_ and the\ndocs_.\n\nUsage\n~~~~~\nIn R, ``python-abp`` can be imported with ``reticulate``:\n\n.. code-block:: R\n\n > library(reticulate)\n > use_virtualenv(\"~/path/to/env\", required=TRUE) # If using virtualenv\n > abp <- import(\"abp.filters.rpy\")\n\nNow you can use the functions with ``abp$functionname``, e.g.\n``abp$line2dict(\"@@||g.doubleclick.net/pagead/$subdocument,domain=hon30.org\")``\n\nFor more information about the reticulate package, see their guide_.\n\n.. _User Guide: https://virtualenv.pypa.io/en/latest/userguide/#usage\n.. _docs: https://docs.python.org/3/library/venv.html\n.. _guide: https://rstudio.github.io/reticulate/", "description_content_type": "text/x-rst", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://hg.adblockplus.org/python-abp/", "keywords": "filterlist adblockplus ABP", "license": "GPLv3", "maintainer": "", "maintainer_email": "", "name": "python-abp", "package_url": "https://pypi.org/project/python-abp/", "platform": "", "project_url": "https://pypi.org/project/python-abp/", "project_urls": { "Homepage": "https://hg.adblockplus.org/python-abp/" }, "release_url": "https://pypi.org/project/python-abp/0.1.3/", "requires_dist": null, "requires_python": "", "summary": "A library for working with Adblock Plus filter lists.", "version": "0.1.3" }, "last_serial": 5395462, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "67cd6759bbe5f02d7f9a8056b1e94b97", "sha256": "f952f39062f145e790b23a5f3b9104dacdcc36a459b8cafe90967db4de714d8c" }, "downloads": -1, "filename": "python_abp-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "67cd6759bbe5f02d7f9a8056b1e94b97", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 31786, "upload_time": "2019-01-09T00:41:46", "url": "https://files.pythonhosted.org/packages/ba/d5/6d528597a4bcf0475b7307f091f7dff06d2eb7a6168b3b9076d2368b85b9/python_abp-0.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a61df78a3ac205d1eb19cd0925176de0", "sha256": "33fa58beb34e1392c64a20a47947d7a8d416ddead6814a2c3f5ab511a45374f8" }, "downloads": -1, "filename": "python-abp-0.1.0.tar.gz", "has_sig": false, "md5_digest": "a61df78a3ac205d1eb19cd0925176de0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 106469, "upload_time": "2019-01-09T00:41:48", "url": "https://files.pythonhosted.org/packages/68/1a/a1094bc77c02f364251046b4edcdc49e25721af539bdcee5a08a3709b044/python-abp-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "922e7c40d836a5a2011775214ade12b7", "sha256": "f819ff67899746cff747cc7b229c6e3ab61264f98d8db1e0b81d788b3eb74f2f" }, "downloads": -1, "filename": "python-abp-0.1.1.tar.gz", "has_sig": false, "md5_digest": "922e7c40d836a5a2011775214ade12b7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 95139, "upload_time": "2019-03-05T15:46:15", "url": "https://files.pythonhosted.org/packages/10/74/7e8f26ddea4b462cedf826b6b3c2c39da1045d3fffb968ea1c5f032e1f62/python-abp-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "6994065b373a92198b97f70aad0bb939", "sha256": "ed19cb034f1180b11c77fd983a2be5824986f08836088a0ae376d8954be50d73" }, "downloads": -1, "filename": "python-abp-0.1.2.tar.gz", "has_sig": false, "md5_digest": "6994065b373a92198b97f70aad0bb939", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 177701, "upload_time": "2019-05-10T15:43:03", "url": "https://files.pythonhosted.org/packages/c7/fe/75338a3737f8ec8c0fa9a3893cc991a3816500b5b7328d4230a6951b0203/python-abp-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "f89749a9703901751fe0377725fc727a", "sha256": "86264fde7a955a6908feb8eed63e362556cbcd8473c8e35cddb41dc8691e577b" }, "downloads": -1, "filename": "python-abp-0.1.3.tar.gz", "has_sig": false, "md5_digest": "f89749a9703901751fe0377725fc727a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 134173, "upload_time": "2019-06-13T11:04:57", "url": "https://files.pythonhosted.org/packages/08/a4/7dc70a08934488cbfcc7f33057fdd9c24eddaedf6859a12d65a7e9dda87f/python-abp-0.1.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f89749a9703901751fe0377725fc727a", "sha256": "86264fde7a955a6908feb8eed63e362556cbcd8473c8e35cddb41dc8691e577b" }, "downloads": -1, "filename": "python-abp-0.1.3.tar.gz", "has_sig": false, "md5_digest": "f89749a9703901751fe0377725fc727a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 134173, "upload_time": "2019-06-13T11:04:57", "url": "https://files.pythonhosted.org/packages/08/a4/7dc70a08934488cbfcc7f33057fdd9c24eddaedf6859a12d65a7e9dda87f/python-abp-0.1.3.tar.gz" } ] }