{ "info": { "author": "James Graham", "author_email": "james@hoppipolla.co.uk", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Text Processing :: Markup :: HTML" ], "description": "html5lib\n========\n\n.. image:: https://travis-ci.org/html5lib/html5lib-python.png?branch=master\n :target: https://travis-ci.org/html5lib/html5lib-python\n\nhtml5lib is a pure-python library for parsing HTML. It is designed to\nconform to the WHATWG HTML specification, as is implemented by all major\nweb browsers.\n\n\nUsage\n-----\n\nSimple usage follows this pattern:\n\n.. code-block:: python\n\n import html5lib\n with open(\"mydocument.html\", \"rb\") as f:\n document = html5lib.parse(f)\n\nor:\n\n.. code-block:: python\n\n import html5lib\n document = html5lib.parse(\"

Hello World!\")\n\nBy default, the ``document`` will be an ``xml.etree`` element instance.\nWhenever possible, html5lib chooses the accelerated ``ElementTree``\nimplementation (i.e. ``xml.etree.cElementTree`` on Python 2.x).\n\nTwo other tree types are supported: ``xml.dom.minidom`` and\n``lxml.etree``. To use an alternative format, specify the name of\na treebuilder:\n\n.. code-block:: python\n\n import html5lib\n with open(\"mydocument.html\", \"rb\") as f:\n lxml_etree_document = html5lib.parse(f, treebuilder=\"lxml\")\n\nWhen using with ``urllib2`` (Python 2), the charset from HTTP should be\npass into html5lib as follows:\n\n.. code-block:: python\n\n from contextlib import closing\n from urllib2 import urlopen\n import html5lib\n\n with closing(urlopen(\"http://example.com/\")) as f:\n document = html5lib.parse(f, transport_encoding=f.info().getparam(\"charset\"))\n\nWhen using with ``urllib.request`` (Python 3), the charset from HTTP\nshould be pass into html5lib as follows:\n\n.. code-block:: python\n\n from urllib.request import urlopen\n import html5lib\n\n with urlopen(\"http://example.com/\") as f:\n document = html5lib.parse(f, transport_encoding=f.info().get_content_charset())\n\nTo have more control over the parser, create a parser object explicitly.\nFor instance, to make the parser raise exceptions on parse errors, use:\n\n.. code-block:: python\n\n import html5lib\n with open(\"mydocument.html\", \"rb\") as f:\n parser = html5lib.HTMLParser(strict=True)\n document = parser.parse(f)\n\nWhen you're instantiating parser objects explicitly, pass a treebuilder\nclass as the ``tree`` keyword argument to use an alternative document\nformat:\n\n.. code-block:: python\n\n import html5lib\n parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder(\"dom\"))\n minidom_document = parser.parse(\"

Hello World!\")\n\nMore documentation is available at https://html5lib.readthedocs.io/.\n\n\nInstallation\n------------\n\nhtml5lib works on CPython 2.6+, CPython 3.3+ and PyPy. To install it,\nuse:\n\n.. code-block:: bash\n\n $ pip install html5lib\n\n\nOptional Dependencies\n---------------------\n\nThe following third-party libraries may be used for additional\nfunctionality:\n\n- ``datrie`` can be used under CPython to improve parsing performance\n (though in almost all cases the improvement is marginal);\n\n- ``lxml`` is supported as a tree format (for both building and\n walking) under CPython (but *not* PyPy where it is known to cause\n segfaults);\n\n- ``genshi`` has a treewalker (but not builder); and\n\n- ``chardet`` can be used as a fallback when character encoding cannot\n be determined.\n\n\nBugs\n----\n\nPlease report any bugs on the `issue tracker\n`_.\n\n\nTests\n-----\n\nUnit tests require the ``pytest`` and ``mock`` libraries and can be\nrun using the ``py.test`` command in the root directory;\n``ordereddict`` is required under Python 2.6. All should pass.\n\nTest data are contained in a separate `html5lib-tests\n`_ repository and included\nas a submodule, thus for git checkouts they must be initialized::\n\n $ git submodule init\n $ git submodule update\n\nIf you have all compatible Python implementations available on your\nsystem, you can run tests on all of them using the ``tox`` utility,\nwhich can be found on PyPI.\n\n\nQuestions?\n----------\n\nThere's a mailing list available for support on Google Groups,\n`html5lib-discuss `_,\nthough you may get a quicker response asking on IRC in `#whatwg on\nirc.freenode.net `_.\n\nChange Log\n----------\n\n0.99999999/1.0b9\n~~~~~~~~~~~~~~~~\n\nReleased on July 14, 2016\n\n* **Added ordereddict as a mandatory dependency on Python 2.6.**\n\n* Added ``lxml``, ``genshi``, ``datrie``, ``charade``, and ``all``\n extras that will do the right thing based on the specific\n interpreter implementation.\n\n* Now requires the ``mock`` package for the testsuite.\n\n* Cease supporting DATrie under PyPy.\n\n* **Remove ``PullDOM`` support, as this hasn't ever been properly\n tested, doesn't entirely work, and as far as I can tell is\n completely unused by anyone.**\n\n* Move testsuite to ``py.test``.\n\n* **Fix #124: move to webencodings for decoding the input byte stream;\n this makes html5lib compliant with the Encoding Standard, and\n introduces a required dependency on webencodings.**\n\n* **Cease supporting Python 3.2 (in both CPython and PyPy forms).**\n\n* **Fix comments containing double-dash with lxml 3.5 and above.**\n\n* **Use scripting disabled by default (as we don't implement\n scripting).**\n\n* **Fix #11, avoiding the XSS bug potentially caused by serializer\n allowing attribute values to be escaped out of in old browser versions,\n changing the quote_attr_values option on serializer to take one of\n three values, \"always\" (the old True value), \"legacy\" (the new option,\n and the new default), and \"spec\" (the old False value, and the old\n default).**\n\n* **Fix #72 by rewriting the sanitizer to apply only to treewalkers\n (instead of the tokenizer); as such, this will require amending all\n callers of it to use it via the treewalker API.**\n\n* **Drop support of charade, now that chardet is supported once more.**\n\n* **Replace the charset keyword argument on parse and related methods\n with a set of keyword arguments: override_encoding, transport_encoding,\n same_origin_parent_encoding, likely_encoding, and default_encoding.**\n\n* **Move filters._base, treebuilder._base, and treewalkers._base to .base\n to clarify their status as public.**\n\n* **Get rid of the sanitizer package. Merge sanitizer.sanitize into the\n sanitizer.htmlsanitizer module and move that to saniziter. This means\n anyone who used sanitizer.sanitize or sanitizer.HTMLSanitizer needs no\n code changes.**\n\n* **Rename treewalkers.lxmletree to .etree_lxml and\n treewalkers.genshistream to .genshi to have a consistent API.**\n\n* Move a whole load of stuff (inputstream, ihatexml, trie, tokenizer,\n utils) to be underscore prefixed to clarify their status as private.\n\n\n0.9999999/1.0b8\n~~~~~~~~~~~~~~~\n\nReleased on September 10, 2015\n\n* Fix #195: fix the sanitizer to drop broken URLs (it threw an\n exception between 0.9999 and 0.999999).\n\n\n0.999999/1.0b7\n~~~~~~~~~~~~~~\n\nReleased on July 7, 2015\n\n* Fix #189: fix the sanitizer to allow relative URLs again (as it did\n prior to 0.9999/1.0b5).\n\n\n0.99999/1.0b6\n~~~~~~~~~~~~~\n\nReleased on April 30, 2015\n\n* Fix #188: fix the sanitizer to not throw an exception when sanitizing\n bogus data URLs.\n\n\n0.9999/1.0b5\n~~~~~~~~~~~~\n\nReleased on April 29, 2015\n\n* Fix #153: Sanitizer fails to treat some attributes as URLs. Despite how\n this sounds, this has no known security implications. No known version\n of IE (5.5 to current), Firefox (3 to current), Safari (6 to current),\n Chrome (1 to current), or Opera (12 to current) will run any script\n provided in these attributes.\n\n* Pass error message to the ParseError exception in strict parsing mode.\n\n* Allow data URIs in the sanitizer, with a whitelist of content-types.\n\n* Add support for Python implementations that don't support lone\n surrogates (read: Jython). Fixes #2.\n\n* Remove localization of error messages. This functionality was totally\n unused (and untested that everything was localizable), so we may as\n well follow numerous browsers in not supporting translating technical\n strings.\n\n* Expose treewalkers.pprint as a public API.\n\n* Add a documentEncoding property to HTML5Parser, fix #121.\n\n\n0.999\n~~~~~\n\nReleased on December 23, 2013\n\n* Fix #127: add work-around for CPython issue #20007: .read(0) on\n http.client.HTTPResponse drops the rest of the content.\n\n* Fix #115: lxml treewalker can now deal with fragments containing, at\n their root level, text nodes with non-ASCII characters on Python 2.\n\n\n0.99\n~~~~\n\nReleased on September 10, 2013\n\n* No library changes from 1.0b3; released as 0.99 as pip has changed\n behaviour from 1.4 to avoid installing pre-release versions per\n PEP 440.\n\n\n1.0b3\n~~~~~\n\nReleased on July 24, 2013\n\n* Removed ``RecursiveTreeWalker`` from ``treewalkers._base``. Any\n implementation using it should be moved to\n ``NonRecursiveTreeWalker``, as everything bundled with html5lib has\n for years.\n\n* Fix #67 so that ``BufferedStream`` to correctly returns a bytes\n object, thereby fixing any case where html5lib is passed a\n non-seekable RawIOBase-like object.\n\n\n1.0b2\n~~~~~\n\nReleased on June 27, 2013\n\n* Removed reordering of attributes within the serializer. There is now\n an ``alphabetical_attributes`` option which preserves the previous\n behaviour through a new filter. This allows attribute order to be\n preserved through html5lib if the tree builder preserves order.\n\n* Removed ``dom2sax`` from DOM treebuilders. It has been replaced by\n ``treeadapters.sax.to_sax`` which is generic and supports any\n treewalker; it also resolves all known bugs with ``dom2sax``.\n\n* Fix treewalker assertions on hitting bytes strings on\n Python 2. Previous to 1.0b1, treewalkers coped with mixed\n bytes/unicode data on Python 2; this reintroduces this prior\n behaviour on Python 2. Behaviour is unchanged on Python 3.\n\n\n1.0b1\n~~~~~\n\nReleased on May 17, 2013\n\n* Implementation updated to implement the `HTML specification\n `_ as of 5th May\n 2013 (`SVN `_ revision r7867).\n\n* Python 3.2+ supported in a single codebase using the ``six`` library.\n\n* Removed support for Python 2.5 and older.\n\n* Removed the deprecated Beautiful Soup 3 treebuilder.\n ``beautifulsoup4`` can use ``html5lib`` as a parser instead. Note that\n since it doesn't support namespaces, foreign content like SVG and\n MathML is parsed incorrectly.\n\n* Removed ``simpletree`` from the package. The default tree builder is\n now ``etree`` (using the ``xml.etree.cElementTree`` implementation if\n available, and ``xml.etree.ElementTree`` otherwise).\n\n* Removed the ``XHTMLSerializer`` as it never actually guaranteed its\n output was well-formed XML, and hence provided little of use.\n\n* Removed default DOM treebuilder, so ``html5lib.treebuilders.dom`` is no\n longer supported. ``html5lib.treebuilders.getTreeBuilder(\"dom\")`` will\n return the default DOM treebuilder, which uses ``xml.dom.minidom``.\n\n* Optional heuristic character encoding detection now based on\n ``charade`` for Python 2.6 - 3.3 compatibility.\n\n* Optional ``Genshi`` treewalker support fixed.\n\n* Many bugfixes, including:\n\n * #33: null in attribute value breaks XML AttValue;\n\n * #4: nested, indirect descendant,