{ "info": { "author": "Peter Thomassen", "author_email": "peter.thomassen@securesystems.de", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Topic :: Internet :: Name Service (DNS)" ], "description": "# DNS-based Public Suffix List handling for Python\n\nThis Python package provides a `PSL` class for [querying the Public\nSuffix List (PSL)](https://publicsuffix.zone/) via the DNS. By utilizing\nthe library, one can retrieve information about the public suffix\nstatus of a domain as well as the PSL rules governing it. There is also\na corresponding command-line tool, `psl-dns_query`, enabling convenient\nuse of the library from the shell.\n\nPublic suffix information is based on DNS lookups only; no rule\nmatching is performed at lookup time. To make this possible, the PSL\nrules have been encoded in the DNS itself (currently under the\nDNSSEC-enabled zone `query.publicsuffix.zone`). This facilitates easy\nquerying without the need to keep the PSL at hand. The PSL zone is\nmaintained by [SSE](https://securesystems.de/) and usually updated once\na day.\n\nThe `Parser` class (along with the `psl-dns_parse` command) is used to\niterate over a [PSL file](https://publicsuffix.org/list/public_suffix_list.dat)\nand convert the ruleset into a list of DNS Resource Record sets for\nsubmission to the DNS operator. The tool adds an extra `TXT` record at\nthe root of the PSL zone, containing the parsing timestamp as well as\nthe PSL file SHA-256 hash for currentness checking.\n\nThe package also contains the `psl-dns_check` command (based on the\n`Checker` class) to iterate over a PSL file and query the DNS for each\nrule encountered, to verify whether the PSL zone contents are in\nagreement with the file. (Note that DNS caching may cause update\ndelays; after a zone update, you may be receiving outdated information\nuntil the TTL of the PSL DNS records expires. To make sure, specify one\nof the PSL zone's authoritative servers as the `resolver` argument.)\n\n**Note:** DNS resolvers learn about the domains that get queried, so\ndepending on the use case, using this service may not be up to your\nprivacy standards. It is possible though to set up a private copy of\nthe query zone and configure a local resolver to avoid query leaks.\n\n## Usage\n\n### Python\nThe following examples show how to query the PSL via DNS using the\n`PSL` class. For advanced use, please refer to the source.\n\nExample use cases for the `Parser` and `Checker` classes can be found\nin the scripts under `psl/commands/`.\n\n#### Initialize\n```python\n>>> from psl_dns import PSL\n>>> psl = PSL()\n```\n\n#### Query public suffix status of a domain (for the rules, see below)\n```python\n>>> psl.is_public_suffix('com')\nTrue\n>>> psl.is_public_suffix('checkip.dedyn.io')\nFalse\n>>> psl.is_public_suffix('takatsu.kawasaki.jp')\nTrue\n>>> psl.is_public_suffix('www.ikuoufukushi.takatsu.kawasaki.jp')\nFalse\n>>> psl.is_public_suffix('city.kawasaki.jp')\nFalse\n>>> psl.is_public_suffix('www.library.city.kawasaki.jp')\nFalse\n```\n\n#### Get the public suffix for a domain\n```python\n>>> psl.get_public_suffix('com')\n'com'\n>>> psl.get_public_suffix('checkip.dedyn.io')\n'dedyn.io'\n```\n\nThe following examples are based on PSL wildcard rules. Wildcard labels\nare expanded into the respective labels of the domain of interest:\n\n```python\n>>> psl.get_public_suffix('takatsu.kawasaki.jp') # Wildcard *.kawasaki.jp\n'takatsu.kawasaki.jp'\n>>> psl.get_public_suffix('www.ikuoufukushi.takatsu.kawasaki.jp') # same\n'takatsu.kawasaki.jp'\n>>> psl.get_public_suffix('city.kawasaki.jp') # Wildcard exception\n'jp'\n>>> psl.get_public_suffix('www.library.city.kawasaki.jp') # same\n'jp'\n```\n\nIf the queried domain has a trailing dot, the dot is preserved in the\nresponse. Furthermore, IDDA mode is preserved so that Unicode queries\nreturn Unicode responses, and Punycode queries return Punycode responses:\n\n```python\n>>> psl.get_public_suffix('www.xn--55qx5d.cn')\n'xn--55qx5d.cn'\n>>> psl.get_public_suffix('www.\u516c\u53f8.cn.')\n'\u516c\u53f8.cn.'\n```\n\n#### Get the set of rules applicable for a domain\n```python\n>>> psl.get_rules('com')\n{'com'}\n>>> psl.get_rules('checkip.dedyn.io')\n{'dedyn.io'}\n>>> psl.get_rules('takatsu.kawasaki.jp')\n{'*.kawasaki.jp'}\n>>> psl.get_rules('www.ikuoufukushi.takatsu.kawasaki.jp')\n{'*.kawasaki.jp'}\n>>> psl.get_rules('city.kawasaki.jp') # Note wildcard exception\n{'jp', '!city.kawasaki.jp', '*.kawasaki.jp'}\n>>> psl.get_rules('www.library.city.kawasaki.jp') # same\n{'jp', '!city.kawasaki.jp', '*.kawasaki.jp'}\n```\n\nRules are always returned in Unicode encoding and without a trailing\ndot, consistent with the encoding in the Public Suffix List itself:\n\n```python\n>>> psl.get_rules('www.xn--55qx5d.cn.')\n{'\u516c\u53f8.cn'}\n```\n\n#### Rules with inline wildcards\nUnfortunately, rules with inline wildcard labels `*` (i.e. wildcards\nthat are not at the leftmost position) cannot be represented using DNS\nlookups. Luckily, the PSL does not contain any such rules as of the\ntime of this writing (but this may change).\n\nTo demonstrate what would happen in such a case, a few test rules have\nbeen added to the PSL zone under the `*.wildcard.test` domain. (As\nthese rules are made up, they are not included in the PSL checksum\ncalculation.)\n\nWhen querying the public suffix (status) for a domain that falls into\nthe realm of a wildcard label which acts as an inline label in at\nleast one PSL rule, an `UnsupportedRule` exception is thrown:\n\n```python\n# Query public suffix status\n>>> psl.is_public_suffix('unsupported.wildcard.test')\nTraceback (most recent call last):\n File \"\", line 1, in \n ...\n raise UnsupportedRule\npsl.exceptions.UnsupportedRule: Domain unsupported.wildcard.test is affected by an unsupported rule\n\n# Get the public suffix\n>>> psl.get_public_suffix('unsupported.wildcard.test')\nTraceback (most recent call last):\n File \"\", line 1, in \n ...\n raise UnsupportedRule\npsl.exceptions.UnsupportedRule: Domain unsupported.wildcard.test is affected by an unsupported rule\n```\n\nHowever, you can retrieve the relevant rules for manual consumption:\n\n```python\n# Get the applicable rules\n>>> psl.get_rules('unsupported.wildcard.test')\n{'*.wildcard.test', '!except.inline.*.wildcard.test', 'inline.*.wildcard.test', '*.inline.*.wildcard.test'}\n```\n\nThis behavior applies to the entire DNS subtree that is defined by the\nfirst (right-most) wildcard label in the rule.\n\n\n### Command line\n\n#### psl-dns_query\nThis is a command-line interface to the `PSL` class demonstrated in the\nprevious section.\n\n```sh\n$ psl-dns_query -h\nusage: psl-dns_query [-h] [--zone ZONE] [--resolver RESOLVER]\n [--timeout TIMEOUT] [-l] [-c] [-v]\n domain\n\nQuery the PSL via DNS and check the PSL status of a domain.\n\nReturns the the word \"public\" or \"private\", followed by the public\nsuffix that covers the queried domain. IDNA mode and trailing dots\n(if given) are preserved.\n\nPublic Suffix List (PSL) rules with inline wildcards are not fully\nsupported. If the queried name is governed by such a rule, the word\n\"unknown\" is returned.\n\nOptionally, the set of applicable rules and the PSL checksum can be\ndisplayed.\n\nExit codes: 0 (public), 1 (private), or 2 (unknown).\n\npositional arguments:\n domain Domain to query\n\noptional arguments:\n -h, --help show this help message and exit\n --zone ZONE PSL zone to use (default: query.publicsuffix.zone)\n --resolver RESOLVER DNS resolver to use instead of system resolver\n (default: None)\n --timeout TIMEOUT DNS query timeout (seconds) (default: 5)\n -l Show set of applicable rules (default: False)\n -c Show PSL checksum (default: False)\n -v, --verbose Increase output verbosity (default: 0)\n```\n\n##### Retrieve status and public suffix\n```sh\n# Plain\n$ psl-dns_query com\npublic com\n\n# Same, followed by the set of relevant rules, in no particular order\n$ psl-dns_query www.ck -l\nprivate *\n*.ck\n!www.ck\n*\n```\n\n#### psl-dns_parse\n```sh\n$ psl-dns_parse -h\nusage: psl-dns_parse [-h] [--zone ZONE] [--format FORMAT] [-l] [-v] psl_file\n\nPrint rules from a Public Suffix List (PSL) file in DNS RRsets format.\n\npositional arguments:\n psl_file Path to PSL file\n\noptional arguments:\n -h, --help show this help message and exit\n --zone ZONE PSL zone to use (default: query.publicsuffix.zone)\n --format FORMAT Output format to use (default: deSEC)\n -l List available formats (default: False)\n -v, --verbose Increase output verbosity (default: 0)\n```\n\n##### Convert current PSL file to deSEC RRsets\n```sh\n# Note: This produces very long output\n$ time psl-dns_parse <(curl https://publicsuffix.org/list/public_suffix_list.dat) | jq .\n[\n {\n \"subname\": \"ac\",\n \"ttl\": 86400,\n \"type\": \"PTR\",\n \"records\": [\n \"ac.\"\n ]\n },\n ... # shortened for readability\n {\n \"subname\": \"\",\n \"ttl\": 86400,\n \"type\": \"TXT\",\n \"records\": [\n \"\\\"1555895008 d205f587d61c6bbf05bec818776da1dd030ce68f2e8912fea732158b9a33cc54\\\"\"\n ]\n }\n]\n\nreal\t0m1.262s\nuser\t0m0.475s\nsys\t0m0.239s\n```\n\n#### psl-dns_check\n```sh\n$ psl-dns_check -h\nusage: psl-dns_check [-h] [--resolver RESOLVER] [--timeout TIMEOUT]\n [--zone ZONE] [-v]\n psl_file\n\nCheck rules from the Public Suffix List (PSL) via DNS and output\ninconsistencies.\n\npositional arguments:\n psl_file Path to PSL file\n\noptional arguments:\n -h, --help show this help message and exit\n --resolver RESOLVER DNS resolver to use instead of system resolver\n (default: None)\n --timeout TIMEOUT DNS query timeout (in seconds) (default: 5)\n --zone ZONE PSL zone to use (default: query.publicsuffix.zone)\n -v, --verbose Increase output verbosity (default: 0)\n```\n\n##### Verifying the correctness of the PSL zone\n```sh\n$ time psl-dns_check -v <(curl https://publicsuffix.org/list/public_suffix_list.dat)\n... # shortened for readability\nINFO:psl:Querying for zone.id.query.publicsuffix.zone. TXT\nINFO:psl:Querying for zone.id.query.publicsuffix.zone. PTR\nINFO:psl:Querying for query.publicsuffix.zone. TXT\nWARNING:psl:Hash mismatch! Input PSL file appears to differ from remote version.\n8684 rules with 3 inconsistencies found\n\nreal\t13m42.366s\nuser\t0m38.560s\nsys\t0m8.383s\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/sse-secure-systems/psl-dns", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "psl-dns", "package_url": "https://pypi.org/project/psl-dns/", "platform": "", "project_url": "https://pypi.org/project/psl-dns/", "project_urls": { "Homepage": "https://github.com/sse-secure-systems/psl-dns" }, "release_url": "https://pypi.org/project/psl-dns/1.0rc4/", "requires_dist": [ "dnspython (>=1.14.0)" ], "requires_python": "", "summary": "Query the Public Suffix List (PSL) via DNS and check the PSL status of a domain.", "version": "1.0rc4" }, "last_serial": 5383714, "releases": { "1.0b1": [ { "comment_text": "", "digests": { "md5": "76cd887d2fc5fbfccdadb1dedceef7bf", "sha256": "5c7b60bdfbd39122069a9e77d1e791fb7fb0d935726c3d53ccb68dcbaede4a87" }, "downloads": -1, "filename": "psl_dns-1.0b1-py3-none-any.whl", "has_sig": false, "md5_digest": "76cd887d2fc5fbfccdadb1dedceef7bf", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12856, "upload_time": "2019-04-22T03:31:00", "url": "https://files.pythonhosted.org/packages/9f/21/3806399fa4c9f7a0590f47e0b45f425f1632ac0c57015294ce8755ded3e0/psl_dns-1.0b1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "23a5c6512e9101804d33d3fd6f47b998", "sha256": "6e395a8061ea0caba812ae25c430737f44be5723725d859402153a098406d52b" }, "downloads": -1, "filename": "psl-dns-1.0b1.tar.gz", "has_sig": false, "md5_digest": "23a5c6512e9101804d33d3fd6f47b998", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13425, "upload_time": "2019-04-22T03:31:02", "url": "https://files.pythonhosted.org/packages/aa/9d/ee202012dbc97b9e37bf55cb3ab3e21a32bdaa47cebec869abb97dbe9982/psl-dns-1.0b1.tar.gz" } ], "1.0b2": [ { "comment_text": "", "digests": { "md5": "c8c6171e703ec08e81f2f0a2ef153bb7", "sha256": "e469e72731fdfc261e6618ab5bc259e86fffc75cecd30ee2522c86232a96ea86" }, "downloads": -1, "filename": "psl_dns-1.0b2-py3-none-any.whl", "has_sig": false, "md5_digest": "c8c6171e703ec08e81f2f0a2ef153bb7", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12834, "upload_time": "2019-04-23T22:45:14", "url": "https://files.pythonhosted.org/packages/57/e7/66c40f2b38ddd6216a590c2ada85bd86db0bef4c6126c831a7fcfd65b631/psl_dns-1.0b2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ab84ff4dee58713343a816407f0cc4d8", "sha256": "b0a1cad54cdcc5fe5bcbbbbce0041f588b74b7fec5772166afe48d2fb56a431f" }, "downloads": -1, "filename": "psl-dns-1.0b2.tar.gz", "has_sig": false, "md5_digest": "ab84ff4dee58713343a816407f0cc4d8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13400, "upload_time": "2019-04-23T22:45:16", "url": "https://files.pythonhosted.org/packages/c7/f0/d0bfe3c6b0ec289076061797488e74cab11c6e3af409528ebe32613dcfc0/psl-dns-1.0b2.tar.gz" } ], "1.0b3": [ { "comment_text": "", "digests": { "md5": "d603fad8a9d0c2615388ff2ed899b86e", "sha256": "512d2ebcc038827aebdb5ca8eb0c8b159c702288355820c7f0308a5fbf354336" }, "downloads": -1, "filename": "psl_dns-1.0b3-py3-none-any.whl", "has_sig": false, "md5_digest": "d603fad8a9d0c2615388ff2ed899b86e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20345, "upload_time": "2019-04-23T23:11:17", "url": "https://files.pythonhosted.org/packages/9c/fd/9097185c838d4f0090a181bdae69cffe6f4ea694a87557639c4666f9de2a/psl_dns-1.0b3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b1abfb77f8b3d23178e799a6ad27448c", "sha256": "fdd56b935692f6fc07becd1cd0b068f60c5ac3c730cb7884aa60509ab7ae5251" }, "downloads": -1, "filename": "psl-dns-1.0b3.tar.gz", "has_sig": false, "md5_digest": "b1abfb77f8b3d23178e799a6ad27448c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10507, "upload_time": "2019-04-23T23:11:18", "url": "https://files.pythonhosted.org/packages/2e/e0/d55835fec1316bb03cde0e87e3a35ee0a804c7337ff6fc0ef83eade8dae9/psl-dns-1.0b3.tar.gz" } ], "1.0rc1": [ { "comment_text": "", "digests": { "md5": "7a4a8ebb9d9180936bcd676835ed2a52", "sha256": "33308265dbbc6d4f5ce4bbd1ef2444f9ea2857250d406e80d7fa5344c4a40d81" }, "downloads": -1, "filename": "psl_dns-1.0rc1-py3-none-any.whl", "has_sig": false, "md5_digest": "7a4a8ebb9d9180936bcd676835ed2a52", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20672, "upload_time": "2019-05-04T19:40:40", "url": "https://files.pythonhosted.org/packages/93/98/2b9e66ba02ef9048b401c2b1432d2307b577cfb14a37d89ea5b971721a02/psl_dns-1.0rc1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "eb1109a1bb20f25a3a6a5f9f85bc52d3", "sha256": "0a6d03139683ffdc5c0b2fd6859a950cd58ed81439d0e68e594727f375f5a18f" }, "downloads": -1, "filename": "psl-dns-1.0rc1.tar.gz", "has_sig": false, "md5_digest": "eb1109a1bb20f25a3a6a5f9f85bc52d3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13984, "upload_time": "2019-05-04T19:40:43", "url": "https://files.pythonhosted.org/packages/7e/eb/7fc34062df51769a8c06b231a2152ae9e132f789832b60c70d209b866d79/psl-dns-1.0rc1.tar.gz" } ], "1.0rc2": [ { "comment_text": "", "digests": { "md5": "c3b5d95c8039b8e26b21d0011e441359", "sha256": "215f1a387d109798a24a73bf5018b80c166b30557fb01b1c9562621da0098879" }, "downloads": -1, "filename": "psl_dns-1.0rc2-py3-none-any.whl", "has_sig": false, "md5_digest": "c3b5d95c8039b8e26b21d0011e441359", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20955, "upload_time": "2019-05-04T20:25:00", "url": "https://files.pythonhosted.org/packages/60/35/83ba4f6bdff8047167d556e3e023f0500430eb94fdfdd5bb5b7de2ccd9c5/psl_dns-1.0rc2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "efedcc706999c0b504cefb2b66be0dc8", "sha256": "d3e504a466b5e4567cbd5a28a7dacbca2247107ed8550f619b956a0ffeb0f2b1" }, "downloads": -1, "filename": "psl-dns-1.0rc2.tar.gz", "has_sig": false, "md5_digest": "efedcc706999c0b504cefb2b66be0dc8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14363, "upload_time": "2019-05-04T20:25:04", "url": "https://files.pythonhosted.org/packages/9f/8d/239757f0f377dc22c548fe8840d19f1217668b5e38a5d65652efce856358/psl-dns-1.0rc2.tar.gz" } ], "1.0rc3": [ { "comment_text": "", "digests": { "md5": "6ce3baa3ec69eae75e016bb2245370de", "sha256": "95091ef51bf39e1613447d0a7cc614d82e28067c0dde4230407479f489373ff2" }, "downloads": -1, "filename": "psl_dns-1.0rc3-py3-none-any.whl", "has_sig": false, "md5_digest": "6ce3baa3ec69eae75e016bb2245370de", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15072, "upload_time": "2019-05-24T23:44:04", "url": "https://files.pythonhosted.org/packages/09/39/971acbba24673e72a9cf83c94d217ce080b6af839358ddb733365d9dd013/psl_dns-1.0rc3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "304609ffae92d704ba4018d60ae69e1d", "sha256": "28227d6c59d7b5e790fcca3f5377de727b6541a10540e51f17f955c0eb949676" }, "downloads": -1, "filename": "psl-dns-1.0rc3.tar.gz", "has_sig": false, "md5_digest": "304609ffae92d704ba4018d60ae69e1d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15336, "upload_time": "2019-05-24T23:44:06", "url": "https://files.pythonhosted.org/packages/f2/4a/5ac7f78a19d2f529d060ff24e91f4b2fad6558f1dad4d73ea67559f368b6/psl-dns-1.0rc3.tar.gz" } ], "1.0rc4": [ { "comment_text": "", "digests": { "md5": "941fff93863201d85d3559f60163470b", "sha256": "a794ef08bf3314484d3966ccf07f511722b485552f864d9f36e31fa723b6bfb5" }, "downloads": -1, "filename": "psl_dns-1.0rc4-py3-none-any.whl", "has_sig": false, "md5_digest": "941fff93863201d85d3559f60163470b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15276, "upload_time": "2019-06-10T22:36:09", "url": "https://files.pythonhosted.org/packages/d6/22/56e1b735d1f3b5ca20aad7b18c9f6590d887347b011408c2543a1a212a50/psl_dns-1.0rc4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4507e1ed6be9391eaac5e09a178c3a68", "sha256": "6870c57e057e10c05cbcd84c4b48eeff9b23f1f3dba64dc1c32df740d57f8f23" }, "downloads": -1, "filename": "psl-dns-1.0rc4.tar.gz", "has_sig": false, "md5_digest": "4507e1ed6be9391eaac5e09a178c3a68", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15673, "upload_time": "2019-06-10T22:36:13", "url": "https://files.pythonhosted.org/packages/ce/0e/5677f91c1479240fc6f44b6520dc9aed644341d6afc3cfeb43eb258d1acf/psl-dns-1.0rc4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "941fff93863201d85d3559f60163470b", "sha256": "a794ef08bf3314484d3966ccf07f511722b485552f864d9f36e31fa723b6bfb5" }, "downloads": -1, "filename": "psl_dns-1.0rc4-py3-none-any.whl", "has_sig": false, "md5_digest": "941fff93863201d85d3559f60163470b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15276, "upload_time": "2019-06-10T22:36:09", "url": "https://files.pythonhosted.org/packages/d6/22/56e1b735d1f3b5ca20aad7b18c9f6590d887347b011408c2543a1a212a50/psl_dns-1.0rc4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4507e1ed6be9391eaac5e09a178c3a68", "sha256": "6870c57e057e10c05cbcd84c4b48eeff9b23f1f3dba64dc1c32df740d57f8f23" }, "downloads": -1, "filename": "psl-dns-1.0rc4.tar.gz", "has_sig": false, "md5_digest": "4507e1ed6be9391eaac5e09a178c3a68", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15673, "upload_time": "2019-06-10T22:36:13", "url": "https://files.pythonhosted.org/packages/ce/0e/5677f91c1479240fc6f44b6520dc9aed644341d6afc3cfeb43eb258d1acf/psl-dns-1.0rc4.tar.gz" } ] }