{ "info": { "author": "Manuel Barkhau", "author_email": "mbarkhau@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Environment :: Other Environment", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Operating System :: Unix", "Programming Language :: Python", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# [lib3to6][repo_ref]\n\nCompile Python 3.6+ code to Python 2.7+ compatible code. The idea is quite\nsimilar to Bable https://babeljs.io/. Develop using the newest interpreter and\nuse (most) new language features without sacrificing backward compatibility.\n\nProject/Repo:\n\n[![MIT License][license_img]][license_ref]\n[![Supported Python Versions][pyversions_img]][pyversions_ref]\n[![PyCalVer v201902.0030][version_img]][version_ref]\n[![PyPI Version][pypi_img]][pypi_ref]\n[![PyPI Downloads][downloads_img]][downloads_ref]\n\nCode Quality/CI:\n\n[![Build Status][build_img]][build_ref]\n[![Type Checked with mypy][mypy_img]][mypy_ref]\n[![Code Coverage][codecov_img]][codecov_ref]\n[![Code Style: sjfmt][style_img]][style_ref]\n\n\n| Name | role | since | until |\n|-------------------------------------|-------------------|---------|-------|\n| Manuel Barkhau (mbarkhau@gmail.com) | author/maintainer | 2018-09 | - |\n\n\n\n\n\n[](TOC)\n\n - [Project Status (as of 2019-02-21): Experimental](#project-status-as-of-2019-02-21-experimental)\n - [Getting started with Development](#getting-started-with-development)\n - [Motivation](#motivation)\n - [Feature Support](#feature-support)\n - [How it works](#how-it-works)\n - [FAQ](#faq)\n\n[](TOC)\n\n\n## Project Status (as of 2019-02-21): Experimental\n\nI've been using this library mainly for the\n[PyCalVer](https://pypi.org/project/pycalver/) project without any issues. I\nwon't be adding any new fixers or checkers until Python 3.8 nears its release.\nPlease give it a try and send your feedback.\n\nThe ultimate goal would be to cover all cases documented on\nhttp://python-future.org and either:\n\n 1. Transpile to code that will work on any version\n 2. Raise an error, ideally pointing to a page and section on\n python-future.org or other documentation describing\n alternative methods of writing backwards compatible code.\n\nhttps://docs.python.org/3.X/whatsnew/ also contains much info on\nAPI changes that might be checked for, but checks and fixers for\nthese will only be written if they are common enough, otherwise\nit's just too much work (patches are welcome though).\n\n\n## Getting started with Development\n\n\n```shell\n$ git clone https://gitlab.com/mbarkhau/lib3to6.git\n$ cd lib3to6/\nlib3to6 $ make install\n...\nlib3to6 $ make test\n...\nlib3to6 $ make help\n```\n\n\n## Motivation\n\n\nThe main motivation for this project is to be able to use `mypy`\nwithout sacrificing compatibility to older versions of python.\n\n```python\n# my_module/__init__.py\ndef hello(who: str) -> None:\n import sys\n print(f\"Hello {who} from {sys.version.split()[0]}!\")\n\n\nprint(__file__)\nhello(\"\u4e16\u754c\")\n```\n\n\n```bash\n$ pip install lib3to6\n$ python -m lib3to6 my_module/__init__.py\n```\n\n\n```python\n# -*- coding: utf-8 -*-\nfrom __future__ import absolute_import\nfrom __future__ import division\nfrom __future__ import print_function\nfrom __future__ import unicode_literals\n\n\ndef hello(who):\n import sys\n print('Hello {0} from {1}!'.format(who, sys.version.split()[0]))\n\n\nprint(__file__)\nhello('\u4e16\u754c')\n```\n\n\nFixes are applied to match the semantics of python3 code as\nclose as possible, even when running on a python2.7 interpreter.\n\nSome fixes that have been applied:\n\n - PEP263 magic comment to declare the coding of the python\n source file. This allows the string literal `\"\u4e16\u754c\"` to\n be decoded correctly.\n - `__future__` imports have been added. This includes the well\n known print statement -> function change. The unicode_literals\n - Type annotations have been removed\n - f string -> \"\".format conversion\n\n\nThe cli command `lib3to6` is nice for demo purposes,\nbut for your project it is better to use it in your\nsetup.py file.\n\n\n```python\n# setup.py\n\nimport sys\nimport setuptools\n\npackages = setuptools.find_packages(\".\")\npackage_dir = {\"\": \".\"}\n\nif any(arg.startswith(\"bdist\") for arg in sys.argv):\n import lib3to6\n package_dir = lib3to6.fix(package_dir)\n\nsetuptools.setup(\n name=\"my-module\",\n version=\"201808.1\",\n packages=packages,\n package_dir=package_dir,\n)\n```\n\n\n```bash\n~/my-module $ python setup.py bdist_wheel --python-tag=py2.py3\nrunning bdist_wheel\n...\n~/my-module$ ls -1 dist/\nmy_module-201808.1-py2.py3-none-any.whl\n\n~/my-module$ python3 -m pip install dist/my_module-201808.1-py2.py3-none-any.whl\nProcessing ./dist/my_module-201808.1-py2.py3-none-any.whl\nInstalling collected packages: my-module\nSuccessfully installed my-module-201808.1\n\n~/my-module$ python2 -m pip install dist/my_module-201808.1-py2.py3-none-any.whl\nProcessing ./dist/my_module-201808.1-py2.py3-none-any.whl\nInstalling collected packages: my-module\nSuccessfully installed my-module-201808.1\n```\n\n\nTo make sure we're importing my_module from the installation, as\nopposed to from the local directory, we have to switch\ndirectories.\n\n\n```bash\n~/$ python3 -c \"import my_module\"\n/home/user/my-module/my_module/__init__.py\nHello \u4e16\u754c from 3.6.5!\n\n~/my-module$ cd ..\n~/$ python3 -c \"import my_module\"\n/home/user/envs/py36/lib/python3.6/site-packages/my_module/__init__.py\nHello \u4e16\u754c from 3.6.5!\n\n~$ python2 -c \"import my_module\"\n/home/user/envs/py27/lib/python2.7/site-packages/my_module/__init__.py\nHello \u4e16\u754c from 2.7.15!\n```\n\n\n## Feature Support\n\nNot all new language features have a semantic equivalent in older\nversions. To the extent these can be detected, an error will be\nreported when these features are used.\n\nAn (obviously non exhaustive) list of features which are **not\nsupported**:\n\n - async/await\n - yield from\n - @/__matmul__ operator\n\nFeatures which **are supported**:\n\n - PEP 498: formatted string literals.\n - Eliding of annotations\n - Unpacking generalizations\n - Keyword only arguments\n - PEP 515: underscores in numeric literals\n - map/zip/filter to itertools equivalents\n - Convert class based typing.NamedTuple usage to assignments\n\nSome new libraries have backports, which warnings will point to:\n\n - typing\n - pathlib\n - secrets\n - ipaddress\n - csv -> backports.csv\n - lzma -> backports.lzma\n - enum -> flufl.enum\n\n\n## How it works\n\nThis project works at the level of the python abstract syntax\ntree (AST). The AST is transformed so that is only uses\nconstructs that are also valid in older versions of python. For\nexample it will translate f-strings to normal strings using the\n``str.format`` method.\n\n```python\n>>> import sys\n>>> sys.version_info\n'3.6.5'\n>>> import lib3to6\n>>> py3_source = 'f\"Hello {1 + 1}!\"'\n>>> cfg = {\"fixers\": [\"f_string_to_str_format\"]}\n>>> py2_source = lib3to6.transpile_module(cfg, py3_source)\n\n>>> print(py3_source)\nf\"Hello {1 + 1}!\"\n>>> print(py2_source)\n# -*- coding: utf-8 -*-\n\"Hello {0}!\".format(1 + 1)\n\n>>> print(lib3to6.parsedump_ast(py3_source))\nModule(body=[Expr(value=JoinedStr(values=[\n Str(s='Hello '),\n FormattedValue(\n value=BinOp(\n left=Num(n=1),\n op=Add(),\n right=Num(n=1),\n ),\n conversion=-1,\n format_spec=None,\n ),\n Str(s='!'),\n]))])\n>>> print(lib3to6.parsedump_ast(py2_source))\nModule(body=[Expr(value=Call(\n func=Attribute(\n value=Str(s='Hello {0}!'),\n attr='format',\n ctx=Load(),\n ),\n args=[BinOp(\n left=Num(n=1),\n op=Add(),\n right=Num(n=1),\n )],\n keywords=[]\n))])\n```\n\n\nOf course this does not cover every aspect of compatibility.\nChanges in APIs cannot be translated automatically in this way.\n\nAn obvious example, is that there is no way to transpile code\nwhich uses `async` and `await`. In this case, `lib3to6`\nwill simply raise a CheckError. This applies only to your source\ncode though, so if import use a library which uses `async` and\n`await`, everything may look fine until you run your tests\non python 2.7.\n\nA more subtle example is the change in semantics of the builtin\n`open` function.\n\n```bash\n$ cat open_example.py\nwith open(\"myfile.txt\", mode=\"w\", encoding=\"utf-8\") as fh:\n fh.write(\"Hello W\u00f6rld!\")\n$ python2 open_example.py\nTraceback (most recent call last):\n File \"\", line 1, in \nTypeError: 'encoding' is an invalid keyword argument for this function\n```\n\n\nUsually there are alternative ways to write equivalent code that\nworks on all versions of python. For these common\nincompatibilities lib3to6 will raise an error and suggest an\nalternative, such as in this case using `io.open` instead.\n\n```bash\n$ lib3to6 open_example.py\nTraceback (Most recent call last):\n11 lib3to6 --> sys.exit(main())\n764 core.py __call__ --> return self.main(*args, **kwargs)\n717 core.py main --> rv = self.invoke(ctx)\n956 core.py invoke --> return ctx.invoke(self.callback, **ctx.params)\n555 core.py invoke --> return callback(*args, **kwargs)\n55 __main__.py main --> fixed_source_text = transpile.transpile_module(cfg, source_text)\n260 transpile.py transpile_module --> checker(cfg, module_tree)\n158 checkers.py __call__ --> raise common.CheckError(msg, node)\nCheckError: Prohibited keyword argument 'encoding' to builtin.open. on line 1 of open_example.py\n```\n\n\nHere `lib3to6` you will give you a ``CheckError`, however it\nremains your responsibility to write your code so that this\nsyntactic translation is semantically equivalent in both python3\nand python2.\n\n`lib3to6` uses the python `ast` module to parse your code. This\nmeans that you need a modern python interpreter to transpile from\nmodern python to legacy python interpreter. You cannot transpile\nfeatures which your interpreter cannot parse. The intended use is\nfor developers of libraries who use the most modern python\nversion, but want their libraries to work on older versions.\n\n\n## FAQ\n\n - Q: Isn't the tagline \"Compatibility Matters\" ironic,\n considering that python 3.6+ is required to build a wheel?\n - A: The irony is not lost. The issue is, how to parse source\n code from a newer version of python than the python\n interpreter itself supports. You can install lib3to6 on\n older versions of python, but you'll be limited to the\n features supported by that version. For example, you won't be\n able to use f\"\" strings on python 3.5, but most annotations\n will work fine.\n\n - Q: Why keep python2.7 alive? Just let it die already!\n - A: Indeed, and lib3to6 can help with that. Put yourself in the\n shoes of somebody who is working on an old codebase. It's not\n realistic hold all other development efforts while the\n codebase is migrated and tested, while everything else waits.\n\n Instead an incremental approach is usually the only option.\n With lib3to6, individual modules of the codebase can be\n migrated to python3, leaving the rest of the codebase\n untouched. The project can still run in a python 2.7\n environment, while developers increasingly move to using\n python 3.\n\n Additionally, lib3to6 is not just for compatibility with\n python 2.7, it also allows you to use new features like f\"\"\n strings and variable annotations, while still maintaining\n compatibility with older versions of python 3.\n\n - Q: Why not `lib3to2`?\n - A: I can't honestly say much about `lib3to2`. It seems to not\n be maintained and looking at the source I thought it would be\n easier to just write something new that worked on the AST level.\n The scope of `lib3to6` is more general than 3to2, as you can\n use it even if all you care about is converting from python 3.6\n to 3.5.\n\n\n[repo_ref]: https://gitlab.com/mbarkhau/lib3to6\n\n[build_img]: https://gitlab.com/mbarkhau/lib3to6/badges/master/pipeline.svg\n[build_ref]: https://gitlab.com/mbarkhau/lib3to6/pipelines\n\n[codecov_img]: https://gitlab.com/mbarkhau/lib3to6/badges/master/coverage.svg\n[codecov_ref]: https://mbarkhau.gitlab.io/lib3to6/cov\n\n[license_img]: https://img.shields.io/badge/License-MIT-blue.svg\n[license_ref]: https://gitlab.com/mbarkhau/lib3to6/blob/master/LICENSE\n\n[mypy_img]: https://img.shields.io/badge/mypy-checked-green.svg\n[mypy_ref]: https://mbarkhau.gitlab.io/lib3to6/mypycov\n\n[style_img]: https://img.shields.io/badge/code%20style-%20sjfmt-f71.svg\n[style_ref]: https://gitlab.com/mbarkhau/straitjacket/\n\n[pypi_img]: https://img.shields.io/badge/PyPI-wheels-green.svg\n[pypi_ref]: https://pypi.org/project/lib3to6/#files\n\n[downloads_img]: https://pepy.tech/badge/lib3to6/month\n[downloads_ref]: https://pepy.tech/project/lib3to6\n\n[version_img]: https://img.shields.io/static/v1.svg?label=PyCalVer&message=v201902.0030&color=blue\n[version_ref]: https://pypi.org/project/pycalver/\n\n[pyversions_img]: https://img.shields.io/pypi/pyversions/lib3to6.svg\n[pyversions_ref]: https://pypi.python.org/pypi/lib3to6\n\n\n\n# Changelog for https://gitlab.com/mbarkhau/lib3to6\n\n## v201902.0030\n\n - Fix python 2 builtins were not always overridden correctly.\n - Fix pypy compatability testing\n - Better mypy coverage\n\n\n## v201812.0021-beta\n\n - Recursivly apply some fixers.\n\n\n## v201812.0020-alpha\n\n - Move to gitlab.com\n - Use bootstrapit\n - Fix bugs based on use with pycalver\n\n\n## v201809.0019-alpha\n\n - CheckErrors include log line numbers\n - Transpile errors now include filenames\n - Added fixers for renamed modules, e.g.\n .. code-block:: diff\n\n - import queue\n + try:\n + import queue\n + except ImportError:\n + import Queue as queue\n\n\n## v201808.0014-alpha\n\n - Better handling of package_dir\n - Change to `CalVer Versioning `_\n - Remove console script in favour of simple ``python -m lib3to6``\n - Rename from ``three2six`` -> ``lib3to6``\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://gitlab.com/mbarkhau/lib3to6", "keywords": "six lib2to3 astor ast", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "lib3to6", "package_url": "https://pypi.org/project/lib3to6/", "platform": "", "project_url": "https://pypi.org/project/lib3to6/", "project_urls": { "Homepage": "https://gitlab.com/mbarkhau/lib3to6" }, "release_url": "https://pypi.org/project/lib3to6/201902.30/", "requires_dist": [ "pathlib2", "astor", "typing", "click" ], "requires_python": ">=3.6", "summary": "Compile Python 3.6+ code to Python 2.7+", "version": "201902.30" }, "last_serial": 4855982, "releases": { "201902.30": [ { "comment_text": "", "digests": { "md5": "116fcaf00263ba607bbd559796fd21da", "sha256": "b2229862f23bdc3b1c002fbef733f75df56942ee654f518640a4a80349ffc4bd" }, "downloads": -1, "filename": "lib3to6-201902.30-py36.py37-none-any.whl", "has_sig": false, "md5_digest": "116fcaf00263ba607bbd559796fd21da", "packagetype": "bdist_wheel", "python_version": "py36.py37", "requires_python": ">=3.6", "size": 26277, "upload_time": "2019-02-22T21:41:36", "url": "https://files.pythonhosted.org/packages/27/46/6359288b8283b7e045cc7a576ac0f85b59ec232e90ba4262853b322117b6/lib3to6-201902.30-py36.py37-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6be2c2edaa4dd730730318be82239e26", "sha256": "561e06285dd6a676deb0ab09094a97867e3c668ed583eba079721794315e817d" }, "downloads": -1, "filename": "lib3to6-201902.30.tar.gz", "has_sig": false, "md5_digest": "6be2c2edaa4dd730730318be82239e26", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 38834, "upload_time": "2019-02-22T21:41:38", "url": "https://files.pythonhosted.org/packages/2a/be/e0b8f6c36a356d12359e4b2d0282b890a7182d1baef0abcd4cf2225867ee/lib3to6-201902.30.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "116fcaf00263ba607bbd559796fd21da", "sha256": "b2229862f23bdc3b1c002fbef733f75df56942ee654f518640a4a80349ffc4bd" }, "downloads": -1, "filename": "lib3to6-201902.30-py36.py37-none-any.whl", "has_sig": false, "md5_digest": "116fcaf00263ba607bbd559796fd21da", "packagetype": "bdist_wheel", "python_version": "py36.py37", "requires_python": ">=3.6", "size": 26277, "upload_time": "2019-02-22T21:41:36", "url": "https://files.pythonhosted.org/packages/27/46/6359288b8283b7e045cc7a576ac0f85b59ec232e90ba4262853b322117b6/lib3to6-201902.30-py36.py37-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6be2c2edaa4dd730730318be82239e26", "sha256": "561e06285dd6a676deb0ab09094a97867e3c668ed583eba079721794315e817d" }, "downloads": -1, "filename": "lib3to6-201902.30.tar.gz", "has_sig": false, "md5_digest": "6be2c2edaa4dd730730318be82239e26", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 38834, "upload_time": "2019-02-22T21:41:38", "url": "https://files.pythonhosted.org/packages/2a/be/e0b8f6c36a356d12359e4b2d0282b890a7182d1baef0abcd4cf2225867ee/lib3to6-201902.30.tar.gz" } ] }