.. _extending-mypy:

Extending and integrating mypy
==============================

.. _integrating-mypy:

Integrating mypy into another Python application
************************************************

It is possible to integrate mypy into another Python 3 application by
importing ``mypy.api`` and calling the ``run`` function with a parameter of type ``List[str]``, containing
what normally would have been the command line arguments to mypy.

Function ``run`` returns a ``Tuple[str, str, int]``, namely
``(<normal_report>, <error_report>, <exit_status>)``, in which ``<normal_report>``
is what mypy normally writes to ``sys.stdout``, ``<error_report>`` is what mypy
normally writes to ``sys.stderr`` and ``exit_status`` is the exit status mypy normally
returns to the operating system.

A trivial example of using the api is the following

.. code-block:: python

    import sys
    from mypy import api

    result = api.run(sys.argv[1:])

    if result[0]:
        print('\nType checking report:\n')
        print(result[0])  # stdout

    if result[1]:
        print('\nError report:\n')
        print(result[1])  # stderr

    print ('\nExit status:', result[2])

Extending mypy using plugins
****************************

Python is a highly dynamic language and has extensive metaprogramming
capabilities. Many popular libraries use these to create APIs that may
be more flexible and/or natural for humans, but are hard to express using
static types. Extending the PEP 484 type system to accommodate all existing
dynamic patterns is impractical and often just impossible.

Mypy supports a plugin system that lets you customize the way mypy type checks
code. This can be useful if you want to extend mypy so it can type check code
that uses a library that is difficult to express using just PEP 484 types.

The plugin system is focused on improving mypy's understanding
of *semantics* of third party frameworks. There is currently no way to define
new first class kinds of types.

.. note::

   The plugin system is experimental and prone to change. If you want to write
   a mypy plugin, we recommend you start by contacting the mypy core developers
   on `gitter <https://gitter.im/python/typing>`_. In particular, there are
   no guarantees about backwards compatibility. Backwards incompatible changes
   may be made without a deprecation period.

Configuring mypy to use plugins
*******************************

Plugins are Python files that can be specified in a mypy
:ref:`config file <config-file>` using one of the two formats: relative or
absolute path to the plugin to the plugin file, or a module name (if the plugin
is installed using ``pip install`` in the same virtual environment where mypy
is running). The two formats can be mixed, for example:

.. code-block:: ini

    [mypy]
    plugins = /one/plugin.py, other.plugin

Mypy will try to import the plugins and will look for an entry point function
named ``plugin``. If the plugin entry point function has a different name, it
can be specified after colon:

.. code-block:: ini

    [mypy]
    plugins = custom_plugin:custom_entry_point

In following sections we describe basics of the plugin system with
some examples. For more technical details please read docstrings in
`mypy/plugin.py <https://github.com/python/mypy/blob/master/mypy/plugin.py>`_
in mypy source code. Also you can find good examples in the bundled plugins
located in `mypy/plugins <https://github.com/python/mypy/tree/master/mypy/plugins>`_.

High-level overview
*******************

Every entry point function should accept a single string argument
that is a full mypy version and return a subclass of ``mypy.plugins.Plugin``:

.. code-block:: python

   from mypy.plugin import Plugin

   class CustomPlugin(Plugin):
       def get_type_analyze_hook(self, fullname: str):
           # see explanation below
           ...

   def plugin(version: str):
       # ignore version argument if the plugin works with all mypy versions.
       return CustomPlugin

During different phases of analyzing the code (first in semantic analysis,
and then in type checking) mypy calls plugin methods such as
``get_type_analyze_hook()`` on user plugins. This particular method for example
can return a callback that mypy will use to analyze unbound types with given
full name. See full plugin hook methods list :ref:`below <plugin_hooks>`.

Mypy maintains a list of plugins it gets from the config file plus the default
(built-in) plugin that is always enabled. Mypy calls a method once for each
plugin in the list until one of the methods returns a non-``None`` value.
This callback will be then used to customize the corresponding aspect of
analyzing/checking the current abstract syntax tree node.

The callback returned by the ``get_xxx`` method will be given a detailed
current context and an API to create new nodes, new types, emit error messages
etc., and the result will be used for further processing.

Plugin developers should ensure that their plugins work well in incremental and
daemon modes. In particular, plugins should not hold global state due to caching
of plugin hook results.

.. _plugin_hooks:

Current list of plugin hooks
****************************

**get_type_analyze_hook()** customizes behaviour of the type analyzer.
For example, PEP 484 doesn't support defining variadic generic types:

.. code-block:: python

   from lib import Vector

   a: Vector[int, int]
   b: Vector[int, int, int]

When analyzing this code, mypy will call ``get_type_analyze_hook("lib.Vector")``,
so the plugin can return some valid type for each variable.

**get_function_hook()** is used to adjust the return type of a function call.
This is a good choice if the return type of some function depends on *values*
of some arguments that can't be expressed using literal types (for example
a function may return an ``int`` for positive arguments and a ``float`` for
negative arguments). This hook will be also called for instantiation of classes.
For example:

.. code-block:: python

   from contextlib import contextmanager
   from typing import TypeVar, Callable

   T = TypeVar('T')

   @contextmanager  # built-in plugin can infer a precise type here
   def stopwatch(timer: Callable[[], T]) -> Iterator[T]:
       ...
       yield timer()

**get_method_hook()** is the same as ``get_function_hook()`` but for methods
instead of module level functions.

**get_method_signature_hook()** is used to adjust the signature of a method.
This includes special Python methods except ``__init__()`` and ``__new__()``.
For example in this code:

.. code-block:: python

   from ctypes import Array, c_int

   x: Array[c_int]
   x[0] = 42

mypy will call ``get_method_signature_hook("ctypes.Array.__setitem__")``
so that the plugin can mimic the ``ctypes`` auto-convert behavior.

**get_attribute_hook** can be used to give more precise type of an instance
attribute. Note however, that this method is only called for variables that
already exist in the class symbol table. If you want to add some generated
variables/methods to the symbol table you can use one of the three hooks
below.

**get_class_decorator_hook()** can be used to update class definition for
given class decorators. For example, you can add some attributes to the class
to match runtime behaviour:

.. code-block:: python

   from lib import customize

   @customize
   class UserDefined:
       pass

   var = UserDefined
   var.customized  # mypy can understand this using a plugin

**get_metaclass_hook()** is similar to above, but for metaclasses.

**get_base_class_hook()** is similar to above, but for base classes.

**get_dynamic_class_hook()** can be used to allow dynamic class definitions
in mypy. This plugin hook is called for every assignment to a simple name
where right hand side is a function call:

.. code-block:: python

   from lib import dynamic_class

   X = dynamic_class('X', [])

For such definition, mypy will call ``get_dynamic_class_hook("lib.dynamic_class")``.
The plugin should create the corresponding ``mypy.nodes.TypeInfo`` object, and
place it into a relevant symbol table. (Instances of this class represent
classes in mypy and hold essential information such as qualified name,
method resolution order, etc.)

**get_customize_class_mro_hook()** can be used to modify class MRO (for example
insert some entries there) before the class body is analyzed.
