Release v2.6.0

Author: sco1

CHANGELOG.md

@@ -1,9 +1,17 @@
 # Changelog
 Versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html) (`<major>`.`<minor>`.`<patch>`)
 
+## [v2.6.0]
+### Added
+* #98 Add `--dispatch-decorators` to support suppression of all errors from functions decorated by decorators such as `functools.singledispatch` and `functools.singledispatchmethod`.
+* #99 Add `--overload-decorators` to support generic aliasing of the `typing.overload` decorator.
+
+### Fixed
+* #106 Fix incorrect parsing of multiline docstrings with less than two lines of content, causing incorrect line numbers for yielded errors in Python versions prior to 3.8
+
 ## [v2.5.0]
 ### Added
-* #103 add `--allow-untyped-nested` to suppress all errors from dynamically typted nested functions. A function is considered dynamically typed if it does not contain any type hints.
+* #103 Add `--allow-untyped-nested` to suppress all errors from dynamically typted nested functions. A function is considered dynamically typed if it does not contain any type hints.
 
 ## [v2.4.1]
 ### Fixed

README.md

@@ -1,8 +1,11 @@
 # flake8-annotations
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/flake8-annotations)](https://pypi.org/project/flake8-annotations/)
 [![PyPI](https://img.shields.io/pypi/v/flake8-annotations)](https://pypi.org/project/flake8-annotations/)
+[![PyPI - License](https://img.shields.io/pypi/l/flake8-annotations?color=magenta)](https://github.com/sco1/flake8-annotations/blob/master/LICENSE)
+[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-black)](https://github.com/psf/black)
 
-`flake8-annotations` is a plugin for [Flake8](http://flake8.pycqa.org/en/latest/) that detects the absence of [PEP 3107-style](https://www.python.org/dev/peps/pep-3107/) function annotations and [PEP 484-style](https://www.python.org/dev/peps/pep-0484/#type-comments) type comments  (see: [Caveats](#Caveats-for-PEP-484-style-Type-Comments)).
+`flake8-annotations` is a plugin for [Flake8](http://flake8.pycqa.org/en/latest/) that detects the absence of [PEP 3107-style](https://www.python.org/dev/peps/pep-3107/) function annotations and [PEP 484-style](https://www.python.org/dev/peps/pep-0484/#type-comments) type comments (see: [Caveats](#Caveats-for-PEP-484-style-Type-Comments)).
 
 What this won't do: Check variable annotations (see: [PEP 526](https://www.python.org/dev/peps/pep-0526/)), respect stub files, or replace [mypy](http://mypy-lang.org/).
 
@@ -19,7 +22,7 @@ You can verify it's being picked up by invoking the following in your shell:
 
 ```bash
 $ flake8 --version
-3.8.4 (flake8-annotations: 2.5.0, mccabe: 0.6.1, pycodestyle: 2.6.0, pyflakes: 2.2.0) CPython 3.9.0 on Darwin
+3.8.4 (flake8-annotations: 2.6.0, mccabe: 0.6.1, pycodestyle: 2.6.0, pyflakes: 2.2.0) CPython 3.9.0 on Darwin
 ```
 
 ## Table of Warnings
@@ -86,6 +89,61 @@ Allow omission of a return type hint for `__init__` if at least one argument is
 
 Default: `False`
 
+### `--dispatch-decorators`: `list[str]`
+Comma-separated list of decorators flake8-annotations should consider as dispatch decorators. Linting errors are suppressed for functions decorated with at least one of these functions.
+
+Decorators are matched based on their attribute name. For example, `"singledispatch"` will match any of the following:
+  * `import functools; @functools.singledispatch`
+  * `import functools as fnctls; @fnctls.singledispatch`
+  * `from functools import singledispatch; @singledispatch`
+
+**NOTE:** Deeper imports, such as `a.b.singledispatch` are not supported.
+
+See: [Generic Functions](#generic-functions) for additional information.
+
+Default: `"singledispatch, singledispatchmethod"`
+
+### `--overload-decorators`: `list[str]`
+Comma-separated list of decorators flake8-annotations should consider as [`typing.overload`](https://docs.python.org/3/library/typing.html#typing.overload) decorators.
+
+Decorators are matched based on their attribute name. For example, `"overload"` will match any of the following:
+  * `import typing; @typing.overload`
+  * `import typing as t; @t.overload`
+  * `from typing import overload; @overload`
+
+**NOTE:** Deeper imports, such as `a.b.overload` are not supported.
+
+See: [The `typing.overload` Decorator](#the-typingoverload-decorator) for additional information.
+
+Default: `"overload"`
+
+
+## Generic Functions
+Per the Python Glossary, a [generic function](https://docs.python.org/3/glossary.html#term-generic-function) is defined as:
+
+> A function composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm.
+
+In the standard library we have some examples of decorators for implementing these generic functions: [`functools.singledispatch`](https://docs.python.org/3/library/functools.html#functools.singledispatch) and [`functools.singledispatchmethod`](https://docs.python.org/3/library/functools.html#functools.singledispatchmethod). In the spirit of the purpose of these decorators, errors for missing annotations for functions decorated with at least one of these are ignored.
+
+For example, this code:
+
+```py
+import functools
+
+@functools.singledispatch
+def foo(a):
+    print(a)
+
+@foo.register
+def _(a: list) -> None:
+    for idx, thing in enumerate(a):
+        print(idx, thing)
+```
+
+Will not raise any linting errors for `foo`.
+
+Decorator(s) to treat as defining generic functions may be specified by the [`--dispatch-decorators`](#--dispatch-decorators-liststr) configuration option.
+
 ## The `typing.overload` Decorator
 Per the [`typing`](https://docs.python.org/3/library/typing.html#typing.overload) documentation:
 
@@ -109,7 +167,7 @@ def foo(a):
 
 Will not raise linting errors for missing annotations for the arguments & return of the non-decorated `foo` definition.
 
-**NOTE:** If importing directly, the `typing.overload` decorator will not be recognized if it is imported with an alias (e.g. `from typing import overload as please_dont_do_this`). Aliasing of the `typing` module is supported (e.g. `import typing as t; @t.overload`).
+Decorator(s) to treat as `typing.overload` may be specified by the [`--overload-decorators`](#--overload-decorators-liststr) configuration option.
 
 ## Caveats for PEP 484-style Type Comments
 

flake8_annotations/__init__.py

@@ -17,23 +17,34 @@
 
     PY_GTE_38 = False
 
-__version__ = "2.5.0"
+__version__ = "2.6.0"
 
 # The order of AST_ARG_TYPES must match Python's grammar
 # See: https://docs.python.org/3/library/ast.html#abstract-grammar
-AST_ARG_TYPES = ("args", "vararg", "kwonlyargs", "kwarg")
+AST_ARG_TYPES: Tuple[str, ...] = ("args", "vararg", "kwonlyargs", "kwarg")
 if PY_GTE_38:
     # Positional-only args introduced in Python 3.8
     # If posonlyargs are present, they will be before other argument types
     AST_ARG_TYPES = ("posonlyargs",) + AST_ARG_TYPES
 
 AST_FUNCTION_TYPES = Union[ast.FunctionDef, ast.AsyncFunctionDef]
 AST_DEF_NODES = Union[ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef]
+AST_DECORATOR_NODES = Union[ast.Attribute, ast.Call, ast.Name]
 
 
 class Argument:
     """Represent a function argument & its metadata."""
 
+    __slots__ = [
+        "argname",
+        "lineno",
+        "col_offset",
+        "annotation_type",
+        "has_type_annotation",
+        "has_3107_annotation",
+        "has_type_comment",
+    ]
+
     def __init__(
         self,
         argname: str,
@@ -102,6 +113,21 @@ class Function:
     aligns with ast's naming convention.
     """
 
+    __slots__ = [
+        "name",
+        "lineno",
+        "col_offset",
+        "function_type",
+        "is_class_method",
+        "class_decorator_type",
+        "is_return_annotated",
+        "has_type_comment",
+        "has_only_none_returns",
+        "is_nested",
+        "decorator_list",
+        "args",
+    ]
+
     def __init__(
         self,
         name: str,
@@ -113,8 +139,8 @@ def __init__(
         is_return_annotated: bool = False,
         has_type_comment: bool = False,
         has_only_none_returns: bool = True,
-        is_overload_decorated: bool = False,
         is_nested: bool = False,
+        decorator_list: List[AST_DECORATOR_NODES] = None,
         args: List[Argument] = None,
     ):
         self.name = name
@@ -126,8 +152,8 @@ def __init__(
         self.is_return_annotated = is_return_annotated
         self.has_type_comment = has_type_comment
         self.has_only_none_returns = has_only_none_returns
-        self.is_overload_decorated = is_overload_decorated
         self.is_nested = is_nested
+        self.decorator_list = decorator_list
         self.args = args
 
     def is_fully_annotated(self) -> bool:
@@ -142,14 +168,57 @@ def is_dynamically_typed(self) -> bool:
         """Determine if the function is dynamically typed, defined as completely lacking hints."""
         return not any(arg.has_type_annotation for arg in self.args)
 
-    def get_missed_annotations(self) -> List:
+    def get_missed_annotations(self) -> List[Argument]:
         """Provide a list of arguments with missing type annotations."""
         return [arg for arg in self.args if not arg.has_type_annotation]
 
-    def get_annotated_arguments(self) -> List:
+    def get_annotated_arguments(self) -> List[Argument]:
         """Provide a list of arguments with type annotations."""
         return [arg for arg in self.args if arg.has_type_annotation]
 
+    def has_decorator(self, check_decorators: Set[str]) -> bool:
+        """
+        Determine whether the function node is decorated by any of the provided decorators.
+
+        Decorator matching is done against the provided `check_decorators` set, allowing the user
+        to specify any expected aliasing in the relevant flake8 configuration option. Decorators are
+        assumed to be either a module attribute (e.g. `@typing.overload`) or name
+        (e.g. `@overload`). For the case of a module attribute, only the attribute is checked
+        against `overload_decorators`.
+
+        NOTE: Deeper decorator imports (e.g. `a.b.overload`) are not explicitly supported
+        """
+        for decorator in self.decorator_list:
+            # Drop to a helper to allow for simpler handling of callable decorators
+            return self._decorator_checker(decorator, check_decorators)
+        else:
+            return False
+
+    def _decorator_checker(
+        self, decorator: AST_DECORATOR_NODES, check_decorators: Set[str]
+    ) -> bool:
+        """
+        Check the provided decorator for a match against the provided set of check names.
+
+        Decorators are assumed to be of the following form:
+            * `a.name` or `a.name()`
+            * `name` or `name()`
+
+        NOTE: Deeper imports (e.g. `a.b.name`) are not explicitly supported.
+        """
+        if isinstance(decorator, ast.Name):
+            # e.g. `@overload`, where `decorator.id` will be the name
+            if decorator.id in check_decorators:
+                return True
+        elif isinstance(decorator, ast.Attribute):
+            # e.g. `@typing.overload`, where `decorator.attr` will be the name
+            if decorator.attr in check_decorators:
+                return True
+        elif isinstance(decorator, ast.Call):  # pragma: no branch
+            # e.g. `@overload()` or `@typing.overload()`, where `decorator.func` will be `ast.Name`
+            # or `ast.Attribute`, which we can check recursively
+            return self._decorator_checker(decorator.func, check_decorators)
+
     def __str__(self) -> str:
         """
         Format the Function object into a readable representation.
@@ -176,8 +245,8 @@ def __repr__(self) -> str:
             f"is_return_annotated={self.is_return_annotated}, "
             f"has_type_comment={self.has_type_comment}, "
             f"has_only_none_returns={self.has_only_none_returns}, "
-            f"is_overload_decorated={self.is_overload_decorated}, "
             f"is_nested={self.is_nested}, "
+            f"decorator_list={self.decorator_list}, "
             f"args={self.args}"
             ")"
         )
@@ -194,7 +263,6 @@ def from_function_node(cls, node: AST_FUNCTION_TYPES, lines: List[str], **kwargs
         following kwargs will be overridden:
           * function_type
           * class_decorator_type
-          * is_overload_decorated
           * args
         """
         # Extract function types from function name
@@ -204,8 +272,8 @@ def from_function_node(cls, node: AST_FUNCTION_TYPES, lines: List[str], **kwargs
         if kwargs.get("is_class_method", False):
             kwargs["class_decorator_type"] = cls.get_class_decorator_type(node)
 
-        # Check for `typing.overload` decorator
-        kwargs["is_overload_decorated"] = cls.has_overload_decorator(node)
+        # Store raw decorator list for use by property methods
+        kwargs["decorator_list"] = node.decorator_list
 
         new_function = cls(node.name, node.lineno, node.col_offset, **kwargs)
 
@@ -261,25 +329,25 @@ def colon_seeker(node: AST_FUNCTION_TYPES, lines: List[str]) -> Tuple[int, int]:
         if node.lineno == node.body[0].lineno:
             return Function._single_line_colon_seeker(node, lines[node.lineno - 1])
 
-        def_end_lineno = node.body[0].lineno - 1
-
         # With Python < 3.8, the function node includes the docstring & the body does not, so
         # we have rewind through any docstrings, if present, before looking for the def colon
+        # We should end up with lines[def_end_lineno - 1] having the colon
+        def_end_lineno = node.body[0].lineno
         if not PY_GTE_38:
-            # This list index is a little funky, since we've already subtracted 1 outside of this
-            # context, we can leave it as-is since it will index the list to the line prior to where
-            # the function node's body begins.
             # If the docstring is on one line then no rewinding is necessary.
-            n_triple_quotes = lines[def_end_lineno].count('"""')
+            n_triple_quotes = lines[def_end_lineno - 1].count('"""')
             if n_triple_quotes == 1:  # pragma: no branch
                 # Docstring closure, rewind until the opening is found & take the line prior
                 while True:
                     def_end_lineno -= 1
                     if '"""' in lines[def_end_lineno - 1]:
                         # Docstring has closed
-                        def_end_lineno -= 1
                         break
 
+        # Once we've gotten here, we've found the line where the docstring begins, so we have
+        # to step up one more line to get to the close of the def
+        def_end_lineno -= 1
+
         # Use str.rfind() to account for annotations on the same line, definition closure should
         # be the last : on the line
         def_end_col_offset = lines[def_end_lineno - 1].rfind(":") + 1
@@ -407,29 +475,6 @@ def get_class_decorator_type(
         else:
             return None
 
-    @staticmethod
-    def has_overload_decorator(function_node: AST_FUNCTION_TYPES) -> bool:
-        """
-        Determine whether the provided function node is decorated by `typing.overload`.
-
-        NOTE: For simplicity, this check will not identify the `typing.overload` decorator if it has
-        been aliased (e.g. `from typing import overload as please_dont_do_this`).
-        """
-        # Depending on how the decorator is imported, it will appear in the function node's
-        # decorator list as an instance of either `ast.Name` (e.g. `@overload`) or `ast.Attribute`
-        # (e.g. `@typing.overload`)
-        # It assumed that the overload decorator will never be present as a callable
-        # (e.g. `@overload()`)
-        for decorator in function_node.decorator_list:
-            if isinstance(decorator, ast.Name):
-                if decorator.id == "overload":
-                    return True
-            elif isinstance(decorator, ast.Attribute):
-                if decorator.attr == "overload":
-                    return True
-        else:
-            return False
-
 
 class FunctionVisitor(ast.NodeVisitor):
     """An ast.NodeVisitor instance for walking the AST and describing all contained functions."""