TYP: Type annotations in pandas/io/formats/style.py #30403

ShaharNaveh · 2019-12-22T13:19:42Z

pandas/io/formats/style.py

jbrockmendel · 2019-12-22T16:18:52Z

pandas/io/formats/style.py

@@ -1459,7 +1487,7 @@ def pipe(self, func, *args, **kwargs):
        return com.pipe(self, func, *args, **kwargs)


-def _is_visible(idx_row, idx_col, lengths):
+def _is_visible(idx_row, idx_col, lengths) -> bool:


can the inputs by typed?

The problem is that mypy is giving false positives, I typed the 'row' as str and the 'col' as int and it went through with no errors.

pandas/io/formats/style.py

jbrockmendel · 2019-12-22T16:24:44Z

pandas/io/formats/style.py

@@ -879,7 +891,7 @@ def set_caption(self, caption):
        self.caption = caption
        return self

-    def set_table_styles(self, table_styles):
+    def set_table_styles(self, table_styles: List) -> "Styler":


can we say anything about what goes in the List?

(same question for other occurrences of List above)

The doc type says it's should be List
but the description says it should be Dict.

Or I misunderstood (more likely).

Each individual table_style should be a dictionary with 'selector' and 'props' keys.

When trying to annotate it as List[str]

mypy gives this error:

pandas/io/formats/style.py:861: error: Argument 1 to "extend" of "list" has incompatible type "List[str]"; expected "Iterable[Tuple[Callable[..., Any], Tuple[Any, ...], Dict[Any, Any]]]" Found 1 error in 1 file (checked 794 source files)

expected "Iterable[Tuple[Callable[..., Any], Tuple[Any, ...], Dict[Any, Any]]]"

Maybe I should leave it unannotated, or leave it as List, or put Any just in case?

jbrockmendel · 2019-12-22T16:25:34Z

pandas/io/formats/style.py

@@ -849,7 +861,7 @@ def use(self, styles):
        self._todo.extend(styles)
        return self

-    def set_uuid(self, uuid):
+    def set_uuid(self, uuid: str) -> "Styler":


i dont actually use this code, is the uuid here going to be specifically a hex str? if so, we should look into whether there is a more specific annotatiton for that

I think the mechanism for this would be NewType, https://mypy.readthedocs.io/en/latest/more_types.html#newtypes

@simonjayhawkins Don't you think that's a bit of an overkill?
By overkill I mean, creating a new class just to annotate one variable in one function, just seems too much.

jbrockmendel · 2019-12-22T16:26:59Z

pandas/io/formats/style.py

@@ -562,7 +573,7 @@ def _update_ctx(self, attrs):
                for pair in col.rstrip(";").split(";"):
                    self.ctx[(i, j)].append(pair)

-    def _copy(self, deepcopy=False):
+    def _copy(self, deepcopy: bool = False):


elsewhere we use "deep" instead of "deepcopy". that wouldnt be something to change here, but might be worth looking into standardizing.

pandas/io/formats/style.py

WillAyd

I think can add a few more looking at docstring

pandas/io/formats/style.py

ShaharNaveh · 2019-12-23T10:26:06Z

Thank you for the code suggestions @WillAyd

simonjayhawkins

Thanks @MomIsBestFriend lgtm. another pass (or two) with mypy --disallow-any-generics to add type parameters to the generics as suggested by @jbrockmendel and mypy --disallow-incomplete-defs to fill in some of the missing annotations would be great. @WillAyd

ShaharNaveh · 2019-12-24T15:09:34Z

pandas/io/formats/style.py

@@ -422,7 +433,7 @@ def format_attr(pair):
            table_attributes=table_attr,
        )

-    def format(self, formatter, subset=None, na_rep: Optional[str] = None):
+    def format(self, formatter, subset=None, na_rep: Optional[str] = None) -> "Styler":


@simonjayhawkins Do you have any suggestion on how to annotate subset?

Or in other words how to represent IndexSlice in a typing form?

from pandas.core.indexing import _IndexSlice (maybe in an if TYPE_CHECKING although not necessary)

Suggested change

def format(self, formatter, subset=None, na_rep: Optional[str] = None) -> "Styler":

def format(self, formatter, subset: Optional[_IndexSlice] = None, na_rep: Optional[str] = None) -> "Styler":

mypy is raising errors.

I stopped after a few, the error messages got pretty linear.

Any suggestions @simonjayhawkins ?

MYPY's logs:

For

subset: Optional[_IndexSlice]

$ mypy pandas pandas/io/formats/style.py:486: error: Argument 1 to "len" has incompatible type "Optional[_IndexSlice]"; expected "Sized" pandas/io/formats/style.py:487: error: Incompatible types in assignment (expression has type "Tuple[Optional[_IndexSlice], Any]", variable has type "Optional[_IndexSlice]") Found 2 errors in 1 file (checked 844 source files)

For

subset: Tuple[Optional[_IndexSlice], Any]

$ mypy pandas pandas/io/formats/style.py:436: error: Incompatible default for argument "subset" (default has type "None", argument has type "Tuple[Optional[_IndexSlice], Any]") pandas/io/formats/style.py:487: error: Incompatible types in assignment (expression has type "Tuple[Tuple[Optional[_IndexSlice], Any], Any]", variable has type "Tuple[Optional[_IndexSlice], Any]") Found 2 errors in 1 file (checked 844 source files)

For

subset: Tuple[Tuple[Optional[_IndexSlice], Any], Any]

$ mypy pandas pandas/io/formats/style.py:436: error: Incompatible default for argument "subset" (default has type "None", argument has type "Tuple[Tuple[Optional[_IndexSlice], Any], Any]") pandas/io/formats/style.py:487: error: Incompatible types in assignment (expression has type "Tuple[Tuple[Tuple[Optional[_IndexSlice], Any], Any], Any]", variable has type "Tuple[Tuple[Optional[_IndexSlice], Any], Any]") Found 2 errors in 1 file (checked 844 source files)

For

subset: Tuple[Tuple[Tuple[Optional[_IndexSlice], Any], Any], Any]

$ mypy pandas pandas/io/formats/style.py:436: error: Incompatible default for argument "subset" (default has type "None", argument has type "Tuple[Tuple[Tuple[Optional[_IndexSlice], Any], Any], Any]") pandas/io/formats/style.py:487: error: Incompatible types in assignment (expression has type "Tuple[Tuple[Tuple[Tuple[Optional[_IndexSlice], Any], Any], Any], Any]", variable has type "Tuple[Tuple[Tuple[Optional[_IndexSlice], Any], Any], Any]") Found 2 errors in 1 file (checked 844 source files)

pandas/io/formats/style.py

WillAyd · 2019-12-26T14:28:18Z

pandas/io/formats/style.py

            return "{key}={value}".format(**pair)

        # for sparsifying a MultiIndex
        idx_lengths = _get_level_lengths(self.index)
        col_lengths = _get_level_lengths(self.columns, hidden_columns)

-        cell_context = dict()
+        cell_context: Dict = dict()


mypy doesn't complain for what ever I put in the subtypes.

Anyway, I don't see cell_context being inserted with any values but in few places .get() gets called on it, I am very confused.

Can you specify what the complaint is?

No, I tried both with stubgen and MonkeyType and they didn't help. Also there is not really a good way to know if the types are actually good or not because what no matter what I put in it mypy won't complain.

Plus, this annotation really doesn't add that much, so I dropped it.

Unless you have a different opinion about dropping that annotation.

WillAyd · 2019-12-26T14:28:49Z

pandas/io/formats/style.py

@@ -611,7 +624,9 @@ def _compute(self):
            r = func(self)(*args, **kwargs)
        return r

-    def _apply(self, func, axis=0, subset=None, **kwargs):
+    def _apply(
+        self, func: Callable, axis: Optional[Axis] = 0, subset=None, **kwargs


Can the args or return type of Callable be annotated?

Only the retrun type (from what I know)

Can you add return types then? Still very helpful

WillAyd · 2019-12-26T14:28:57Z

pandas/io/formats/style.py

@@ -644,7 +659,9 @@ def _apply(self, func, axis=0, subset=None, **kwargs):
        self._update_ctx(result)
        return self

-    def apply(self, func, axis=0, subset=None, **kwargs):
+    def apply(
+        self, func: Callable, axis: Optional[Axis] = 0, subset=None, **kwargs


Same question on Callable

Only the retrun type (from what I know)

jreback · 2019-12-27T16:31:46Z

needs a rebase

ShaharNaveh · 2020-01-16T00:19:50Z

The errors were fixed.

WillAyd

Looks pretty good. We recently added Label to pandas._typing so should use that here in a few places (think I commented on all)

pandas/io/formats/excel.py

pandas/io/formats/style.py

WillAyd · 2020-01-16T00:22:46Z

pandas/io/formats/style.py

            return "{key}={value}".format(**pair)

        # for sparsifying a MultiIndex
        idx_lengths = _get_level_lengths(self.index)
        col_lengths = _get_level_lengths(self.columns, hidden_columns)

-        cell_context = dict()
+        cell_context: Dict = dict()


Can you specify what the complaint is?

pandas/io/formats/style.py

As WillAyd suggested

ShaharNaveh · 2020-01-16T01:25:07Z

(think I commented on all)

You have :)

WillAyd · 2020-01-16T20:44:44Z

pandas/io/formats/style.py

@@ -545,7 +555,7 @@ def render(self, **kwargs):
        d.update(kwargs)
        return self.template.render(**d)

-    def _update_ctx(self, attrs):
+    def _update_ctx(self, attrs: FrameOrSeries) -> None:


Does attrs here not have to be a DataFrame? Looks like it calls iterrows in the implementation which isn't defined for Series

@WillAyd
Then the docstring is not correct.
I fixed both the type annotation and the docstring in 6a06e50

WillAyd

lgtm @MomIsBestFriend can you merge master one more time to be sure?

@simonjayhawkins over to you

ShaharNaveh · 2020-01-18T11:53:19Z

Rebased

(unrelated but important)

@WillAyd have you saw what I wrote here?

WillAyd · 2020-01-20T17:31:16Z

Thanks @MomIsBestFriend

ShaharNaveh marked this pull request as ready for review December 22, 2019 13:58