DOC: docstring validation script improvements

[jorisvandenbossche]

Based on the experience from the sprint, some known issues:

• args, kwargs (or kwds) issue (fixed in Add tests for docstring Validation Script + py27 compat #20061)
• no punctuation needed at end of explanation it last line is a versionadded (Consider directives when validating docstrings parameters #22405)
• not working for some methods that are not imported in pandas like pandas.core.resample.Resampler.backfill
• parameter description starting with bullet point (starting with capital letter error)
• parameter description containing bullet points at the end (missing ending period error) (Fix Validation Script to Not Error Out if Bullet Points Don't End with Period #25461)
• define standard for lists in parameters, enforce the format, and remove error that description needs to end with a period when there is a list
• handle newlines in docstring text (eg in parameter description)
• verify order of the sections (e.g. See Also before Examples...) (DOC: Validate the order of the sections in the docstrings #23133)
• verify that pandas and numpy are not being imported (DOC: Validate that the docstrings do not import pandas or numpy #23134)
• verify that there is no blank line after the title sections
• See Also items do not start with pandas. (DOC: Validate that items in See Also section do not start by pandas. #23136)
• Using string, boolean, integer in the param descriptions (should be str, bool, int) (DOC: Validate parameter types in docstrings (e.g. str instead of string) #23165)
• Returns section: first line with type only if a single return, desc starting with capital and ending with period... (DOC: Validate the Returns section in the docstrings #23138)
• flake8 examples (Validate PEP-8 in docstring examples #23154)
• axis parameter using the same format {0 or 'index', 1 or 'columns'}
• add validations from decisions made in DOC: further clarifications to docstring guide #20309 (quotes to use...)
• decide whether to use NaN, nan, NA... and warn if the others are found
• define standard on when to use backticks (single or double) and quotes, and verify common cases (like None, NaN...)
• Decription of items in See Also starts with capital letter and ends with period. (DOC: Validate description of See Also section in docstrings #23135)

cc @TomAugspurger @datapythonista @WillAyd others that you noticed?

[TomAugspurger]

There were some like

param : {'a', 'b'}
    - a : do this
    - b : do that

that complained about the explanation not starting with a capital letter. Maybe we're OK with that to force a general explanation before going into individual params.

Tangentially related: whitespace normalization for doctests (#20284)

[TomAugspurger]

Under the see also, we could check for and error if they have pandas.DataFrame.apply instead of DataFrame.apply.

It'd be nice to somehow know if the target is valid (a decent amount of DataFrame.groupby.head-like references, but perhaps that's expensive to check?

[jorisvandenbossche]

I think we can easily check if the first character is a - (sometimes adding such a first sentence can be superfluous, and you then have to leave a blank line)

Tangentially related: whitespace normalization for doctests (#20284)

We need to do this for pytest, but it is already handled separately in the validation_script

[datapythonista]

If I'm not wrong, the validation wasn't checking much on the Returns section, at least I've reviewed many PRs without description, or with name+type...

CCing @kynan and @mariocj89, if I'm not wrong they did some work on the kwargs problem during the sprint.

[datapythonista]

Also, I think it could be good to warn if the user is importing pandas or numpy in the examples, or if uses string, boolean, integer, or list-like in the type descriptions.

I can take care of this ticket if nobody is working on it already, I've got couple of fixes to the generation of the csv that I didn't send it yet.

[jorisvandenbossche]

Yes, certainly go ahead with doing a PR

[WillAyd]

@datapythonista if it helps I started on a test for the validation script in #20061 - I can put some time into wrapping that up today so we can be sure we don't fix one thing and break another with this going forward

[TomAugspurger]

The section order should be validated. I believe there's a numpydoc issue about making the order required.

[TomAugspurger]

Would be nice to extract the code examples and pipe them through flake8 if possible.

[jorisvandenbossche]

Would be nice to extract the code examples and pipe them through flake8 if possible.

Yes, that would be a nice one!

[TomAugspurger]

I was looking a bit, apparently flake8 isn't easily called from Python: http://flake8.pycqa.org/en/latest/user/python-api.html Shelling out seems fragile.

…

On Mon, Mar 12, 2018 at 3:11 PM, Joris Van den Bossche < ***@***.***> wrote: Would be nice to extract the code examples and pipe them through flake8 if possible. Yes, that would be a nice one! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20298 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIjvW7KHogiwqWJQER531NyCqsj_Sks5tdtaMgaJpZM4SmiYl> .

[datapythonista]

@WillAyd that sounds. I've already got some changes to the script, but it's more in the validate_all part, which I don't think you'll touch.

I love both ideas @TomAugspurger, validating both things will be very useful.

[jorisvandenbossche]

Other point originally reported in #20318: if the docstring contains a newline, it gives problems in the validation script.

[WillAyd]

Edited to add a bullet for the discussion in #20061 (comment)

[kynan]

Some minor improvements in #25122

[ChiefMilesEdgeworth]

A couple more issues that need to be fixed:

• Allow for *args, **kwargs on one line with a description instead of having to split it each time
• Change signature_parameters so that inspect.signature is used instead of inspect.getfullargspec (I talked about this a bit in Fix PR02 issues in docstrings #27976)

This will solve a lot of the false positives for PR01 and PR02 errors.