[pydoclint] Add docstring-missing-parameter and docstring-extraneous-parameter (DOC101, DOC102)

[augustelalande]

Summary

Add docstring-missing-parameter and docstring-extraneous-parameter (DOC101, DOC102). These rules check that the parameters defined in a functions signature match thos defined in the docstring.

Part of #12434.

Test Plan

Test cases added.

[codspeed-hq]

CodSpeed Performance Report

Merging #13280 will not alter performance

_{Comparing augustelalande:doc10x (e0b12ca) with main (1ba8e61)}

Summary

✅ 32 untouched benchmarks

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+11809 -15 violations, +0 -0 fixes in 5 projects; 50 projects unchanged)

apache/airflow (+6846 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ airflow/api/auth/backend/deny_all.py:38:5: DOC101 Parameter `function` missing from the docstring
+ airflow/api/common/delete_dag.py:43:5: DOC101 These parameters are missing from the docstring: `dag_id`, `keep_records_in_log`, `session`
+ airflow/api/common/mark_tasks.py:111:5: DOC101 These parameters are missing from the docstring: `dag`, `state`, `task_ids`, `run_ids`
+ airflow/api/common/mark_tasks.py:125:5: DOC101 These parameters are missing from the docstring: `tasks`, `downstream`, `upstream`
+ airflow/api/common/mark_tasks.py:145:5: DOC101 These parameters are missing from the docstring: `dag`, `logical_date`, `future`, `past`, `session`
+ airflow/api/common/mark_tasks.py:173:5: DOC101 These parameters are missing from the docstring: `dag`, `run_id`, `future`, `past`, `session`
+ airflow/api/common/mark_tasks.py:202:5: DOC101 These parameters are missing from the docstring: `dag_id`, `run_id`, `state`, `session`
+ airflow/api/common/mark_tasks.py:225:5: DOC101 These parameters are missing from the docstring: `dag`, `run_id`, `commit`, `session`
+ airflow/api/common/mark_tasks.py:266:5: DOC101 These parameters are missing from the docstring: `dag`, `run_id`, `commit`, `session`
+ airflow/api/common/mark_tasks.py:348:5: DOC101 These parameters are missing from the docstring: `new_state`, `dag`, `run_id`, `commit`, `session`
+ airflow/api/common/mark_tasks.py:379:5: DOC101 These parameters are missing from the docstring: `dag`, `run_id`, `commit`, `session`
+ airflow/api/common/mark_tasks.py:56:5: DOC101 These parameters are missing from the docstring: `tasks`, `run_id`, `upstream`, `downstream`, `future`, `past`, `state`, `commit`, `session`
+ airflow/api/common/trigger_dag.py:127:5: DOC101 These parameters are missing from the docstring: `dag_id`, `triggered_by`, `run_id`, `conf`, `logical_date`, `replace_microseconds`, `session`
+ airflow/api/common/trigger_dag.py:51:5: DOC101 These parameters are missing from the docstring: `dag_id`, `dag_bag`, `triggered_by`, `run_id`, `conf`, `logical_date`, `replace_microseconds`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:133:5: DOC101 These parameters are missing from the docstring: `limit`, `offset`, `order_by`, `asset_id`, `source_dag_id`, `source_task_id`, `source_run_id`, `source_map_index`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:166:5: DOC101 These parameters are missing from the docstring: `dag_id`, `uri`, `before`, `permitted_dag_ids`
+ airflow/api_connexion/endpoints/asset_endpoint.py:190:5: DOC101 These parameters are missing from the docstring: `dag_id`, `uri`, `before`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:214:5: DOC101 These parameters are missing from the docstring: `dag_id`, `uri`, `before`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:233:5: DOC101 These parameters are missing from the docstring: `dag_id`, `before`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:263:5: DOC101 These parameters are missing from the docstring: `dag_id`, `before`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:282:5: DOC101 These parameters are missing from the docstring: `uri`, `before`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:315:5: DOC101 These parameters are missing from the docstring: `uri`, `before`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:336:5: DOC101 Parameter `session` missing from the docstring
+ airflow/api_connexion/endpoints/asset_endpoint.py:67:5: DOC101 These parameters are missing from the docstring: `uri`, `session`
+ airflow/api_connexion/endpoints/asset_endpoint.py:94:5: DOC101 These parameters are missing from the docstring: `limit`, `offset`, `uri_pattern`, `dag_ids`, `order_by`, `session`
+ airflow/api_connexion/endpoints/config_endpoint.py:34:5: DOC101 Parameter `conf_dict` missing from the docstring
+ airflow/api_connexion/endpoints/config_endpoint.py:47:5: DOC101 Parameter `config_option` missing from the docstring
+ airflow/api_connexion/endpoints/config_endpoint.py:52:5: DOC101 Parameter `config_section` missing from the docstring
... 6818 additional changes omitted for project

apache/superset (+1520 -13 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ RELEASING/changelog.py:107:9: DOC101 Parameter `git_log` missing from the docstring
+ RELEASING/changelog.py:336:5: DOC101 These parameters are missing from the docstring: `ctx`, `previous_version`, `current_version`
+ RELEASING/changelog.py:348:5: DOC101 Parameter `base_parameters` missing from the docstring
+ RELEASING/changelog.py:381:5: D400 First line should end with a period
- RELEASING/changelog.py:381:5: D400 First line should end with a period
+ RELEASING/changelog.py:381:5: D415 First line should end with a period, question mark, or exclamation point
- RELEASING/changelog.py:381:5: D415 First line should end with a period, question mark, or exclamation point
+ RELEASING/changelog.py:381:5: DOC101 These parameters are missing from the docstring: `base_parameters`, `csv`, `access_token`, `risk`
+ RELEASING/changelog.py:52:9: DOC101 Parameter `other` missing from the docstring
+ RELEASING/changelog.py:87:9: DOC101 Parameter `pr_number` missing from the docstring
... 1482 additional changes omitted for rule DOC101
... 1523 additional changes omitted for project

bokeh/bokeh (+339 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ examples/advanced/extensions/parallel_plot/parallel_plot.py:15:5: DOC101 These parameters are missing from the docstring: `df`, `color`, `palette`
+ examples/interaction/js_callbacks/js_on_event.py:16:5: DOC101 These parameters are missing from the docstring: `div`, `attributes`
+ examples/models/gauges.py:33:5: DOC101 Parameter `val` missing from the docstring
+ examples/models/trail.py:20:5: DOC101 These parameters are missing from the docstring: `p1`, `p2`
+ examples/output/jupyter/push_notebook/Numba Image Example.ipynb:cell 22:3:5: DOC101 These parameters are missing from the docstring: `img`, `tmp`
+ examples/server/app/events_app.py:17:5: DOC101 These parameters are missing from the docstring: `div`, `attributes`
... 302 additional changes omitted for rule DOC101
+ src/bokeh/core/has_props.py:424:9: DOC102 These documented parameters are not in the function's signature: `json`, `models`, `setter(ClientSession`
+ src/bokeh/core/property/dataspec.py:449:9: DOC102 Documented parameter `name` is not in the function's signature
+ src/bokeh/core/property/descriptors.py:399:9: DOC102 These documented parameters are not in the function's signature: `json`, `models`
+ src/bokeh/core/property/descriptors.py:664:9: DOC102 Documented parameter `new` is not in the function's signature
... 329 additional changes omitted for project

zulip/zulip (+581 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ analytics/lib/fixtures.py:19:5: DOC101 These parameters are missing from the docstring: `days`, `business_hours_base`, `non_business_hours_base`, `growth`, `autocorrelation`, `spikiness`, `holiday_rate`, `frequency`, `partial_sum`, `random_seed`
+ analytics/migrations/0015_clear_duplicate_counts.py:8:5: DOC101 These parameters are missing from the docstring: `apps`, `schema_editor`
+ analytics/tests/test_counts.py:225:9: DOC101 These parameters are missing from the docstring: `table`, `arg_keys`, `arg_values`
+ confirmation/models.py:288:5: DOC101 These parameters are missing from the docstring: `user_profile`, `email_type`
+ confirmation/models.py:307:5: DOC101 Parameter `creation_key` missing from the docstring
+ confirmation/models.py:90:5: DOC101 These parameters are missing from the docstring: `confirmation_key`, `confirmation_types`, `mark_object_used`, `allow_used`
+ corporate/lib/activity.py:94:5: DOC101 Parameter `cursor` missing from the docstring
+ corporate/lib/stripe.py:1146:9: DOC101 Parameter `metadata` missing from the docstring
+ corporate/lib/stripe.py:390:5: DOC101 These parameters are missing from the docstring: `customer`, `plan_tier`
+ corporate/lib/stripe.py:5302:5: DOC101 Parameter `remote_server` missing from the docstring
... 571 additional changes omitted for project

astropy/astropy (+2523 -2 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ astropy/__init__.py:137:9: DOC101 Parameter `value` missing from the docstring
+ astropy/config/configuration.py:526:9: DOC101 Parameter `val` missing from the docstring
+ astropy/config/configuration.py:543:5: DOC101 These parameters are missing from the docstring: `packageormod`, `rootname`
+ astropy/config/configuration.py:653:5: DOC101 Parameter `verbose` missing from the docstring
+ astropy/config/configuration.py:768:5: DOC101 These parameters are missing from the docstring: `content`, `template_content`
+ astropy/config/paths.py:179:9: DOC101 Parameter `func` missing from the docstring
... 2398 additional changes omitted for rule DOC101
+ astropy/constants/utils.py:11:5: DOC102 Documented parameter `codata, iaudata` is not in the function's signature
+ astropy/constants/utils.py:44:5: DOC102 Documented parameter `codata, iaudata` is not in the function's signature
+ astropy/coordinates/angles/utils.py:102:5: DOC102 Documented parameter `lon, lat, posang, distance` is not in the function's signature
+ astropy/coordinates/angles/utils.py:34:5: DOC102 Documented parameter `lon1, lat1, lon2, lat2` is not in the function's signature
... 2515 additional changes omitted for project

Changes by rule (9 rules affected)

code	total	+ violation	- violation	+ fix	- fix
DOC101	11624	11624	0	0	0
DOC102	170	170	0	0	0
D415	8	4	4	0	0
D400	4	2	2	0	0
D212	4	2	2	0	0
DOC201	4	2	2	0	0
D202	4	2	2	0	0
D200	4	2	2	0	0
PLW1514	2	1	1	0	0

[MichaReiser]

This is great. I haven't gone through the ecossytem results but I think there are a few cases where we don't parse the parameter names correctly.

[MichaReiser]

Hmm, there's an overlap with https://docs.astral.sh/ruff/rules/undocumented-param/. Not quiet sure how to resolve the overlap.

[augustelalande]

UndocumentedParam in its current implimentation is restricted to google-style docstring only. I would suggest deprecating that rule in favor of this one, since it makes more sense as part of the pydoclint package.

[MichaReiser]

UndocumentedParam in its current implimentation is restricted to google-style docstring only. I would suggest deprecating that rule in favor of this one, since it makes more sense as part of the pydoclint package.

That's fair. But the google style one is less strict than the new one. It doesn't require that you document the parameters of all functions. It only requires that the parameters match the function's parameters if there's an Arguments section.

[augustelalande]

We could add an option to only highlight violations when a section is present. I tried something similar #13302.

[MichaReiser]

We plan to look into the conflict but it may take some time. What we could do to move DOC102 forward is to move its implementation out of this PR and try to see if it is possible to share logic with D417:

ruff/crates/ruff_linter/src/rules/pydocstyle/rules/sections.rs

Lines 1784 to 1846 in 281e6d9

	fn missing_args(checker: &mut Checker, docstring: &Docstring, docstrings_args: &FxHashSet<String>) {
	let Some(function) = docstring.definition.as_function_def() else {
	return;
	};
	// Look for arguments that weren't included in the docstring.
	let mut missing_arg_names: FxHashSet<String> = FxHashSet::default();
	// If this is a non-static method, skip `cls` or `self`.
	for ParameterWithDefault {
	parameter,
	default: _,
	range: _,
	} in function
	.parameters
	.iter_non_variadic_params()
	.skip(usize::from(
	docstring.definition.is_method()
	&& !is_staticmethod(&function.decorator_list, checker.semantic()),
	))
	{
	let arg_name = parameter.name.as_str();
	if !arg_name.starts_with('_') && !docstrings_args.contains(arg_name) {
	missing_arg_names.insert(arg_name.to_string());
	}
	}
	// Check specifically for `vararg` and `kwarg`, which can be prefixed with a
	// single or double star, respectively.
	if let Some(arg) = function.parameters.vararg.as_ref() {
	let arg_name = arg.name.as_str();
	let starred_arg_name = format!("*{arg_name}");
	if !arg_name.starts_with('_')
	&& !docstrings_args.contains(arg_name)
	&& !docstrings_args.contains(&starred_arg_name)
	{
	missing_arg_names.insert(starred_arg_name);
	}
	}
	if let Some(arg) = function.parameters.kwarg.as_ref() {
	let arg_name = arg.name.as_str();
	let starred_arg_name = format!("**{arg_name}");
	if !arg_name.starts_with('_')
	&& !docstrings_args.contains(arg_name)
	&& !docstrings_args.contains(&starred_arg_name)
	{
	missing_arg_names.insert(starred_arg_name);
	}
	}
	if !missing_arg_names.is_empty() {
	if let Some(definition) = docstring.definition.name() {
	let names = missing_arg_names.into_iter().sorted().collect();
	checker.diagnostics.push(Diagnostic::new(
	UndocumentedParam {
	definition: definition.to_string(),
	names,
	},
	function.identifier(),
	));
	}
	}
	}

[charliermarsh]

I agree that the rule makes more sense here semantically (since it's about content rather than style)... though the D rules are a lot more popular so we'd lose something by moving the rule over. I find this one to be a really tough call (and it's making me desperate for re-categorized rulesets with a single coherent docstring section).

Anyway... I think I'm comfortable adding this to DOC in preview, then eventually redirecting D417 to it. I'd like to get @zanieb's input on this one though.

[zanieb]

@charliermarsh that sounds okay to me

[augustelalande]

Thanks guys. Is this ready for merge, or are there more changes requested?

[AllanChain]

This PR has been pending for some time. Just want to gently bump this thread to see if there are any updates or if there's anything blocking its progress. This rule would be very useful, so I'm eager to see it merged.