Remove several functions and objects from PyMC root namespace #6973

ricardoV94 · 2023-10-26T11:17:50Z

All arviz plots must be retrieved from pymc.plots.foo
All arviz stats must be retrieved from pymc.stats.foo
Removed many rare or non-user facing objects
Remove rare or non-user facing VI objects (I am not familiar enough to know which ones should stay and which ones should not)

Also

Move all custom Exceptions to pymc.exceptions.py

The following script prints 174 after and 272 before this PR:

import pymc
from types import ModuleType
len([a for a in dir(pymc) if not isinstance(getattr(pymc, a), ModuleType) and not a.startswith("__")])

The following entries were removed:

CallableTensor
DictToArrayBijection
ELPDData
GeneratorAdapter
ImputationWarning
IncorrectArgumentsError
SamplingError
ShapeError
ShapeWarning
TraceDirectoryError
_calculate_ics
apply_test_function
autocorr
autocorrplot
autocov
bfmi
bool_types
compare
compareplot
complex_types
compute_log_likelihood
cont_inputs
continuous_types
convert_observed_data
densityplot
discrete_types
drop_warning_stat
energyplot
ess
expand_packed_triangular
float_types
forestplot
generator
guess_scaling
hdi
int_types
invlogit
invprobit
isgenerator
join_nonshared_inputs
kde
kdeplot
logaddexp
logit
logsumexp
loo
loo_pit
make_shared_replacements
make_ufunc
mcse
pairplot
plot_autocorr
plot_bf
plot_bpv
plot_compare
plot_density
plot_dist
plot_dist_comparison
plot_dot
plot_ecdf
plot_elpd
plot_energy
plot_ess
plot_forest
plot_hdi
plot_kde
plot_khat
plot_lm
plot_loo_pit
plot_mcse
plot_pair
plot_parallel
plot_posterior
plot_ppc
plot_rank
plot_separation
plot_trace
plot_ts
plot_violin
predictions_to_inference_data
probit
psislw
r2_samples
r2_score
reloo
rhat
smartfloatX
str_for_dist
str_for_model
str_for_potential_or_deterministic
summary
to_inference_data
trace_cov
traceplot
typefilter
waic
weight_predictions
wrap_xarray_ufunc

📚 Documentation preview 📚: https://pymc--6973.org.readthedocs.build/en/6973/

maresb · 2023-10-26T11:29:20Z

Depending on how semver-y people feel, this might be best targeted for PyMC v6.

ricardoV94 · 2023-10-26T11:36:44Z

Depending on how semver-y people feel, this might be best targeted for PyMC v6.

I think that will be a strong argument not to go on with these changes

codecov · 2023-10-26T11:49:22Z

Codecov Report

Merging #6973 (e7e8a54) into main (419af06) will decrease coverage by 48.10%.
The diff coverage is 63.15%.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #6973       +/-   ##
===========================================
- Coverage   88.65%   40.55%   -48.10%     
===========================================
  Files         100      100               
  Lines       16896    16868       -28     
===========================================
- Hits        14979     6841     -8138     
- Misses       1917    10027     +8110

Files	Coverage Δ
pymc/__init__.py	`100.00% <100.00%> (ø)`
pymc/backends/base.py	`64.50% <ø> (-21.34%)`	⬇️
pymc/blocking.py	`87.23% <ø> (-8.60%)`	⬇️
pymc/logprob/utils.py	`77.35% <100.00%> (-22.65%)`	⬇️
pymc/model/core.py	`58.45% <ø> (-32.59%)`	⬇️
pymc/plots/__init__.py	`100.00% <ø> (+4.16%)`	⬆️
pymc/pytensorf.py	`58.66% <ø> (-32.23%)`	⬇️
pymc/sampling/parallel.py	`19.92% <100.00%> (-67.45%)`	⬇️
pymc/stats/__init__.py	`100.00% <ø> (ø)`
pymc/step_methods/hmc/base_hmc.py	`82.35% <100.00%> (-8.41%)`	⬇️
... and 14 more

... and 58 files with indirect coverage changes

ColCarroll · 2023-10-26T14:23:21Z

I'll put in a vote against this -- there are enough pymc scripts and libraries out there that don't follow library updates very closely, but would like to continue to benefit from better sampling routines by having liberal version specifications.

Probably the right way would be to use some deprecations library that issued a warning for ~a year, and provided a copy/paste upgrade path. I'm not sure the benefits (tidier namespace? are there others?) outweigh the work and downsides.

maresb · 2023-10-26T16:24:43Z

pymc/plots/__init__.py

-def alias_deprecation(func, alias: str):
-    original = func.__name__
-
-    @functools.wraps(func)
-    def wrapped(*args, **kwargs):
-        raise FutureWarning(
-            f"The function `{alias}` from PyMC was an alias for `{original}` from ArviZ. "
-            "It was removed in PyMC 4.0. "
-            f"Switch to `pymc.{original}` or `arviz.{original}`."
-        )
-
-    return wrapped
-
-
-# Aliases of ArviZ functions
-autocorrplot = alias_deprecation(az.plot_autocorr, alias="autocorrplot")
-forestplot = alias_deprecation(az.plot_forest, alias="forestplot")
-kdeplot = alias_deprecation(az.plot_kde, alias="kdeplot")
-energyplot = alias_deprecation(az.plot_energy, alias="energyplot")
-densityplot = alias_deprecation(az.plot_density, alias="densityplot")
-pairplot = alias_deprecation(az.plot_pair, alias="pairplot")
-traceplot = alias_deprecation(az.plot_trace, alias="traceplot")
-compareplot = alias_deprecation(az.plot_compare, alias="compareplot")


It looks to me like this has already been issuing deprecation warnings. Was this working and warning against using everything slated to be removed since v4?

That was just for utilities whose names have changed. Those could be safely removed by now yes

Ah, then I agree with @ColCarroll that we should definitely deprecate all the other stuff well in advance of removing it.

Also why remove the deprecation warning? In case you want to clear up the namespace then you could refactor it into something like https://peps.python.org/pep-0562/

I removed this deprecation warning because I removed the objects that were being deprecated as well

These one specifically were deprecated since v4, seems safe no?

Aren't there still lots of people who use PyMC3 because of the name recognition and all the SEO? Seems pretty low-effort to provide explicit instructions for them.

Sounds silly. This will not be the thing that people switching from v3 to v5 will find challenging

Ya, I agree it's silly in the case of these ArviZ warnings, but you could do something similar to provide a transition period for the rest of the stuff. Proof-of-concept: ricardoV94#4

I agree, I am going to try and do that!

twiecki · 2023-10-26T21:18:57Z

I'm in favor. Cruft accumulates in any code base and it requires consistent effort to stem the tide, which this attempts. I don't think many people are using these functions and those can easily adjust their imports. In a version or two we'll all have forgotten about it, just like with aesara and pytensor.

ricardoV94 added maintenance major Include in major changes release notes section request discussion labels Oct 26, 2023

ricardoV94 force-pushed the cleanup_namespace branch from 9fd14d6 to ce10976 Compare October 26, 2023 11:26

ricardoV94 added 2 commits October 26, 2023 13:35

Cleanup root namespace

4856e22

Move all custom Exceptions to exceptions.py

e7e8a54

ricardoV94 force-pushed the cleanup_namespace branch from ce10976 to e7e8a54 Compare October 26, 2023 11:35

maresb reviewed Oct 26, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove several functions and objects from PyMC root namespace #6973

Remove several functions and objects from PyMC root namespace #6973

ricardoV94 commented Oct 26, 2023 •

edited

Loading

maresb commented Oct 26, 2023

ricardoV94 commented Oct 26, 2023

codecov bot commented Oct 26, 2023

ColCarroll commented Oct 26, 2023

maresb Oct 26, 2023

ricardoV94 Oct 26, 2023

maresb Oct 26, 2023

maresb Oct 26, 2023

ricardoV94 Oct 26, 2023 •

edited

Loading

ricardoV94 Oct 26, 2023

maresb Oct 26, 2023

ricardoV94 Oct 26, 2023

maresb Oct 26, 2023

ricardoV94 Oct 27, 2023

twiecki commented Oct 26, 2023

Remove several functions and objects from PyMC root namespace #6973

Are you sure you want to change the base?

Remove several functions and objects from PyMC root namespace #6973

Conversation

ricardoV94 commented Oct 26, 2023 • edited Loading

maresb commented Oct 26, 2023

ricardoV94 commented Oct 26, 2023

codecov bot commented Oct 26, 2023

Codecov Report

ColCarroll commented Oct 26, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ricardoV94 Oct 26, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

twiecki commented Oct 26, 2023

ricardoV94 commented Oct 26, 2023 •

edited

Loading

ricardoV94 Oct 26, 2023 •

edited

Loading