Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when computing multiple prediction intervals in Time Series Tutorial #484

Open
PolZT opened this issue Jul 9, 2024 · 1 comment
Open
Assignees
Labels
Bug Type: bug TS regression Related to time series regression Work in progress The MAPIE team is working on this.

Comments

@PolZT
Copy link

PolZT commented Jul 9, 2024

Hello MAPIE team,

Thanks for this wonderful library. I'm encountering an issue when trying to compute two intervals instead of one in the time series tutorial, but I might be wrong or missing something in my understanding.

Describe the bug

When attempting to compute two intervals instead of one by changing the alpha value from 0.05 to [0.1, 0.05] in the time series tutorial, a ValueError is raised. The error message indicates an issue with evaluating the truth value of an array in the conformity_scores.py file.

To Reproduce

Steps to reproduce the behavior:

  1. Go to the time series tutorial provided in the MAPIE documentation: MAPIE Time Series Tutorial.
  2. Replace the original alpha = 0.05 with alpha = [0.1, 0.05] on line 193 of the tutorial script.
  3. Run the script.
  4. See the error.

Expected behavior

The script should compute and return two prediction intervals without raising any errors.

Screenshots

N/A (the error message is provided below).

Desktop:

  • OS: Windows 10
  • IDE: PyCharm 2024.1 (Community Edition)
  • MAPIE Version: 0.8.6

Additional context

I tried to find out what is causing it and here are some of my findings:

In conformity_scores.py, the get_quantile function (line 213) receives an inconsistent formatting of alpha_np, sometimes receiving a simple NDArray and sometimes receiving an array of arrays.

Error traceback:

y_pred_enbpi_npfit, y_pis_enbpi_npfit = mapie_enbpi.predict(
  File "C:\Users\pzaho\anaconda3\envs\dynamo\lib\site-packages\mapie\regression\time_series_regression.py", line 459, in predict
    return super().predict(
  File "C:\Users\pzaho\anaconda3\envs\dynamo\lib\site-packages\mapie\regression\regression.py", line 645, in predict
    self.conformity_score_function_.get_bounds(
  File "C:\Users\pzaho\anaconda3\envs\dynamo\lib\site-packages\mapie\conformity_scores\conformity_scores.py", line 454, in get_bounds
    quantile_low = self.get_quantile(
  File "C:\Users\pzaho\anaconda3\envs\dynamo\lib\site-packages\mapie\conformity_scores\conformity_scores.py", line 272, in get_quantile
    quantile = signed * np.column_stack([
  File "C:\Users\pzaho\anaconda3\envs\dynamo\lib\site-packages\mapie\conformity_scores\conformity_scores.py", line 276, in <listcomp>
    ) if not (unbounded and _alpha >= 1) else np.inf * np.ones(n_ref)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
@PolZT PolZT added the Bug Type: bug label Jul 9, 2024
@jawadhussein462 jawadhussein462 added Documentation Type: documentation TS regression Related to time series regression and removed Documentation Type: documentation labels Nov 7, 2024
@Valentin-Laurent Valentin-Laurent added the Discussion in progress Discussion ongoing between the Mapie team and the author. label Nov 18, 2024
@jawadhussein462 jawadhussein462 added Backlog This is in the MAPIE team development backlog, yet to be prioritised. Discussion in progress Discussion ongoing between the Mapie team and the author. and removed Discussion in progress Discussion ongoing between the Mapie team and the author. Backlog This is in the MAPIE team development backlog, yet to be prioritised. labels Dec 10, 2024
@Valentin-Laurent
Copy link
Collaborator

Hello @PolZT, thank you for reporting this, and your positive feedback.
We're working on it :)

@Valentin-Laurent Valentin-Laurent self-assigned this Dec 18, 2024
@Valentin-Laurent Valentin-Laurent added Work in progress The MAPIE team is working on this. and removed Discussion in progress Discussion ongoing between the Mapie team and the author. labels Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Type: bug TS regression Related to time series regression Work in progress The MAPIE team is working on this.
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

4 participants