Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

410 mapiequantileregressor predict method causing memoryerror #412

Conversation

LacombeLouis
Copy link
Collaborator

@LacombeLouis LacombeLouis commented Feb 7, 2024

Description

Fix issue #410. This involves simplifying the type of check and making it more memory efficient.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Test with random data
import numpy as np
import pandas as pd
from lightgbm import LGBMRegressor
from sklearn.model_selection import train_test_split

from mapie.regression import MapieQuantileRegressor

random_state = 26
rng = np.random.default_rng(random_state)

# Generate a random train_data
n = 10000000
p = 10
X = rng.standard_normal((n, p))
beta = rng.standard_normal(p)
y = X @ beta + rng.standard_normal(n)

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    train_size=0.8,
    random_state=random_state,
    shuffle=False,
)
X_train, X_calib, y_train, y_calib = train_test_split(
    X_train,
    y_train,
    train_size=0.8,
    random_state=random_state,
    shuffle=False,
)

print(X_test.shape, X_calib.shape, X_train.shape)


estimator = LGBMRegressor(
    objective="quantile", alpha=0.5, random_state=random_state
)

mqr = MapieQuantileRegressor(
    estimator, method="quantile", cv="split", alpha=0.02
)
mqr.fit(
    X_train,
    y_train,
    X_calib=X_calib,
    y_calib=y_calib,
    random_state=random_state,
)

y_pred, y_pis = mqr.predict(X_test)
print("DONE!")

Checklist

  • I have read the contributing guidelines
  • I have updated the HISTORY.rst and AUTHORS.rst files
  • Linting passes successfully : make lint
  • Typing passes successfully : make type-check
  • Unit tests pass successfully : make tests
  • Coverage is 100% : make coverage
  • Documentation builds successfully : make doc

@LacombeLouis LacombeLouis self-assigned this Feb 7, 2024
@LacombeLouis LacombeLouis linked an issue Feb 7, 2024 that may be closed by this pull request
@codecov-commenter
Copy link

codecov-commenter commented Feb 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (614293e) 100.00% compared to head (89c47a4) 100.00%.
Report is 256 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##            master      #412    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           39        39            
  Lines         4616      4877   +261     
  Branches       487       800   +313     
==========================================
+ Hits          4616      4877   +261     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mapie/utils.py Outdated Show resolved Hide resolved
@thibaultcordier thibaultcordier merged commit 41711d1 into master Feb 12, 2024
8 checks passed
@LacombeLouis LacombeLouis deleted the 410-mapiequantileregressor-predict-method-causing-memoryerror branch April 19, 2024 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MapieQuantileRegressor - predict method causing MemoryError
4 participants