Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept directly EnsembleEstimator as model for MapieRegressor #371

Open
vincentblot28 opened this issue Nov 9, 2023 · 0 comments
Open
Labels
Backlog This is in the MAPIE team development backlog, yet to be prioritised. Enhancement Type: enhancement (new feature or request) Other or internal If no other grey tag is relevant or if issue from the MAPIE team Source: developers Proposed by developers.

Comments

@vincentblot28
Copy link
Collaborator

Why this can be useful ?
When creating some custom non-conformity score with your very nice interface, one might want to compare different scores on the same dataset. For instance, I might want to use the gamma conformity score with different powers of y_pred in the denominator.
Right now, if I want to do this, I have to fit a new MapieRegressor each time, and if I'm using the Jackknife method it can be very long

Solution

def fit(X_train, y_train, ...):
        (estimator,
         self.conformity_score_function_,
         agg_function,
         cv,
         X,
         y,
         sample_weight) = self._check_fit_parameters(X, y, sample_weight)
        if not isinstance(estimator, EnsembleRegressor):
            my_regressor = (
                EnsembleRegressor if not self.model_has_std else EnsembleStdRegressor
            )
            self.estimator_ = my_regressor(
                estimator,
                self.method,
                cv,
                agg_function,
                self.n_jobs,
                self.random_state,
                self.test_size,
                self.verbose
            )
        else:
            self.estimator_ = estimator
        if not self.estimator_.is_fitted:
            self.estimator_ = self.estimator_.fit(X, y, sample_weight)

        if self.model_has_std:
            y_pred, y_std = self.estimator_.predict_calib(X)
            self.conformity_scores_ = \
                self.conformity_score_function_.get_conformity_scores(
                    X, y, y_pred, y_std
                )
        else:
            y_pred = self.estimator_.predict_calib(X)
            self.conformity_scores_ = \
                self.conformity_score_function_.get_conformity_scores(
                    X, y, y_pred
                )

        return self

Here, what is done is that if my estimator is an EnsembleRegressor that is already fitted, then I don't need re-fit all the LOO models, by I can change my conformity score (and even change the method plus to minmax if I change the attribute in the EnsembleRegressor estimator

We should also need to have a is_fitted method in the EnsembleRegressor

@vincentblot28 vincentblot28 added the Enhancement Type: enhancement (new feature or request) label Nov 9, 2023
@thibaultcordier thibaultcordier added the Source: developers Proposed by developers. label Dec 11, 2023
@jawadhussein462 jawadhussein462 added Discussion in progress Discussion ongoing between the Mapie team and the author. Regression Related to regression (excluding time series) labels Nov 8, 2024
@Valentin-Laurent Valentin-Laurent added Other or internal If no other grey tag is relevant or if issue from the MAPIE team Backlog This is in the MAPIE team development backlog, yet to be prioritised. and removed Regression Related to regression (excluding time series) Discussion in progress Discussion ongoing between the Mapie team and the author. labels Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backlog This is in the MAPIE team development backlog, yet to be prioritised. Enhancement Type: enhancement (new feature or request) Other or internal If no other grey tag is relevant or if issue from the MAPIE team Source: developers Proposed by developers.
Projects
None yet
Development

No branches or pull requests

4 participants