Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider multiplying the quadratic biases in SelectFromQuadraticModel.correlation_cqm() by 1-alpha. #17

Open
arcondello opened this issue Mar 8, 2024 · 0 comments · May be fixed by #18

Comments

@arcondello
Copy link
Member

Currently, it is possible to construct a dataset where, even with alpha=1, a fixed or random column is chosen because of the penalties in the quadratic term. This contradicts the docstring which claims

            alpha:
                Hyperparameter between 0 and 1 that controls the relative weight of
                the relevance and redundancy terms.
                ``alpha=0`` places no weight on the quality of the features,
                therefore the features will be selected as to minimize the
                redundancy without any consideration to quality.
                ``alpha=1`` places the maximum weight on the quality of the features,
                and therefore will be equivalent to using
                :class:`sklearn.feature_selection.SelectKBest`.

One solution is to multiply the quadratic/redundany terms by 1-alpha to ensure that they are zeroed when alpha=1.

For instance we could replace

# our objective
# we multiply by 2 because the matrix is symmetric
np.fill_diagonal(correlations, correlations[:, -1] * (-2 * alpha * num_features))
with

            diag = np.array(correlations[:, -1] * (-2 * alpha * num_features), copy=True)

            correlations *= (1-alpha)

            # our objective
            # we multiply by 2 because the matrix is symmetric
            np.fill_diagonal(correlations, diag)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant