String "loss" being passed to metric_name() instead of loss func name

[Zanchet2020]

I'm new to ML and I'm just following a tutorial right now, but I'm getting the error:

ValueError: Could not interpret metric identifier: loss

at:
\keras\src\metrics\__init__.py:205, in get(identifier)

It seems like in the wrapper.py line 532 metric_name(key) should receive the name of a loss function, but it's actually receiving the string "loss"

If inside keras_metric_get I manually set identifier = "binary_crossentropy" it works fine

[adriangb]

Would you like to make a PR with a test and a fix since it sounds like you've already figured out where the problem is?

[Zanchet2020]

I can try, though I'm not sure I'm capable of doing it lol

[adriangb]

Give it a shot! Feel free to ask questions.

[CasperKjaerhus]

I am facing this exact issue as well using keras 3.0.5/tensorflow 2.16.1

[johloe11]

Looks like the issue is with how the _fit_keras_model() method in the wrappers.py file tries to filter out Keras keys. The value error seems to have changed. if you change line 535 from if "Unknown metric function" not in str(e): to if "Unknown metric function" not in str(e) and "Could not interpret metric identifier" not in str(e): it works as expected.

[adriangb]

This is definitely a bug. I’ll try to fix it this week, but PRs are always welcome. I’d suggest a more robust pattern (if key == “loss” or key[:4] == “val_” or similar, suggest looking into keras docs).

[lazarust]

Is someone already working on fixing this? I'm now running into it too

[adriangb]

If you're running into this issue I'd guess you're using an unsupported version of Keras (e.g. Keras 3.0). We don't pin upper versions (which is considered best practice). It's an unfortunate tradeoff you need to make as a Python maintainer.

For now you'll either need use a supported version of Keras / Tensorflow (e.g. Tensorflow 2.12.X without the standalone Keras package) or wait for #317

[adriangb]

Could you please try #317 (pip install -U 'git+https://github.com/adriangb/scikeras.git@keras-3') and see if it fixes your problem? Thanks!