Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot access accuracy in results with keras 3 #20529

Open
tanwarsh opened this issue Nov 21, 2024 · 3 comments
Open

Cannot access accuracy in results with keras 3 #20529

tanwarsh opened this issue Nov 21, 2024 · 3 comments
Assignees
Labels
stat:awaiting response from contributor type:support User is asking for help / asking an implementation question. Stackoverflow would be better suited.

Comments

@tanwarsh
Copy link

Hi, I am new to Keras and TensorFlow. I am using keras==3.6.0 and tensorflow==2.18.0.
I created a sequential model and added layers to it. Here is a pseudocode of what I did:

from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import Dense
from keras.layers import Flatten
from keras import metrics

def build_model(self,
                input_shape,
                num_classes,
                conv_kernel_size=(4, 4),
                conv_strides=(2, 2),
                conv1_channels_out=16,
                conv2_channels_out=32,
                final_dense_inputsize=100):
    model = Sequential()
    model.add(Conv2D(conv1_channels_out,
                     kernel_size=conv_kernel_size,
                     strides=conv_strides,
                     activation='relu',
                     input_shape=input_shape))
    model.add(Conv2D(conv2_channels_out,
                     kernel_size=conv_kernel_size,
                     strides=conv_strides,
                     activation='relu'))
    model.add(Flatten())
    model.add(Dense(final_dense_inputsize, activation='relu'))
    model.add(Dense(num_classes, activation='softmax'))
    model.compile(loss="categorical_crossentropy",
                  optimizer="adam",
                  metrics=["accuracy"])
    
    return model

while evaluating the model, I am getting the results as shown below.

print(model.evaluate(self.data_loader.get_valid_loader(batch_size), verbose=1))
print(new_model.metrics_names)

Result:

157/157 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.0788 - loss: 2.3034
[2.3036301136016846, 0.07720000296831131]
['loss', 'compile_metrics']

Expected Result:

157/157 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.0788 - loss: 2.3034
[2.3036301136016846, 0.07720000296831131]
['loss', 'accuracy']

Is my expectation correct, or do I need to access accuracy differently? Also, the results do not change even if I change the metrics while compiling the model. Any guidance would be appreciated.

eg.

model.compile(loss="categorical_crossentropy",
                      optimizer="adam",
                      metrics=[metrics.MeanSquaredError(name='my_mse'),
                               metrics.AUC(name='my_auc'),
                               metrics.BinaryAccuracy(),
                               metrics.Accuracy(),])
@sachinprasadhs sachinprasadhs added the type:support User is asking for help / asking an implementation question. Stackoverflow would be better suited. label Nov 21, 2024
@sachinprasadhs
Copy link
Collaborator

You can get the metrics name and it's value using model.get_metrics_result()
Attaching the Gist here for reference with the usage.

If you train the model, you can also get these metrics details using something like

history = model.fit()
history.history # This will be dictionary with all the metrics details

@tanwarsh
Copy link
Author

Thanks @sachinprasadhs for the quick response. This was helpful.
I have one more doubt.
with keras 3.6.0 code below is not working related to optmizer and I was not able to find anything related to this in documentation.

model.optimizer.get_weights()
model.optimizer.weights

How can we achieve the same functionality?

@sachinprasadhs
Copy link
Collaborator

You can access the optimizer configs using model.optimizer.get_config()

It shows output something like this

{'name': 'adam',
 'learning_rate': 0.0010000000474974513,
 'weight_decay': None,
 'clipnorm': None,
 'global_clipnorm': None,
 'clipvalue': None,
 'use_ema': False,
 'ema_momentum': 0.99,
 'ema_overwrite_frequency': None,
 'loss_scale_factor': None,
 'gradient_accumulation_steps': None,
 'beta_1': 0.9,
 'beta_2': 0.999,
 'epsilon': 1e-07,
 'amsgrad': False}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stat:awaiting response from contributor type:support User is asking for help / asking an implementation question. Stackoverflow would be better suited.
Projects
None yet
Development

No branches or pull requests

2 participants