Error rendering Python plots in a loop (RPC timed out after 30 seconds)

[juliasilge]

Hi.

Not sure if this adds any value, but I've encountered this issue while doing some Python work as well. Sometimes the plots will render successfully. Sometimes only some render, and other times they don't render.

Below is some Python code I've ran as an example. It's being used to plot various ROC curves and feature importance plots for trained models:

# plotting roc-curves
def plot_roc_curve(y_true, y_prob, model_name):
    fpr, tpr, _ = roc_curve(y_true, y_prob)
    roc_auc = roc_auc_score(y_true, y_prob)
    plt.plot(fpr, tpr, label=f'{model_name} (AUC={roc_auc:.2f})')

plt.figure(figsize=(10, 6))
for result in results:
    best_model = result['best_estimator']
    if hasattr(best_model, "predict_proba"):
        y_prob = best_model.predict_proba(X_test)[:, 1]
        plot_roc_curve(y_test, y_prob, result['model_name'])
plt.plot([0, 1], [0, 1], 'k--', label='Random Guess')
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curves for Models')
plt.legend()
plt.show()

plt.close('all')

# feature importance plot for models with `feature_importances_`
for result in results:
    best_model = result['best_estimator']
    if hasattr(best_model, 'feature_importances_'):
        feature_importances = best_model.feature_importances_
        feature_importance_df = pd.Series(feature_importances, index=X_train.columns).sort_values(ascending=True)

        plt.figure(figsize=(10, 6))
        feature_importance_df.plot(kind='barh')
        plt.title(f"{result['model_name']} Feature Importance")
        plt.xlabel('Importance Score')
        plt.ylabel('Features')
        plt.tight_layout()
        plt.show()

plt.close('all')

Here is the render error message:
Error rendering plot to 'Auto' size: RPC timed out after 30 seconds: {"jsonrpc":"2.0","method":"get_intrinsic_size"} (-32603)

Sorry if this was not useful information, but thanks for all of your excellent work. I'm loving the IDE so far 😺.

Originally posted by @loafing-cat in #5020

[juliasilge]

@loafing-cat I see that you are creating plots in a loop; can you create one single plot without error? Is it possible that the loop takes more than 30 sec to run?

Can you create a reproducible example with code we can run that generates this error?

[juliasilge]

Here is a smaller example demonstrating the problem (I am going to hide the previous comments for ease of reading):

import numpy as np
import pandas as pd
import time

df = pd.DataFrame(np.random.randint(1, 7, 6000), columns=['one'])
df['two'] = df['one'] + np.random.randint(1, 7, 6000)

for x in range(7, 12):
    df.plot.hist(bins=x, alpha=0.5)
    time.sleep(10)

Each of the plots individually can be created very quickly, but when they are in a loop, we get an RPC time out because the whole for loop is one statement that we are waiting to finish.

Doing something like this also gets us an RPC timeout if you select both lines and execute at once, but I think this is the same as #1762 i.e. we should break up this code and execute the statements one at a time:

df.plot.hist(bins=10, alpha=0.5)
time.sleep(40)

I don't believe that #1762 fully covers the specific situation here, as breaking up statements and running them individually won't help us with the for loop.

[juliasilge]

We could adjust the plots service so it does not try to render a plot when the runtime is busy.

[testlabauto]

Note to uncomment test step in "R - Verifies basic plot functionality" when this is resolved. At the moment, when this happens, the comparison of the master image against the current plot image fails (the sizing really is slightly off).