Log-tab under grid view is automatically re-fetching completed logs every 3 sec.

[hterik]

Apache Airflow version

2.4.1

What happened

The new inline log-tab under grid view is fantastic.
What's not so great though, is that it is automatically reloading the logs on the /api/v1/dags/.../dagRuns/.../taskInstances/.../logs/1 api endpoint every 3 seconds. Same interval as the reload of the grid status it seems.

This:

• Makes it difficult for users to scroll in the log panel and to select text in the log panel, because it is replaced all the time
• Put unnecessary load on the client and the link between client-webserver.
• Put unnecssary load on the webserver and on the logging-backend, in our case it involves queries to an external Loki server.

This happens even if the TaskLogReader has set metadata["end_of_log"] = True

What you think should happen instead

Logs should not automatically be reloaded if end_of_log=True

For logs which are not at end, some other slower reload or more incremental query/streaming is preferred.

How to reproduce

No response

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.4.0
apache-airflow-providers-common-sql==1.1.0
apache-airflow-providers-docker==3.2.0
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-postgres==5.2.0
apache-airflow-providers-sqlite==3.1.0

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[bbovenzi]

Oh we should probably include end_of_log in the REST API response.

Edit. end_of_log doesn't work here , but I found a fix.