[docker] upgrade to ubuntu 24.04 + use uv for package management

[ebr]

Summary

• upgrades docker image to ubuntu 24.04 (contributed by @rick-github)
• adds support in docker entrypoint for keeping HF cache in the invoke runtime directory, unless specified otherwise - this should help avoid unnecessary redownloads from HF (contributed by @rick-github)
• ubuntu 24.04 ships with python 3.12, so this PR changes our package management backend to use uv, including the installation of python itself. This will both improve performance when using docker cache, and decouple python versioning from the underlying OS.
• fixes: legacy model configs were being copied with their permissions intact. Because we're trying to copy them on every startup, this would cause the application to fail to start when the source code directory is read-only, because it could not overwrite the read-only copies of the configs. This PR checks whether the files already exist and are identical, and if copying is needed, it changes permissions on the copied files.
• same for custom nodes dir

Related Issues / Discussions

Closes #7387
Supercedes #7396

QA Instructions

Build and run the container locally as usual: docker compose up --build, and test the application.

Merge Plan

Community testing might be useful, but can be merged anytime otherwise.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[rick-github]

I successfully built from https://github.com/invoke-ai/InvokeAI.git#ebr/docker-ubuntu-2404 and the UI starts, showing the previously generated images, but the Invoke button is greyed out and the popup says "Unable to queue". If I switch back to my build, the Invoke button works. There are are few errors in the docker logs, but they're the same for both builds.

invoke-1  | [2024-11-30 00:28:41,739]::[uvicorn.error]::INFO --> Started server process [1]
invoke-1  | [2024-11-30 00:28:41,739]::[uvicorn.error]::INFO --> Waiting for application startup.
invoke-1  | [2024-11-30 00:28:41,780]::[uvicorn.error]::INFO --> Application startup complete.
invoke-1  | [2024-11-30 00:28:41,780]::[uvicorn.error]::INFO --> Uvicorn running on http://0.0.0.0:9090 (Press CTRL+C to quit)
invoke-1  | [2024-11-30 00:28:43,084]::[uvicorn.error]::INFO --> ('10.10.22.186', 52382) - "WebSocket /ws/socket.io/?EIO=4&transport=websocket&sid=JTiI0BRQZbre0wjoAAAA" [accepted]
invoke-1  | [2024-11-30 00:28:43,086]::[uvicorn.error]::INFO --> connection open
invoke-1  | [2024-11-30 00:28:44,104]::[uvicorn.error]::INFO --> ('10.10.30.32', 44802) - "WebSocket /ws/socket.io/?EIO=4&transport=websocket&sid=3rfVJSj_WsXhBLIhAAAC" [accepted]
invoke-1  | [2024-11-30 00:28:44,104]::[uvicorn.error]::INFO --> connection open
invoke-1  | [2024-11-30 00:28:44,859]::[uvicorn.error]::INFO --> connection closed
invoke-1  | [2024-11-30 00:28:49,027]::[uvicorn.error]::INFO --> ('10.10.30.32', 58110) - "WebSocket /ws/socket.io/?EIO=4&transport=websocket&sid=lczQzl-fRSTCchltAAAE" [accepted]
invoke-1  | [2024-11-30 00:28:49,028]::[uvicorn.error]::INFO --> connection open

There's no significant differences between the logs:

$ diff  <(sed -e 's/..:..:..,.../xx:xx:xx.xxx/g' -e 's/:[0-9][0-9][0-9][0-9][0-9] - /:port - /g' -e 's/, [0-9][0-9][0-9][0-9][0-9]/, port/' -e 's/\(sid\|t\|index\|ThemeLocaleProvider\|useTranslation\|App\)[-=][-0-9A-Za-z_]*/\1/g' in.local | ansifilter) <(sed -e 's/..:..:..,.../xx:xx:xx.xxx/g' -e 's/:[0-9][0-9][0-9][0-9][0-9] - /:port - /g' -e 's/, [0-9][0-9][0-9][0-9][0-9]/, port/' -e 's/\(sid\|t\|index\|ThemeLocaleProvider\|useTranslation\|App\)[-=][-0-9A-Za-z_]*/\1/g' in.ebr | ansifilter) 
20d19
< invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "POST /ws/socket.io/?EIO=4&transport&t&sid HTTP/1.1" 200
21a21
> invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "POST /ws/socket.io/?EIO=4&transport&t&sid HTTP/1.1" 200
33d32
< invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "GET /assets/App.js HTTP/1.1" 200
34a34
> invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "GET /assets/App.js HTTP/1.1" 200
41d40
< invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "GET /api/v2/models/i/f6f8e862-5364-47ea-9d11-bf25f04aa104 HTTP/1.1" 200
43c42
< invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "GET /assets/inter-latin-wght.woff2 HTTP/1.1" 200
---
> invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "GET /api/v2/models/i/f6f8e862-5364-47ea-9d11-bf25f04aa104 HTTP/1.1" 200
46a46
> invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "GET /assets/inter-latin-wght.woff2 HTTP/1.1" 200
47a48,49
> invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.error]::INFO --> ('10.10.30.32', port) - "WebSocket /ws/socket.io/?EIO=4&transport&sid" [accepted]
> invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.error]::INFO --> connection open
51d52
< invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.error]::INFO --> ('10.10.30.32', port) - "WebSocket /ws/socket.io/?EIO=4&transport&sid" [accepted]
55d55
< invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.error]::INFO --> connection open
59d58
< invoke-1  | [2024-11-30 xx:xx:xx.xxx]::[uvicorn.access]::INFO --> 10.10.30.32:port - "GET /ws/socket.io/?EIO=4&transport&t&sid HTTP/1.1" 200

I had a quick look through the development documentation but I didn't come across a way to enable verbose logging, if such a thing exists.

[ebr]

"Unable to queue" - that's unusual. Do you see any errors in your browser console/devtools?
Verbose logging can be enabled by setting INVOKEAI_LOG_LEVEL=debug; not sure how much it will help, but worth a try.
Are you on NVIDIA or AMD?

[rick-github]

Only one error and it occurs for both builds.

[Deprecation] Listener added for a 'DOMNodeRemoved' mutation event. Support for this event type has been removed, and this event will no longer be fired. See https://chromestatus.com/feature/5083947249172480 for more information.

There are several warnings but again same for both builds.

index-CLK8L2nB.js:461 react-i18next:: You will need to pass in an i18next instance by using initReactI18next
index-CLK8L2nB.js:79  warn  [system]: Unable to parse type of input field collect.item (unable to parse field type)
index-CLK8L2nB.js:79  warn  [system]: Unable to parse type of output field collect.collection (unsupported array item type "")
index-CLK8L2nB.js:79  warn  [system]: Unable to parse type of input field iterate.collection (unsupported array item type "")
index-CLK8L2nB.js:79  warn  [system]: Unable to parse type of output field iterate.item (unable to parse field type)
index-CLK8L2nB.js:79  warn  [system]: Unable to parse type of input field metadata_item.value (unable to parse field type)

Nvidia RTX3080

[rick-github]

I checked out a branch at the same point where I wrote the original patch and applied #7401 to it, Invoke button still unavailable.
I deleted the volume with InvokeAI state and re-downloaded models, Invoke button still unavailable.

[ebr]

@psychedelicious do you have any guesses as to why @rick-github might be seeing the "Unable to queue" tooltip when trying to invoke? I can't reproduce this on my end...

[psychedelicious]

@psychedelicious do you have any guesses as to why @rick-github might be seeing the "Unable to queue" tooltip when trying to invoke? I can't reproduce this on my end...

The tooltip displays the reasons why it is disabled. For example, if there is no connectivity to the server, it will show System disconnected:

All reasons are listed and vary depending on the tab. For example, if there was a connectivity issue, you are on the Workflows tab, and the graph is invalid, you'll see the System disconnected message plus any problems with the graph.

What message(s) do you see @rick-github?

[rick-github]

It seems to be a function of the docker checkout/build. If use context: https://github.com/invoke-ai/InvokeAI.git#ebr/docker-ubuntu-2404 then when the docker image is run, I get the Empty layer eror. If I git clone from head and then manually apply #7401 and build from the local copy, it works fine.

[psychedelicious]

@rick-github Thanks. That problem was resolved in #7410. It's unrelated to the changes in this PR.

[rick-github]

@ebr, could you sync your branch to head? Then testers (eg #7430 ) can pull and build from your branch without hitting the layer issue.

[rick-github]

Confirm that pulling and building from https://github.com/invoke-ai/InvokeAI.git#ebr/docker-ubuntu-2404 works.