Security focused alternative image #2526

spunkedy · 2024-03-26T16:49:02Z

I finally had time to come back around to getting a hardened version.

Let me know if we would like a different pattern, I tried to follow the conventions in place as best as possible.

This enables a ubi8 image which currently has 0 vulnerabilities

ubi8
FIPS
User change

ubi8

Use ubi8 instead of 9 since it is still pending for FIPS

Redhat universal base image allows for a maintained upstrean docker image with security in mind

Fips mode

This does NOT mean that everything that is run here is "FIPS compliant" you still have to configure hackney/req/ssl/ etc to run in these modes, but I figured that was out of scope for this. These changes to the runtime are what is required to be able to enable the others.

Startup fips mode

This introduces a LIVEBOOK_FIPS flag that will show a log and raise an error if there is an issue setting the mode.

User Change

nobody as the user for the execution environment, by default allow for this.

Possible improvements

Possible improvements I see:

I do not see any pre built fips compliant images to build the binaries so I put it into the docker itself.
Maybe some more documentation around:
- giving example on how to extend this alternative image for adding system packages
- using a fips compliant live books
- how to create dedicated livebook pre built images

CLAassistant · 2024-03-26T16:49:08Z

All committers have signed the CLA.

josevalim · 2024-03-26T18:20:28Z

Thank you! A pull request with the LIVEBOOK_FIPS flag will be welcome but the additional Docker images is probably best handled by the community for now. :)

spunkedy · 2024-03-26T21:46:51Z

Thank you! A pull request with the LIVEBOOK_FIPS flag will be welcome but the additional Docker images is probably best handled by the community for now. :)

Should this also be a place for the cuda/rocm versions with the changes to the cuda upstream docker images that have been recently?

Sounds good, are we ok with having the readme for the livebook fips give links to the community versions so that people can understand the constraints of how to be able to use it?

josevalim · 2024-03-26T22:15:24Z

Should this also be a place for the cuda/rocm versions with the changes to the cuda upstream docker images that have been recently?

I am not sure I get it.

Sounds good, are we ok with having the readme for the livebook fips give links to the community versions so that people can understand the constraints of how to be able to use it?

That's fine by me. Perhaps it is more important to link to the Dockerfile themselves then to the pre-built images because I assume people will want to double check those anyway?

spunkedy · 2024-03-26T22:45:12Z

Should this also be a place for the cuda/rocm versions with the changes to the cuda upstream docker images that have been recently?

I am not sure I get it.

I thought the nvidia docker image pipeline wouldn't be supported long term. I could be wrong

https://github.com/NVIDIA/nvidia-docker?tab=readme-ov-file#deprecation-notice

I wasn't sure if you were suggesting that all non debian/hex based livebook images should go to a community version instead of them being maintained as release packaged targets here.

Since XLA recently added ROCm I figured if there was going to be a community maintained downstream artifacts for docker images, to do it in such a way that might look like:

  graph TD;
    livebook_version_release-->community_docker_images;
    community_docker_images-->cuda;
    community_docker_images-->secured;
    community_docker_images-->rocm;
    community_docker_images-->alpine;

Sounds good, are we ok with having the readme for the livebook fips give links to the community versions so that people can understand the constraints of how to be able to use it?

That's fine by me. Perhaps it is more important to link to the Dockerfile themselves then to the pre-built images because I assume people will want to double check those anyway?

Sounds good, they probably would, not only that they might want to extend. The images might be just a baseline for extending

josevalim · 2024-03-26T22:49:51Z

Great, so please ping us once this is ready to go from your side. :) Thanks!

jonatanklosko · 2024-03-27T08:19:06Z

I thought the nvidia docker image pipeline wouldn't be supported long term. I could be wrong

@spunkedy I think the container toolkit is separate from the Docker images themselves. The images are built from https://gitlab.com/nvidia/container-images/cuda and that is not deprecated.

For ROCm we still don't have precompiled XLA, because there aren't many users and it is maintenance cost to keep the right version matrix and test the builds. If it becomes more stable and popular in the future, then we can consider having Livebook images as well.

spunkedy · 2024-03-27T15:19:58Z

I thought the nvidia docker image pipeline wouldn't be supported long term. I could be wrong

@spunkedy I think the container toolkit is separate from the Docker images themselves. The images are built from https://gitlab.com/nvidia/container-images/cuda and that is not deprecated.

For ROCm we still don't have precompiled XLA, because there aren't many users and it is maintenance cost to keep the right version matrix and test the builds. If it becomes more stable and popular in the future, then we can consider having Livebook images as well.

Thanks! I don't use the CUDA containers, happy to be wrong here!

spunkedy · 2024-03-27T16:04:44Z

@josevalim

Based upon the feedback, I added the documentation, examples, and more descriptions.

Specifically used hex docs for this. I went back and forth between having it be just a sub section of docs/deployment/docker.md and docs/deployment/fips.md but since it isn't technically a docker specific thing I put it in docs/deployment/fips.md

docs/deployment/fips.md

jonatanklosko · 2024-04-02T07:47:56Z

lib/livebook.ex

+      if :crypto.enable_fips_mode(true) do
+        IO.puts("[Livebook] FIPS mode set")
+      else
+        raise "Could not set FIPS mode but was asked to"


Suggested change

raise "Could not set FIPS mode but was asked to"

Livebook.Config.abort!(

"requested FIPS mode via LIVEBOOK_FIPS, but this Erlang installation was compiled without FIPS support"

)

lib/livebook.ex

README.md

jonatanklosko · 2024-04-02T07:52:43Z

docs/deployment/fips.md

+### Error on startup
+
+```bash
+export LIVEBOOK_FIPS=true
+_build/prod/rel/livebook/bin/livebook start_iex 
+
+ERROR! Config provider Config.Reader failed with:
+** (RuntimeError) Could not set FIPS mode but was asked to
+    (livebook 0.13.0-dev) lib/livebook.ex:242: Livebook.config_runtime/0
+        ...
+
+```


Instead of showing the error, I would just tell them that FIPS is enabled with LIVEBOOK_FIPS. When it fails, the error should be clear enough to tell them what's going on.

🤔

Here are some of my thoughts around each

Hard error:

Garuntees the environment in which it is run from the container perspective

Going through security audits. showing configuration at the environment level makes it easier

Soft warning:

to garuntee the environment, someone would have to do something like a supervisor script that makes sure the environment is that way on startup

could give people a false sense of it being turned on if it gets restarted in a node that doesn't support it for some reason or another

Would this be better?

LIVEBOOK_FIPS true/false - IO puts information for logging

LIVEBOOK_FIPS_ENFORCE true/false - Hard error and explain what needs to be true to allow for it

Oh sorry, I wasn't clear. We should definitely exit with clear error. My points was to remove the example from the docs and make the error itself clear enough in case they run into it.

Co-authored-by: Jonatan Kłosko <[email protected]>

github-actions · 2024-04-02T13:35:57Z

Uffizzi Ephemeral Environment `deployment-49304`

☁️ https://app.uffizzi.com/github.com/livebook-dev/livebook/pull/2526

📄 View Application Logs etc.

What is Uffizzi? Learn more!

github-actions · 2024-04-02T13:35:59Z

Uffizzi Preview deployment-49304 was deleted.

spunkedy · 2024-04-02T15:08:02Z

Gotcha. Will get that in later today my time.

…

On Tue, Apr 2, 2024 at 8:42 AM Jonatan Kłosko ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In docs/deployment/fips.md <#2526 (comment)> : > +### Error on startup + +```bash +export LIVEBOOK_FIPS=true +_build/prod/rel/livebook/bin/livebook start_iex + +ERROR! Config provider Config.Reader failed with: +** (RuntimeError) Could not set FIPS mode but was asked to + (livebook 0.13.0-dev) lib/livebook.ex:242: Livebook.config_runtime/0 + ... + +``` Oh sorry, I wasn't clear. We should definitely exit with clear error. My points was to remove the example from the docs and make the error itself clear enough in case they run into it. — Reply to this email directly, view it on GitHub <#2526 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFIUL7GVFXMZGKGFCSCSQDY3KYUFAVCNFSM6AAAAABFJJ47HKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSNZTHA3TAOJYHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

spunkedy · 2024-04-02T15:08:28Z

Sounds good will make those changes

…

On Tue, Apr 2, 2024 at 8:45 AM Jonatan Kłosko ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In docs/deployment/fips.md <#2526 (comment)> : > +_build/prod/rel/livebook/bin/livebook start_iex + +ERROR! Config provider Config.Reader failed with: +** (RuntimeError) Could not set FIPS mode but was asked to + (livebook 0.13.0-dev) lib/livebook.ex:242: Livebook.config_runtime/0 + ... + +``` + +This means that your elixir/erlang environmet was NOT compiled with FIPS enabled. + +### Docker example +To do this in docker, you will need to build it a little bit differently. You can see a full example below. This should be considered psuedo code, you will want to adapt it to your needs. You should consider having a base image for the erlang/elixir portion with FIPS turned on and then overlay with a [multi stage build](https://docs.docker.com/build/building/multi-stage/). + + +```docker Yeah, we can keep everything before WORKDIR /build_app. — Reply to this email directly, view it on GitHub <#2526 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFIUL5GNGWEB4EGLNW5SYLY3KY7PAVCNFSM6AAAAABFJJ47HKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSNZTHA3TSNRVGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

spunkedy · 2024-04-22T11:23:18Z

Ugh, the rebase added everything, let me see if I can adjust that

spunkedy · 2024-04-22T13:47:47Z

@jonatanklosko I think I have applied everything you were asking, let me know otherwise.

docs/deployment/fips.md

jonatanklosko

@josevalim feel free to merge!

jonatanklosko · 2024-04-22T14:47:02Z

@spunkedy thanks!

josevalim · 2024-04-22T20:48:46Z

💚 💙 💜 💛 ❤️

enabling ubi8 and fips mode for docker images

a78268c

spunkedy added 2 commits March 27, 2024 11:00

Livebook fips mode examples

e15bba5

Livebook fips mode examples

192c2cf

josevalim reviewed Mar 27, 2024

View reviewed changes

docs/deployment/fips.md Show resolved Hide resolved

jonatanklosko reviewed Apr 2, 2024

View reviewed changes

lib/livebook.ex Outdated Show resolved Hide resolved

jonatanklosko reviewed Apr 2, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

jonatanklosko reviewed Apr 2, 2024

View reviewed changes

spunkedy and others added 2 commits April 2, 2024 08:32

Update lib/livebook.ex

d4cbf3a

Co-authored-by: Jonatan Kłosko <[email protected]>

Update README.md

c4961c8

Co-authored-by: Jonatan Kłosko <[email protected]>

spunkedy force-pushed the feature-ubi branch from ea9e6db to c4961c8 Compare April 22, 2024 11:30

spunkedy added 2 commits April 22, 2024 08:16

cleanup

497a7fc

feedback applied

da82f54

jonatanklosko reviewed Apr 22, 2024

View reviewed changes

docs/deployment/fips.md Outdated Show resolved Hide resolved

jonatanklosko reviewed Apr 22, 2024

View reviewed changes

docs/deployment/fips.md Outdated Show resolved Hide resolved

Apply suggestions from code review

f103d2f

jonatanklosko reviewed Apr 22, 2024

View reviewed changes

docs/deployment/fips.md Outdated Show resolved Hide resolved

Update docs/deployment/fips.md

afb662c

jonatanklosko approved these changes Apr 22, 2024

View reviewed changes

josevalim added 2 commits April 22, 2024 22:46

Update fips.md

174bc3c

Update README.md

4577717

josevalim merged commit 9eb5cbd into livebook-dev:main Apr 22, 2024
5 of 6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Security focused alternative image #2526

Security focused alternative image #2526

spunkedy commented Mar 26, 2024

CLAassistant commented Mar 26, 2024 •

edited

Loading

josevalim commented Mar 26, 2024

spunkedy commented Mar 26, 2024

josevalim commented Mar 26, 2024

spunkedy commented Mar 26, 2024 •

edited

Loading

josevalim commented Mar 26, 2024

jonatanklosko commented Mar 27, 2024

spunkedy commented Mar 27, 2024

spunkedy commented Mar 27, 2024

jonatanklosko Apr 2, 2024 •

edited

Loading

jonatanklosko Apr 2, 2024

spunkedy Apr 2, 2024

jonatanklosko Apr 2, 2024

github-actions bot commented Apr 2, 2024 •

edited

Loading

github-actions bot commented Apr 2, 2024 •

edited

Loading

spunkedy commented Apr 2, 2024 via email

spunkedy commented Apr 2, 2024 via email

spunkedy commented Apr 22, 2024

spunkedy commented Apr 22, 2024

jonatanklosko left a comment

jonatanklosko commented Apr 22, 2024

josevalim commented Apr 22, 2024

-        raise "Could not set FIPS mode but was asked to"
+        Livebook.Config.abort!(
+          "requested FIPS mode via LIVEBOOK_FIPS, but this Erlang installation was compiled without FIPS support"
+        )

Security focused alternative image #2526

Security focused alternative image #2526

Conversation

spunkedy commented Mar 26, 2024

ubi8

Fips mode

Startup fips mode

User Change

Possible improvements

CLAassistant commented Mar 26, 2024 • edited Loading

josevalim commented Mar 26, 2024

spunkedy commented Mar 26, 2024

josevalim commented Mar 26, 2024

spunkedy commented Mar 26, 2024 • edited Loading

josevalim commented Mar 26, 2024

jonatanklosko commented Mar 27, 2024

spunkedy commented Mar 27, 2024

spunkedy commented Mar 27, 2024

jonatanklosko Apr 2, 2024 • edited Loading

Choose a reason for hiding this comment

jonatanklosko Apr 2, 2024

Choose a reason for hiding this comment

spunkedy Apr 2, 2024

Choose a reason for hiding this comment

jonatanklosko Apr 2, 2024

Choose a reason for hiding this comment

github-actions bot commented Apr 2, 2024 • edited Loading

Uffizzi Ephemeral Environment deployment-49304

github-actions bot commented Apr 2, 2024 • edited Loading

spunkedy commented Apr 2, 2024 via email

spunkedy commented Apr 2, 2024 via email

spunkedy commented Apr 22, 2024

spunkedy commented Apr 22, 2024

jonatanklosko left a comment

Choose a reason for hiding this comment

jonatanklosko commented Apr 22, 2024

josevalim commented Apr 22, 2024

CLAassistant commented Mar 26, 2024 •

edited

Loading

spunkedy commented Mar 26, 2024 •

edited

Loading

jonatanklosko Apr 2, 2024 •

edited

Loading

github-actions bot commented Apr 2, 2024 •

edited

Loading

Uffizzi Ephemeral Environment `deployment-49304`

github-actions bot commented Apr 2, 2024 •

edited

Loading