[REQ] support for app-level patching

[sherifkayad]

What is your question?

Hey there 👋 .. First of all thanks for this great project! .. Just was curious, if there's a plan to also (maybe conditionally) support patching application specific dependencies and not only OS-level dependencies.

Some background / example context: A Spring Boot App that's running on eclipse-temurin:17-jre-alpine might have the list of vulnerabilties below:

myregistry.com/some-app:latest (alpine 3.17.3)

Total: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 2, HIGH: 0, CRITICAL: 0)

┌────────────┬───────────────┬──────────┬───────────────────┬───────────────┬────────────────────────────────────────────────────────────┐
│  Library   │ Vulnerability │ Severity │ Installed Version │ Fixed Version │                           Title                            │
├────────────┼───────────────┼──────────┼───────────────────┼───────────────┼────────────────────────────────────────────────────────────┤
│ libcrypto3 │ CVE-2023-1255 │ MEDIUM   │ 3.0.8-r3          │ 3.0.8-r4      │ Input buffer over-read in AES-XTS implementation on 64 bit │
│            │               │          │                   │               │ ARM                                                        │
│            │               │          │                   │               │ https://avd.aquasec.com/nvd/cve-2023-1255                  │
├────────────┤               │          │                   │               │                                                            │
│ libssl3    │               │          │                   │               │                                                            │
│            │               │          │                   │               │                                                            │
│            │               │          │                   │               │                                                            │
└────────────┴───────────────┴──────────┴───────────────────┴───────────────┴────────────────────────────────────────────────────────────┘
2023-05-12T17:08:53.318+0200    INFO    Table result includes only package filenames. Use '--format json' option to get the full path to the package file.

Java (jar)

Total: 8 (UNKNOWN: 0, LOW: 0, MEDIUM: 5, HIGH: 1, CRITICAL: 2)

┌──────────────────────────────────────────────────┬──────────────────┬──────────┬───────────────────┬───────────────┬──────────────────────────────────────────────────────────────┐
│                     Library                      │  Vulnerability   │ Severity │ Installed Version │ Fixed Version │                            Title                             │
├──────────────────────────────────────────────────┼──────────────────┼──────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ org.springframework:spring-web (application.jar) │ CVE-2016-1000027 │ CRITICAL │ 5.3.27            │ 6.0.0         │ spring: HttpInvokerServiceExporter readRemoteInvocation      │
│                                                  │                  │          │                   │               │ method untrusted java deserialization                        │
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2016-1000027                 │
├──────────────────────────────────────────────────┼──────────────────┤          ├───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ org.yaml:snakeyaml (application.jar)             │ CVE-2022-1471    │          │ 1.30              │ 2.0           │ Constructor Deserialization Remote Code Execution            │
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2022-1471                    │
│                                                  ├──────────────────┼──────────┤                   ├───────────────┼──────────────────────────────────────────────────────────────┤
│                                                  │ CVE-2022-25857   │ HIGH     │                   │ 1.31          │ Denial of Service due to missing nested depth limitation for │
│                                                  │                  │          │                   │               │ collections                                                  │
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2022-25857                   │
│                                                  ├──────────────────┼──────────┤                   │               ├──────────────────────────────────────────────────────────────┤
│                                                  │ CVE-2022-38749   │ MEDIUM   │                   │               │ Uncaught exception in                                        │
│                                                  │                  │          │                   │               │ org.yaml.snakeyaml.composer.Composer.composeSequenceNode     │
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2022-38749                   │
│                                                  ├──────────────────┤          │                   │               ├──────────────────────────────────────────────────────────────┤
│                                                  │ CVE-2022-38750   │          │                   │               │ Uncaught exception in                                        │
│                                                  │                  │          │                   │               │ org.yaml.snakeyaml.constructor.BaseConstructor.constructObj- │
│                                                  │                  │          │                   │               │ ect                                                          │
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2022-38750                   │
│                                                  ├──────────────────┤          │                   │               ├──────────────────────────────────────────────────────────────┤
│                                                  │ CVE-2022-38751   │          │                   │               │ Uncaught exception in                                        │
│                                                  │                  │          │                   │               │ java.base/java.util.regex.Pattern$Ques.match                 │       
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2022-38751                   │       
│                                                  ├──────────────────┤          │                   ├───────────────┼──────────────────────────────────────────────────────────────┤       
│                                                  │ CVE-2022-38752   │          │                   │ 1.32          │ Uncaught exception in java.base/java.util.ArrayList.hashCode │       
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2022-38752                   │       
│                                                  ├──────────────────┤          │                   │               ├──────────────────────────────────────────────────────────────┤       
│                                                  │ CVE-2022-41854   │          │                   │               │ DoS via stack overflow                                       │       
│                                                  │                  │          │                   │               │ https://avd.aquasec.com/nvd/cve-2022-41854                   │       
└──────────────────────────────────────────────────┴──────────────────┴──────────┴───────────────────┴───────────────┴──────────────────────────────────────────────────────────────┘  

With Copa, I am so glad that I can get libcrypto3 & libssl3 patched and hence get no base image vulnerabilties anymore. However, what about the lovely snakeyaml in my application? 😄 .. Despite using the latest (or maybe I could say fairly new) version of the Spring Boot framework, the upstream project didn't update the vulnerable library .. soooo

[sozercan]

@sherifkayad glad to hear copa worked to patch OS vulnerabilities!

We would love to patch app/framework level vulns but I think it depends on the language. Since copa does not have access to application source code and doesn't rebuild it from source, it will not work for all cases (like Go, for example). This requires more design work on how it may or may not work for each language.

This might be better fit for interpreted languages like python or javascript. This is assuming those packages can be tracked down to and pulled from repositories like npm or pypi, etc.

I am not super familiar with Java dependencies, but if they can pulled from a central repo without rebuilding the app and ensure updates won't have breaking changes, than it might be possible.

[sherifkayad]

@sozercan thanks for your feedback 🙂 .. Java dependencies can be pulled / tracked from the Maven central repository (exactly the same idea like NPM).

However, I think you are right with the re-compilation issue to make sure the app would work still as expected. Probably for NPM (or even Go) you have to re-compile the app anyways. Isn't that right?

[RobertKielty]

Not only would a userland application have to be recompiled, CI for the application would have to be re-run to ensure that expected behaviors of the application remained the same.

Would it make sense to have Copacetic deliver a patched (but fundamentally untested) userland application? I think not, because CI tools fulfill that function for userland applications already and the variablity accross how applications are tested using those tools is just too great.

The Copacetic docs should probably note that patching userland applications is out of scope for the project if it has not done so already?

[sozercan]

We are working on a related project called Dalec to be able to define and create applications, which can then be patched using copa.

With Dalec, users can create a spec to define and create application packages, and then containers with minimal runtime dependencies. After testing and publishing the application packages and containers, and then installing via applicable package managers, copa will be able to "patch" them. Dalec is in early stages right now, any feedback is welcome!

[ashnamehrotra]

Java patching investigation - https://docs.google.com/document/d/1CqoGLv5sLpUIay9-jwfZgiq8DN5fGskhGe92ObR_8ao/edit?usp=sharing

[reneleonhardt]

Java patching investigation - https://docs.google.com/document/d/1CqoGLv5sLpUIay9-jwfZgiq8DN5fGskhGe92ObR_8ao/edit?usp=sharing

If you need a challenge when Hello World is too easy... 😅
https://hub.docker.com/r/confluentinc/cp-kafka is notoriously known for never rebuilding images (OS patches) and using the oldest app dependencies (even if newer patch releases require no code changes and fix high or critical security vulnerabilities like in ZooKeeper).
But be warned, their images are complicated and huge because they use not only 1 classpath, but 2 classpath folders, with slightly different dependency versions... it's multiple images in one 🤷