Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a means to disable or otherwise configure the DataNucleus L2 cache #493

Closed
nscuro opened this issue May 23, 2023 · 0 comments · Fixed by #494
Closed

Provide a means to disable or otherwise configure the DataNucleus L2 cache #493

nscuro opened this issue May 23, 2023 · 0 comments · Fixed by #494

Comments

@nscuro
Copy link
Collaborator

nscuro commented May 23, 2023

As identified in DependencyTrack/dependency-track#218 and DependencyTrack/dependency-track#903, the DataNucleus L2 cache is one of the areas that can prevent Alpine-based applications from being horizontally scalable or HA.

The L2 cache is currently enabled per default. This means that all objects written to the datastore will be cached in-memory:

Objects are placed in the L2 cache when you commit() the transaction of a PersistenceManager. This means that you only have datastore-persisted objects in that cache. Also, if an object is deleted during a transaction then at commit it will be removed from the L2 cache if it is present.

While this behavior works fine for normal CRUD-type applications, it does not for applications like Dependency-Track, that regularly iterate over multiple hundreds of thousands (or millions) of records, and modify them. With record sets in such high numbers, the cost-to-benefit ratio is simply not good.

For applications like Dependency-Track, global usage of the L2 cache is skyrocketing RAM usage, and puts unnecessary pressure on the GC.

Additionally, the L2 cache is only effective when records are fetched by their primary key. In modern applications, lookups by primary key are very rare, as most of them use secondary IDs intended for external consumption (e.g. UUIDs).

In order to support horizontal scalability, high availability, and a generally reduced resource footprint, it should be possible to globally disable the DataNucleus L2 cache.

As a nice-to-have, additional configuration options may be introduced to allow for usage of distributed caches like Redis or Hazelcast (which is supported by DN). Users wishing to utilize these integrations may do so.

nscuro added a commit to nscuro/Alpine that referenced this issue May 24, 2023
This change is mostly meant to address stevespringett#493, but may be useful for other use cases as well.

Due to the sheer amount of configuration options in DataNucleus (and potentially other frameworks), it is not practical to add `AlpineKey`s for all of them.

With this change, it is possible to disable the DataNucleus L2 cache by either:

* Setting the property `alpine.datanucleus.cache.level2.type=none` in `application.properties`, or
* Setting the environment variable `ALPINE_DATANUCLEUS_CACHE_LEVEL2_TYPE=none`

Signed-off-by: nscuro <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant