PAYARA-3811: Optimize deployment times

[pdudits]

Description

This is a feature.

Profiling of deployment of large EAR with tens of EJB module reveals, that significant amount of deployment time was spend in filesystem access, especially on Windows. Thousands of calls were made to File.exists() and File.isDirectory. This was caused by ASURLClassLoader, which delegated all of resource existence checks to filesystem and FileArchive doing recursive folder scanning.

ASURLClassLoader did have map, that should serve as cache, but it cached positive lookups, whereas negative lookups are more expensive -- because every possible class source is scanned in order, and therefore there may be tens of negative lookups until positive one is found. Also, the use of this map was disabled because of generated folders.

This PR replaces these hot code paths:

• ASURLClassloader walks each class folder and caches up to three levels of subdirectories. This give precise enough hit rate to avoid file misses
• Since some of the folders contained generated classes, which are constructed after classloader is created, it will watch for filesystem changes to extend its set of prefixes.
• FileArchive uses file tree walking instead of recursive calls to file methods

Testing

New tests

DirWatcherTest tests the new decider for possibly valid files.

Testing Performed

Existing unit tests for FileArchive.

Ran against private large .ear with following results:

[Payara 5.191] [INFO] [] [javax.enterprise.system.core] [
app was successfully deployed in 72 787 milliseconds.]] 
[Payara 5.194-SNAPSHOT] [INFO] [] [javax.enterprise.system.core] [[
app was successfully deployed in 51 453 milliseconds.]] 

Test suites executed

Keeping that for Jenkins.

Testing Environment

Windows 10, 1.8.0_221, Maven 3.6.2

Notes for Reviewers

The PR only affects performance of directory-based classloaders (i. e. EJB generated folders, WEB-INF/classes), handling of JAR files is unaffected.

[pdudits]

Jenkins test please

[pdudits]

Jenkins test please

[jbee]

It would be nice if the file IO could be extracted to a black box util that just tells you "cached" if a certain file exists somewhere relative to a root folder. At least I'd assume this is the problem it solves.

[jbee]

privilegedCheckForFile is this some sort of Java accessibility check and if so should we keep having one?

[pdudits]

It's hard to tell because this codepath was not invoked for at least 8 years. But I can add privileged actions around filesystem access

[jbee]

I am happy to gamble on something that wasn't used in 8 years and not do any more than you already did. But that is just me. Don't know @Pandrex247 thinks about it.

[dmatej]

I would like to see all TCK tests results on this, because classloading magic changed over time ... in SGES was strict classpath prefix and suffix, isolated untouchable endorsed, and fragile JNDI/classloading issues with EAR/WAR/EJB(outer/inner) scopes. Then came GF3 and everything changed. Javadoc of this class from 2011 says @since JDK 1.4, so I would expect it was something around SGES and perhaps even SJAS7 mechanisms.

Simply said: it is a zombie :D

Btw on line 826 of original file is incorrect string concatenation and also append("\n") is slower than append('\n'). And hasItem manipulates it's parameter - it is used in cycle so it does it again and again until JIT maybe would optimize it. Blah.

[AlanRoth]

LGTM

[pdudits]

Jenkins test please

[pdudits]

Jenkins test please

[jbee]

LGTM but then I have no idea what the classloaders where supposed to do so I cannot judge the semantics in the context.

[pdudits]

Jenkins test please

[pdudits]

Jenkins test please