Possible Startup Issue with Payara

[smillidge]

Very very occasionally typically after unzipping a brand new installation we see the following error

MultiException stack 1 of 5
org.glassfish.hk2.api.UnsatisfiedDependencyException: There was no object available for injection at SystemInjecteeImpl(requiredType=HazelcastRuntimeConfiguration,parent=HazelcastCore,qualifiers={@javax.inject.Named(value=default-instance-name)},position=-1,optional=false,self=false,unqualified=null,525022973)
…
MultiException stack 2 of 5
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of fish.payara.nucleus.hazelcast.HazelcastCore errors were found
…
MultiException stack 3 of 5
java.lang.IllegalStateException: Unable to perform operation: resolve on fish.payara.nucleus.hazelcast.HazelcastCore
…
MultiException stack 4 of 5
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of fish.payara.nucleus.exec.ClusterExecutionService errors were found
…
MultiException stack 5 of 5
java.lang.IllegalStateException: Unable to perform operation: resolve on fish.payara.nucleus.exec.ClusterExecutionService

and the server fails to start. Restarting the server always results in a successful start up.

I unzipped and started a domain 10+ times and saw the error once.

[mikecroft]

I created a new branch in the payara-vagrant repo to work with Payara prerelease on Ubuntu 15.04 (with OpenJDK 8u45 from the default repos) and I see this issue repeatedly. Running asadmin start-domain payaradomain always fails with that Vagrant box with the above MultiException, it doesn't start on subsequent attempts.

[mikecroft]

To use the Vagrantfile above to verify:

1. Install Vagrant from http://www.vagrantup.com/downloads.html
2. git clone -b prerelease https://github.com/payara/payara-vagrant.git
3. run vagrant up to download the Virtual Box machine image and start up (starts with 4GB ram by default)
4. run vagrant ssh to log in to the box
5. cd /opt/payara/payara-prerelease/payara41
6. bin/asadmin start-domain payaradomain

[smillidge]

Try adding

<batch-runtime-configuration></batch-runtime-configuration>
<hazelcast-runtime-configuration></hazelcast-runtime-configuration>
<diagnostic-service></diagnostic-service>

empty hazelcast-runtime-configuration between the batch-runtime and the diagnostic-service sections before running it to see if that resolves it.

[mikecroft]

that worked first time. There were two diagnostic-service empty elements and no batch-runtime-configuration element, though, so before it looked like:

76       <connector-service></connector-service>
77       <transaction-service tx-log-dir="${com.sun.aas.instanceRoot}/logs"></transaction-service>
78       <diagnostic-service />
79       <security-service>

and after, the empty element got expanded and the batch element added:

76       <connector-service></connector-service>
77       <transaction-service tx-log-dir="${com.sun.aas.instanceRoot}/logs"></transaction-service>
78       <hazelcast-runtime-configuration></hazelcast-runtime-configuration>
79       <batch-runtime-configuration></batch-runtime-configuration>
80       <diagnostic-service></diagnostic-service>
81       <security-service>