Erroneous Timeout: IOP00410037: Timeout while reading data in buffer manager

[glassfishrobot]

Sporadic error when transferring large byte arrays (e.g. 10 MB)

Bug is "com.sun.corba.ee.impl.encoding.BufferManagerReadStream.java"

The "underflow" method the "fragmentQueue.wait(orb.getORBData().fragmentReadTimeout())" returns unexplainable,
but it is NOT the due to the timeout, since it returnes after some milliseconds.
It looks like it get's notified without having the "fragmentQueue" properly enqueued, which I cannot explain.
So there is this "IOP00410037: Timeout while reading data in buffer manager",
but it is definitely no time out, also because it is thrown after some milliseconds, and this time out actually is 18 seconds.

This can be easily reproduced with the following very simple test.
It takes about a couple hundred tries (might take some minutes) until I get this error.
Even when client and server run on the same machine.

---

package de.test.client;

import javax.naming.Context;
import javax.naming.InitialContext;
import de.test.service.remote.TimeoutTestService;

public class TimeoutTestClient {
private static int tryNumber = 0;
private static boolean running = true;

public static void main(String[] args) throws Exception {
Context initialContext = new InitialContext();
TimeoutTestService timeoutTestService = (TimeoutTestService) initialContext.lookup(TimeoutTestService.class.getName());

long startTime = 0;
while (running) {
try

{ tryNumber++; startTime = System.currentTimeMillis(); timeoutTestService.getBytes(10000000); }

catch (Exception e)

{ System.out.println("Error occured at try number: " + tryNumber + " after " + (System.currentTimeMillis() - startTime) + " millis"); e.printStackTrace(); running = false; }

}
}
}

---

package de.test.service.remote;

import javax.ejb.Remote;

@Remote
public interface TimeoutTestService {
byte[] getBytes(int size);
}

---

package de.test;

import javax.ejb.Stateless;
import de.test.service.remote.TimeoutTestService;

@stateless
public class TimeoutTestServiceBean implements TimeoutTestService {
public byte[] getBytes(int size) {
byte[] bytes = new byte[size];
for (int i = 0; i < bytes.length; i++)

{ bytes[i] = 0x42; }

return bytes;
}
}

Environment

Windows 7, Glassfish 3.1, jdk1.6.0_26

Affected Versions

[current]

[glassfishrobot]

@glassfishrobot Commented
Reported by wigun

[glassfishrobot]

@glassfishrobot Commented
wigun said:
Suggested solution: Check whether the bufferReadManagerTimeout is real.

**BufferManagerReadStream.java**...

@Transport
public ByteBufferWithInfo underflow (ByteBufferWithInfo bbwi)
...

boolean interrupted = false;
int fragmentReadTimeout = orb.getORBData().fragmentReadTimeout();
long timeInMillisBeforeFragmentQueueWait = System.currentTimeMillis();
try {
    // Bug 6372405
    fragmentQueue.wait(fragmentReadTimeout);
} catch (InterruptedException e) {
    interrupted = true ;
}

// Bug 6372405 if (!interrupted && fragmentQueue.size() == 0 && (System.currentTimeMillis() - timeInMillisBeforeFragmentQueueWait) >= fragmentReadTimeout) {
    throw wrapper.bufferReadManagerTimeout() ;
}

[glassfishrobot]

@glassfishrobot Commented
wigun said:
Explanation: See java doc http://docs.oracle.com/javase/6/docs/api/java/lang/Object.html:

"A thread can also wake up without being notified, interrupted, or timing out, a so-called spurious wakeup. While this will rarely occur in practice, applications must guard against it by testing for the condition that should have caused the thread to be awakened, and continuing to wait if the condition is not satisfied."

[glassfishrobot]

@glassfishrobot Commented
wigun said:
Alternative (better?) solution:

**BufferManagerReadStream.java**...
    int fragmentReadTimeout = orb.getORBData().fragmentReadTimeout();
    long timeBeforeWait = System.currentTimeMillis();
    boolean interrupted = false ;
    try {
        while (fragmentQueue.size() == 0 && System.currentTimeMillis() < (timeBeforeWait + fragmentReadTimeout)) {
            // Bug 6372405
            fragmentQueue.wait(fragmentReadTimeout);
        }
    } catch (InterruptedException e) {
        interrupted = true ;
    }
...

[glassfishrobot]

@glassfishrobot Commented
jthoennes said:
We have filed an Oracle Service Request (SR 3-5014082933)
and believe we are also hit by this issues since we load large amounts of data every day.

Sporadically, the download fails with the above error message.

See also the old bug GLASSFISH-2872 were Ken Cavanaugh fixed an issue by guarding against such thread wakeups.

So please: Any answer from development here?

[glassfishrobot]

@glassfishrobot Commented
boernd said:
We manually patched the glassfish-corba-orb.jar file with the "Alternative (better?) solution" which seems to work for us so far. If anybody wants to have the modified jar file, plz contact me.

Regards
Bernd

[glassfishrobot]

@glassfishrobot Commented
cwolf77 said:
We are facing this problem with glassfish-3.2.1.
Is there any plan to fix this issue or is there an official patch available somewhere?

Thx
Kind Regards,
Christian

[glassfishrobot]

@glassfishrobot Commented
ebratt said:
If you have a support contract, please open a support case and reference this JIRA. Thank you.

[glassfishrobot]

@glassfishrobot Commented
alfredo_5869 said:
Hi Bernard, i need jar file modified, you can send me please?

Thanks!

[glassfishrobot]

@glassfishrobot Commented
This issue was imported from java.net JIRA GLASSFISH_CORBA-6

[glassfishrobot]

@KTakita Commented
Hello.

Did the development team decide how to fix this issue?
And are you planning to fix it?

[glassfishrobot]

• Issue Imported From: Erroneous Timeout: IOP00410037: Timeout while reading data in buffer manager javaee/glassfish-corba#6
• Original Issue Raised By:@glassfishrobot
• Original Issue Assigned To: @russgold

[ahubold]

I have proposed a pull request to fix this bug. Please have a look: #41

We repeatedly see errors caused by this bug. It would be great if you could treat this with high priority.

For the record, the same bug has been reported to the Oracle ORB but has never been fixed:. See https://bugs.openjdk.java.net/browse/JDK-8160235