Connections left open in 4.1.10.0-RC1

[JWollnik]

We are currently testing 4.1.10.0-RC1 and observed error messages about connections left open:

13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - **************************************************************************************
13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ********  WARNING: POTENTIAL CONNECTION LEAK  ******************
13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ********  finalizer has run and found a connection left opened, please check your code to ensure that all connections are closed
13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ********  connection is:irods://username.homezone@dev:1247/homezone/home/johannes/main/1484830466782, will attempt to disconnect
13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - **************************************************************************************

Here's the code snippet used for testing:

	public static void main(String[] args) throws JargonException {
		IRODSFileSystem fs = null;
		try {
			fs = IRODSFileSystem.instance();
			IRODSSession session = fs.getIrodsSession();
			session.setX509TrustManager(SSL.getTrustManager());
			IRODSAccount account = createIrodsAccount(args);
			account.setAuthenticationScheme(AuthScheme.PAM);
			IRODSAccessObjectFactory f = fs.getIRODSAccessObjectFactory();
			f.authenticateIRODSAccount(account);
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			if (fs != null) {
				fs.close();
			}
			System.gc();
			System.runFinalization();
		}
	}

The error messages also occurr with standard authorization. In 4.0.2.6-RELEASE we have not seen such error messages.

[michael-conway]

Thanks! I'll do a quick test to verify. I'm about to do an RC2 that cleaned up some PAM + SSL negotiation issues, this may be related. The fs.close() should indeed take care of any open connections.

…

On Thu, Jan 19, 2017 at 8:13 AM, JWollnik ***@***.***> wrote: We are currently testing 4.1.10.0-RC1 and observed error messages about connections left open in our log: 13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ************************************************************************************** 13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ******** WARNING: POTENTIAL CONNECTION LEAK ****************** 13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ******** finalizer has run and found a connection left opened, please check your code to ensure that all connections are closed 13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ******** connection ***@***.***:1247/homezone/home/johannes/main/1484830466782, will attempt to disconnect 13:54:27.270 [Finalizer] ERROR org.irods.jargon.core.connection.AbstractConnection - ************************************************************************************** Here's the code snippet used for testing: public static void main(String[] args) throws JargonException { IRODSFileSystem fs = null; try { fs = IRODSFileSystem.instance(); IRODSSession session = fs.getIrodsSession(); session.setX509TrustManager(SSL.getTrustManager()); IRODSAccount account = createIrodsAccount(args); account.setAuthenticationScheme(AuthScheme.PAM); IRODSAccessObjectFactory f = fs.getIRODSAccessObjectFactory(); f.authenticateIRODSAccount(account); } catch (Exception e) { e.printStackTrace(); } finally { if (fs != null) { fs.close(); } System.gc(); System.runFinalization(); } } The error messages also occurr with standard authorization. In 4.0.2.6-RELEASE we have not seen such error messages. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#222>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABC-LWGTFv_GOyfVgZQz86roozru-Dysks5rT2F7gaJpZM4LoGEr> .

[marc-flesch]

Dear Michael,

any timelines when RC2 will be made available to the public?

Just asking,
Marc

[michael-conway]

I can do one in next half hour. Was about to anyhow to address some other field testing...

…

On Jan 19, 2017 9:55 AM, "Marc Flesch" ***@***.***> wrote: Dear Michael, any timelines when RC2 will be made available to the public? Just asking, Marc — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#222 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABC-LXhXzkp8rnzo8Kdhsx2t__gnkiPZks5rT3lSgaJpZM4LoGEr> .

[michael-conway]

Alternately, the 4.1.10.0-SNAPSHOT is also available on maven and is up-to-date. FWIW here is the test I added...I observe no finalizer messages, so I think other related issues that we cleaned up this week could have been at play.

/**
	 * refers to Connections left open in 4.1.10.0-RC1 #222
	 */
	@Test
	public void testConnAndFinalizationViaIrodsFileSystemBug222()
			throws Exception {

		int times = 50;

		IRODSFileSystem fs = null;

		for (int i = 0; i < times; i++) {

			try {
				fs = IRODSFileSystem.instance();
				IRODSSession session = fs.getIrodsSession();
				IRODSAccount account = testingPropertiesHelper
						.buildPamIrodsAccountFromTestProperties(testingProperties);
				account.setAuthenticationScheme(AuthScheme.PAM);
				IRODSAccessObjectFactory f = fs.getIRODSAccessObjectFactory();
				f.authenticateIRODSAccount(account);
			} catch (Exception e) {
				e.printStackTrace();
			} finally {
				if (fs != null) {
					fs.close();
				}
				System.gc();
				System.runFinalization();
			}

			Assert.assertTrue("should be no conns in map", fs.getIrodsSession()
					.getIRODSCommandsMap() == null);
		}

	}

[michael-conway]

would it be ok to test via the 4.1.10.0-SNAPSHOT?

Then if you see that cleaned up, it would be a point to pull the trigger on the rc2. There is another test out in the field on PAM this morning, and clearing these two issues would be a nice point to cut the RC2.

If you use Maven, the snapshots are available using these repos...

<repositories>
		<repository>
			<id>dice.repository snaps</id>
			<name>dice.repository.snapshots</name>
			<url>https://raw.github.com/DICE-UNC/DICE-Maven/master/snapshots</url>
			<releases>
				<enabled>true</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
				<updatePolicy>always</updatePolicy>
				<checksumPolicy>warn</checksumPolicy>
			</snapshots>
		</repository>
		<repository>
			<id>dice.repository</id>
			<name>dice.repository</name>
			<url>https://raw.github.com/DICE-UNC/DICE-Maven/master/releases</url>
			<releases>
				<enabled>true</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
				<updatePolicy>always</updatePolicy>
				<checksumPolicy>warn</checksumPolicy>
			</snapshots>
		</repository>
		
	</repositories>

and these coordinates

<dependency>
				<groupId>org.irods.jargon</groupId>
				<artifactId>jargon-core</artifactId>
				<version>${jargon.version}</version>
			</dependency>

with the version at 4.1.10.0-SNAPSHOT

Would that be OK as the plan?

[JWollnik]

Hi Michael,

thanks for your quick response. I tested against 4.1.10.0-SNAPSHOT and observed with the snippet from above no issues anymore!

However, I was playing around with new negotiation option and had the same connection leakage warning when using the NO_NEGOTIATION option with 4.1.10.0-SNAPSHOT.

	public static void main(String[] args) throws JargonException {
		IRODSFileSystem fs = null;
		try {
			fs = IRODSFileSystem.instance();
			IRODSSession session = fs.getIrodsSession();
			SettableJargonProperties props = new SettableJargonProperties(session.getJargonProperties());
			props.setNegotiationPolicy(ClientServerNegotiationPolicy.SslNegotiationPolicy.NO_NEGOTIATION);
			session.setJargonProperties(props);
			session.setX509TrustManager(SSL.getTrustManager());
			IRODSAccount account = createIrodsAccount(args);
			account.setAuthenticationScheme(AuthScheme.PAM);
			IRODSAccessObjectFactory f = fs.getIRODSAccessObjectFactory();
			f.authenticateIRODSAccount(account);
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			if (fs != null) {
				fs.close();
			}
			System.runFinalization();
			System.gc();
		}
	}

[michael-conway]

Awesome, I will expand the test coverage (we have been in the process of adding various permutations of negotiation and auth method through this 4.1.10 effort). I'll add that test with that additional stance and make sure it's OK. This is very helpful.

…

On Thu, Jan 19, 2017 at 11:38 AM, JWollnik ***@***.***> wrote: Hi Michael, thanks for your quick response. I tested against 4.1.10.0-SNAPSHOT and observed with the snippet from above no issues anymore! However, I was playing around with new negotiation option and had the same connection leakage warning when using the NO_NEGOTIATION option with 4.1.10.0-SNAPSHOT. public static void main(String[] args) throws JargonException { IRODSFileSystem fs = null; try { fs = IRODSFileSystem.instance(); IRODSSession session = fs.getIrodsSession(); SettableJargonProperties props = new SettableJargonProperties(session.getJargonProperties()); props.setNegotiationPolicy(ClientServerNegotiationPolicy.SslNegotiationPolicy.NO_NEGOTIATION); session.setJargonProperties(props); session.setX509TrustManager(SSL.getTrustManager()); IRODSAccount account = createIrodsAccount(args); account.setAuthenticationScheme(AuthScheme.PAM); IRODSAccessObjectFactory f = fs.getIRODSAccessObjectFactory(); f.authenticateIRODSAccount(account); } catch (Exception e) { e.printStackTrace(); } finally { if (fs != null) { fs.close(); } System.runFinalization(); System.gc(); } } — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#222 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABC-LXqYo37R8nSzMce4Xb8DE4_h75l9ks5rT5F9gaJpZM4LoGEr> .

[marc-flesch]

Thank you for looking into this Michael!

For us it would be of importance to have an official tag (e.g. RC2) to be able to embed your code in our testing and validation suite. Whenever you feel comfortable with your changes, can you please pull the trigger on RC2? That would be of great help for us. Thx Marc.

[michael-conway]

Understood, I'm looking at that other no-negotiation issue and if I can iron that out and get confirmation on testing of #215 that would be a nice point. I suspect either tomorrow or Monday latest if we can keep the changes in SNAPSHOT land so as not to thrash too much on release candidates?

…

On Thu, Jan 19, 2017 at 12:55 PM, Marc Flesch ***@***.***> wrote: Thank you for looking into this Michael! For us it would be of importance to have an official tag (e.g. RC2) to be able to embed your code in our testing and validation suite. Whenever you feel comfortable with your changes, can you please pull the trigger on RC2 for us? That would be of great help for us, thx Marc. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#222 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABC-LdCNxeqiZ-CM2D0UMfxpPOka2PEiks5rT6OugaJpZM4LoGEr> .

[michael-conway]

I added a test and can see those messages, will clean that up. NB that the message indicates the finalizer is properly closing them in the background.

[marc-flesch]

Thank you Michael,

we will include RC2 in our tomorrows nightly build and check.

Best,
Marc.

[michael-conway]

thanks, I'm testing it now in REST, Cloud Browser, and WebDav, so far so good, and I expect that full release this week barring any developments