HADOOP-17396. ABFS: testRenameFileOverExistingFile Fails after Contract test update

[snvijaya]

Post updates to rename on existing file test, ABFS contract test is having failure. Updates were made in the AbstractContractTest class in https://issues.apache.org/jira/browse/HADOOP-17365.

To align to test expectation of no exception but a false return from rename, ABFS tests need config "fs.contract.rename-returns-false-if-dest-exists" set to true.

Also the abstract class was reusing a static test folder path. Hence on test re-run, where same storage container is used, source and destination path creations will fail with FileAlreadyExists, leading to test failures. This is also fixed.

[Test results pasted at the end of conversation tab].

[snvijaya]

HNS-OAuth

[INFO] Results:
[INFO]
[INFO] Tests run: 88, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 66
[INFO] Results:
[INFO]
[WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 141

HNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 88, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemDelegationSAS.testCheckAccess:94 » AbfsRestOperation O...
[ERROR] ITestAzureBlobFileSystemDelegationSAS.testRootPath:343 » AbfsRestOperation Ope...
[INFO]
[ERROR] Tests run: 457, Failures: 0, Errors: 2, Skipped: 24
[INFO] Results:
[INFO]
[WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 24

NonHNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 88, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 245
[INFO] Results:
[INFO]
[WARNING] Tests run: 208, Failures: 0, Errors: 0, Skipped: 24

SAS Test failure is tracked in JIRA: https://issues.apache.org/jira/browse/HADOOP-17397

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 21s		trunk passed
+1 💚	compile	0m 35s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	0m 29s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	0m 22s		trunk passed
+1 💚	mvnsite	0m 34s		trunk passed
+1 💚	shadedclient	18m 8s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 28s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	0m 25s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	0m 57s		Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 54s		trunk passed
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 29s		the patch passed
+1 💚	compile	0m 30s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	0m 30s		the patch passed
+1 💚	compile	0m 24s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	0m 24s		the patch passed
+1 💚	checkstyle	0m 15s		the patch passed
+1 💚	mvnsite	0m 27s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	shadedclient	16m 53s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 22s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	0m 21s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	findbugs	0m 56s		the patch passed
			_ Other Tests _
+1 💚	unit	1m 23s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 29s		The patch does not generate ASF License warnings.
		83m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2491/1/artifact/out/Dockerfile
GITHUB PR	#2491
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 7105c78e7057 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 235947e
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2491/1/testReport/
Max. process+thread count	570 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2491/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

[steveloughran]

+1, merged to trunk and branch-3.3; sorry for this. I would have liked object store tests, but we can't insist on that for hadoop-common stuff.

regarding the setup change, test teardown is meant to clean that up...what has probably happened is that the failure of the first attempt left things in a mess. Happens more to me when I'm working in an IDE and stopping test runs before getting to teardown(). Nothing wrong in adding the code though.