Maven Plugin: support deferred failure when running in the integration test phase

[stefanwendelmann]

Which version and edition of Flyway are you using?

<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>5.2.4</version>

If this is not the latest version, can you reproduce the issue with the latest one as well?

(Many bugs are fixed in newer releases and upgrading will often resolve the issue)

is latest

Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)

maven-plugin

Which database are you using (type & version)?

MS SQL in Docker
mcr.microsoft.com/mssql/server:latest

Which operating system are you using?

Windows 10 + Docker Toolbox

What did you do?

(Please include the content causing the issue, any relevant configuration settings, the SQL statement that failed (if relevant) and the command you ran.)

<build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.18.1</version>
        <configuration>
          <skip>true</skip>
        </configuration>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.18.1</version>
        <executions>
          <execution>
            <id>run-tests</id>
            <phase>integration-test</phase>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      
      <plugin>
        <groupId>io.fabric8</groupId>
        <artifactId>docker-maven-plugin</artifactId>
        <version>0.28.0</version>
        <configuration>
          <useColor>false</useColor>
          <images>
            <image>
              <name>mcr.microsoft.com/mssql/server</name>
              <alias>mssql</alias>
              <run>
                <env>
                  <ACCEPT_EULA>Y</ACCEPT_EULA>
                  <SA_PASSWORD>${mssql.test.pw}</SA_PASSWORD>
                </env>
                <ports>
                  <port>${mssql.test.port}:1433</port>
                </ports>
                <wait>
                  <log>Recovery is complete. This is an informational message only. No user action is required.</log>
                  <time>20000</time>
                </wait>
              </run>
            </image>
          </images>
        </configuration>
        <executions>
          <execution>
            <id>start</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start</goal>
            </goals>
          </execution>
          <execution>
            <id>stop</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
        
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>sql-maven-plugin</artifactId>
        <version>1.5</version>
        <dependencies>
          <dependency>
            <groupId>com.microsoft.sqlserver</groupId>
            <artifactId>mssql-jdbc</artifactId>
            <version>7.0.0.jre8</version> 
          </dependency>
        </dependencies>
        <configuration>
          <driver>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver>
          <url>jdbc:sqlserver://${mssql.test.ip}:${mssql.test.port}</url>
          <username>sa</username>
          <password>${mssql.test.pw}</password>
        </configuration>
        <executions>
          <execution>
            <id>create-db</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>execute</goal>
            </goals>
            <configuration>
              <autocommit>true</autocommit>
              <sqlCommand>CREATE DATABASE QuoLoco COLLATE Latin1_General_CS_AS</sqlCommand>
            </configuration>
          </execution>
        </executions>
      </plugin>
      
      <plugin>
        <groupId>org.flywaydb</groupId>
        <artifactId>flyway-maven-plugin</artifactId>
        <version>5.2.4</version>
        <executions>
          <execution>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>migrate</goal>
            </goals>
            <configuration>
              <url>jdbc:sqlserver://${mssql.test.ip}:${mssql.test.port};databaseName=QuoLoco</url>
              <user>sa</user>
              <password>${mssql.test.pw}</password>
              <serverId>flyway-db-docker-test</serverId>
              <baselineOnMigrate>true</baselineOnMigrate>
              <baselineVersion>0</baselineVersion>
              <table>flyway_schema_history</table>
            </configuration>
          </execution>
        </executions>
        <dependencies>
          <dependency>
            <groupId>com.microsoft.sqlserver</groupId>
            <artifactId>mssql-jdbc</artifactId>
            <version>7.0.0.jre8</version> 
          </dependency>
        </dependencies>
      </plugin>
      
    </plugins>
  </build>

$ mvn verify                                                                                                                           
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------------< de.itout.ci.test:FlywayIntegrationTest >---------------
[INFO] Building FlywayIntegrationTest 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ FlywayIntegrationTest ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 17 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ FlywayIntegrationTest ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ FlywayIntegrationTest ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\Daten\nbprojects\test\FlywayIntegrationTest\src\test\resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ FlywayIntegrationTest ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ FlywayIntegrationTest ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ FlywayIntegrationTest ---
[INFO] Building jar: C:\Daten\nbprojects\test\FlywayIntegrationTest\target\FlywayIntegrationTest-0.0.1-SNAPSHOT.jar
[INFO] 
[INFO] --- docker-maven-plugin:0.28.0:start (start) @ FlywayIntegrationTest ---
[INFO] DOCKER> [mcr.microsoft.com/mssql/server:latest] "mssql": Start container 5fa8433aa6a2
[INFO] DOCKER> Pattern 'Recovery is complete. This is an informational message only. No user action is required.' matched for container 5fa8433aa6a2
[INFO] DOCKER> [mcr.microsoft.com/mssql/server:latest] "mssql": Waited on log out 'Recovery is complete. This is an informational message only. No user action is required.' 8398 ms
[INFO] 
[INFO] --- sql-maven-plugin:1.5:execute (create-db) @ FlywayIntegrationTest ---
[INFO] Executing commands
[INFO] 1 of 1 SQL statements executed successfully
[INFO] 
[INFO] --- flyway-maven-plugin:5.2.4:migrate (default) @ FlywayIntegrationTest ---
[INFO] Flyway Community Edition 5.2.4 by Boxfuse
[INFO] Database: jdbc:sqlserver://192.168.99.100:1433;useBulkCopyForBatchInsert=false;cancelQueryTimeout=-1;sslProtocol=TLS;jaasConfigurationName=SQLJDBCDriver;statementPoolingCacheSize=0;serverPreparedStatementDiscardThreshold=10;enablePrepareOnFirstPreparedStatementCall=false;fips=false;socketTimeout=0;authentication=NotSpecified;authenticationScheme=nativeAuthentication;xopenStates=false;sendTimeAsDatetime=true;trustStoreType=JKS;trustServerCertificate=false;TransparentNetworkIPResolution=true;serverNameAsACE=false;sendStringParametersAsUnicode=true;selectMethod=direct;responseBuffering=adaptive;queryTimeout=-1;packetSize=8000;multiSubnetFailover=false;loginTimeout=15;lockTimeout=-1;lastUpdateCount=true;encrypt=false;disableStatementPooling=true;databaseName=QuoLoco;columnEncryptionSetting=Disabled;applicationName=Microsoft JDBC Driver for SQL Server;applicationIntent=readwrite; (Microsoft SQL Server 14.0)
[INFO] Successfully validated 17 migrations (execution time 00:00.600s)
[INFO] Creating Schema History table: [QuoLoco].[dbo].[flyway_schema_history]
[INFO] Current version of schema [dbo]: << Empty Schema >>
[INFO] Migrating schema [dbo] to version 0.0.0 - Create Base
[WARNING] DB: Warning! The maximum key length for a clustered index is 900 bytes. The index 'PK_EmpfangseinheitenSensoreinheiten' has maximum length of 1060 bytes. For some combination of large values, the insert/update operation will fail. (SQL State: S0001 - Error Code: 1945)
[INFO] Migrating schema [dbo] to version 0.1.0 - Normalisierung SE-1003 DROP PK
[ERROR] Migration of schema [dbo] to version 0.1.0 - Normalisierung SE-1003 DROP PK failed! Changes successfully rolled back.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.108 s
[INFO] Finished at: 2018-12-19T09:28:55+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:5.2.4:migrate (default) on project FlywayIntegrationTest: org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException: 
[ERROR] Migration V0.1.0__Normalisierung_SE-1003_DROP_PK.sql failed
[ERROR] -----------------------------------------------------------
[ERROR] SQL State  : S00062
[ERROR] Error Code : 2812
[ERROR] Message    : Could not find stored procedure 'sp_executeSQL'.
[ERROR] Location   : C:\Daten\nbprojects\test\FlywayIntegrationTest\src\main\resources\db\migration\V0.1.0__Normalisierung_SE-1003_DROP_PK.sql (C:\Daten\nbprojects\test\FlywayIntegrationTest\src\main\resources\db\migration\V0.1.0__Normalisierung_SE-1003_DROP_PK.sql)
[ERROR] Line       : 1
[ERROR] Statement  : A Statement that will fail on purpose to test post-integration-test cleanup
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

What did you expect to see?

In would expect the post-integration-test to run and so stop my docker container.
The Test should fail and the build too but after post-integration.

What did you see instead?

Docker Container is still running, post-integration-test is not reached

Relations: fabric8io/docker-maven-plugin#915

Thanks in advance for your help.

[MikielAgutu]

I don't think we should expect this to work out of the box, unfortunately. It looks like we need to do some specific work to integrate the Maven plugin with Failsafe. See this comment in the Scala Test plugin repo:

[The plugin should] not fail the build by throwing MojoFailureException if there are any problems (tests which either had errors or failed). It just writes the results to a file...

...[The plugin] would need to create a summary report as output... Those mojos should never throw MojoFailureException. Then a VerifyMojo should be created to read the summary reports and throw MojoFailureException if the build needs to be failed.

We therefore won't commit to doing this work right away. However feel free to contribute a PR. The next comment in that thread has an example of how someone implemented a proof-of-concept solution for the Scala Test plugin. The code on his branch is here if you would like to use it as a starting point.

[DoodleBobBuffPants]

It seems like one of the solutions in this SO post is what you're after. Does this work for you?

[DoodleBobBuffPants]

Closing until further activity