Get full IT working again (#523)

* Spanner Works 1. remove comments from pom.xml 2. explicitly pass the SystemPropertyValues in local pom — not sure how it worked earlier. 3. Fix a comment in the QuickStart sample 4. Restore stdOut in both IT samples * Minor README tweak storage-transfer * ErrorProne 1. Turn on ErrorProne - `mvn clean verify -DskipTests` passes 2. Fixes for ErrorProne - mostly removing compiler plugin, java 1.8 for most samples. (No more Java 5) 3. Contributing now mentions gradle and testing 4. Fix storage-transfer README 5. back out some changes to MAVEN_OPTS * Debugging Magic 1. What project are these tests being run on? 2. Skip spanner for now. * change project_ID 1. enabled cloud-samples-tests 2. change the project 3. remove the maven memory hack. 4. remove the debugging code. * revert to j-d-s-t for projectID that BQ uses
GoogleCloudPlatform · Feb 17, 2017 · f466b49 · f466b49
1 parent f9ea543
commit f466b49
Show file tree

Hide file tree

Showing 14 changed files with 96 additions and 112 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -42,8 +42,8 @@ before_install:
 - openssl aes-256-cbc -K $encrypted_eb858daba67b_key -iv $encrypted_eb858daba67b_iv -in secrets.env.enc -out secrets.env -d
   && set +x && source secrets.env && set -x
   || true
+# Cross project - GOOGLE_APPLICATION_CREDENTIALS uses cloud-docs-tests, but BQ uses G_C_P (argh!)
 - export GOOGLE_CLOUD_PROJECT=java-docs-samples-tests
-- export MAVEN_OPTS='-Xmx6g -Xms1g -XX:-UseGCOverheadLimit -XX:+UseG1GC -XX:+PrintFlagsInitial -XX:+PrintFlagsFinal'
 # Skip the install step, since Maven will download the dependencies we need
 # when the test build runs.
 # http://stackoverflow.com/q/31945809/101923

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,5 +1,11 @@
 # How to become a contributor and submit your own code
 
+* [Contributor License Agreements](#Contributor-License-Agreements)
+* [Contributing a Patch](#Contributing-a-Patch)
+* [Build Tools](#build-tools)
+* [Integration Testing](#testing)
+* [Style](#Style)
+
 ## Contributor License Agreements
 
 We'd love to accept your sample apps and patches! Before we can take them, we
@@ -31,6 +37,35 @@ accept your pull requests.
 1. Ensure that your code has an appropriate set of unit tests which all pass.
 1. Submit a pull request.
 
+## Build Tools
+
+All new samples should build and run integration tests with both [Maven](https://maven.apache.org/) and [Gradle](https://gradle.org/).
+
+## Testing
+
+All samples must have Integration Tests (ie. They need to run against a real service) that run with
+`mvn verify` & `gradle build test`.  If we need to enable an API, let us know.
+
+Your `build.gradle` should have the following section:
+
+```groovy
+
+test {
+  useJUnit()
+  testLogging.showStandardStreams = true
+  beforeTest { descriptor ->
+     logger.lifecycle("test: " + descriptor + "  Running")
+  }
+
+  onOutput { descriptor, event ->
+     logger.lifecycle("test: " + descriptor + ": " + event.message )
+  }
+  afterTest { descriptor, result ->
+    logger.lifecycle("test: " + descriptor + ": " + result )
+  }
+}
+```
+
 ## Style
 
 Samples in this repository follow the [Google Java Style Guide][java-style].
@@ -47,7 +82,7 @@ tool or IntelliJ plugin.
 
 ### Running the Linter
 
-To run the checkstyle plugin on an existing sample, run
+To run the checkstyle & ErrorProne plugins on an existing sample, run
 
 ```shell
 mvn clean verify -DskipTests
@@ -73,3 +108,5 @@ uses the common Checkstyle configuration.
 ```
 
 This is just used for testing. The sample should build without a parent defined.
+
+
diff --git a/datastore/pom.xml b/datastore/pom.xml
@@ -31,6 +31,12 @@
   <description>
     Example snippets for Datastore concepts and getting started documentation.
   </description>
+
+  <properties>
+    <maven.compiler.target>1.8</maven.compiler.target>
+    <maven.compiler.source>1.8</maven.compiler.source>
+  </properties>
+
   <dependencies>
     <dependency>
       <groupId>com.google.cloud</groupId>
@@ -49,15 +55,7 @@
   <build>
     <plugins>
 <!-- // [START maven]-->
-      <plugin>
-        <groupId>org.apache.maven.plugins</groupId>
-        <artifactId>maven-compiler-plugin</artifactId>
-        <version>2.5.1</version>
-        <configuration>
-          <source>1.7</source>
-          <target>1.7</target>
-        </configuration>
-      </plugin>
+
 <!-- // [END maven]-->
 <!-- // [START exec] -->
       <plugin>

diff --git a/logging/pom.xml b/logging/pom.xml
@@ -12,6 +12,11 @@
     <relativePath>..</relativePath>
   </parent>
 
+  <properties>
+    <maven.compiler.target>1.8</maven.compiler.target>
+    <maven.compiler.source>1.8</maven.compiler.source>
+  </properties>
+
   <dependencies>
     <dependency>
         <groupId>com.google.apis</groupId>
@@ -43,19 +48,4 @@
     </dependency>
   </dependencies>
 
-  <build>
-      <sourceDirectory>src/main/java</sourceDirectory>
-    <plugins>
-      <plugin>
-        <groupId>org.apache.maven.plugins</groupId>
-        <artifactId>maven-compiler-plugin</artifactId>
-        <version>3.2</version>
-        <configuration>
-          <source>5</source>
-          <target>5</target>
-        </configuration>
-      </plugin>
-    </plugins>
-  </build>
-
 </project>
diff --git a/monitoring/v2/pom.xml b/monitoring/v2/pom.xml
@@ -20,6 +20,11 @@
     </repository>
   </repositories>
 
+  <properties>
+    <maven.compiler.target>1.8</maven.compiler.target>
+    <maven.compiler.source>1.8</maven.compiler.source>
+  </properties>
+
   <dependencies>
     <dependency>
       <groupId>com.google.apis</groupId>
@@ -68,19 +73,4 @@
     </dependency>
   </dependencies>
 
-  <build>
-      <sourceDirectory>src/main/java</sourceDirectory>
-    <plugins>
-      <plugin>
-        <groupId>org.apache.maven.plugins</groupId>
-        <artifactId>maven-compiler-plugin</artifactId>
-        <version>3.2</version>
-        <configuration>
-          <source>5</source>
-          <target>5</target>
-        </configuration>
-      </plugin>
-    </plugins>
-  </build>
-
 </project>
diff --git a/pom.xml b/pom.xml
@@ -25,7 +25,7 @@
     <parent>
         <groupId>com.google.cloud.samples</groupId>
         <artifactId>shared-configuration</artifactId>
-        <version>1.0.1</version>
+        <version>1.0.2</version>
     </parent>
 
     <properties>
@@ -142,14 +142,7 @@
             <artifactId>appengine-api-1.0-sdk</artifactId>
             <version>${appengine.sdk.version}</version>
         </dependency>
-<!--
-        <dependency>
-            <groupId>javax.servlet</groupId>
-            <artifactId>servlet-api</artifactId>
-            <version>2.5</version>
-            <scope>provided</scope>
-        </dependency>
- -->
+
         <dependency>
             <groupId>jstl</groupId>
             <artifactId>jstl</artifactId>

diff --git a/spanner/cloud-client/pom.xml b/spanner/cloud-client/pom.xml
@@ -95,6 +95,18 @@
           </execution>
         </executions>
       </plugin>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-failsafe-plugin</artifactId>
+        <version>2.19.1</version>
+        <configuration>
+          <systemPropertyVariables>
+            <spanner.test.instance>${spanner.test.instance}</spanner.test.instance>
+            <spanner.sample.database>${spanner.sample.database}</spanner.sample.database>
+            <spanner.quickstart.database>${spanner.quickstart.database}</spanner.quickstart.database>
+          </systemPropertyVariables>
+        </configuration>
+      </plugin>
     </plugins>
   </build>
 

diff --git a/spanner/cloud-client/src/main/java/com/example/spanner/QuickstartSample.java b/spanner/cloud-client/src/main/java/com/example/spanner/QuickstartSample.java
@@ -40,7 +40,7 @@ public static void main(String... args) throws Exception {
     SpannerOptions options = SpannerOptions.newBuilder().build();
     Spanner spanner = options.getService();
 
-    // Name of your database. Eg: projects/my-project/instances/instanceId/databases/databaseId
+    // Name of your instance & database.
     String instanceId = args[0];
     String databaseId = args[1];
     try {

diff --git a/spanner/cloud-client/src/test/java/com/example/spanner/QuickstartSampleIT.java b/spanner/cloud-client/src/test/java/com/example/spanner/QuickstartSampleIT.java
@@ -37,6 +37,7 @@ public class QuickstartSampleIT {
   // This database needs to exist for test to pass.
   private String dbId = System.getProperty("spanner.quickstart.database");
   private ByteArrayOutputStream bout;
+  private PrintStream stdOut = System.out;
   private PrintStream out;
 
   @Before
@@ -48,7 +49,7 @@ public void setUp() {
 
   @After
   public void tearDown() {
-    System.setOut(null);
+    System.setOut(stdOut);
   }
 
   @Test

diff --git a/spanner/cloud-client/src/test/java/com/example/spanner/SpannerSampleIT.java b/spanner/cloud-client/src/test/java/com/example/spanner/SpannerSampleIT.java
@@ -39,16 +39,18 @@
 @SuppressWarnings("checkstyle:abbreviationaswordinname")
 public class SpannerSampleIT {
   // The instance needs to exist for tests to pass.
-  String instanceId = System.getProperty("spanner.test.instance");
-  String databaseId = System.getProperty("spanner.sample.database");
+  private final String instanceId = System.getProperty("spanner.test.instance");
+  private final String databaseId = System.getProperty("spanner.sample.database");
   DatabaseId dbId;
   DatabaseAdminClient dbClient;
 
   private String runSample(String command) throws Exception {
+    PrintStream stdOut = System.out;
     ByteArrayOutputStream bout = new ByteArrayOutputStream();
     PrintStream out = new PrintStream(bout);
     System.setOut(out);
     SpannerSample.main(new String[]{command, instanceId, databaseId});
+    System.setOut(stdOut);
     return bout.toString();
   }
 

diff --git a/speech/grpc/pom.xml b/speech/grpc/pom.xml
@@ -52,28 +52,10 @@ limitations under the License.
     <grpc-protobuf-version>1.0.3</grpc-protobuf-version>
     <maven-compiler-plugin-version>3.6.0</maven-compiler-plugin-version>
     <xolstice-protobuf-maven-plugin-version>0.5.0</xolstice-protobuf-maven-plugin-version>
-  </properties>
 
-  <profiles>
-    <profile>
-      <id>jdk7</id>
-      <activation>
-        <jdk>1.7</jdk>
-      </activation>
-      <properties>
-        <jdk.version>1.7</jdk.version>
-      </properties>
-    </profile>
-    <profile>
-      <id>jdk8</id>
-      <activation>
-        <jdk>1.8</jdk>
-      </activation>
-      <properties>
-        <jdk.version>1.8</jdk.version>
-      </properties>
-    </profile>
-  </profiles>
+    <maven.compiler.target>1.8</maven.compiler.target>
+    <maven.compiler.source>1.8</maven.compiler.source>
+  </properties>
 
   <!-- // [START dependency] -->
   <dependencies>
@@ -156,34 +138,10 @@ limitations under the License.
   </dependencies>
   <!-- // [END dependency] -->
 
-  <pluginRepositories>
-    <pluginRepository>
-      <releases>
-        <updatePolicy>never</updatePolicy>
-      </releases>
-      <snapshots>
-        <enabled>false</enabled>
-      </snapshots>
-      <id>central</id>
-      <name>Central Repository</name>
-      <url>https://repo.maven.apache.org/maven2</url>
-    </pluginRepository>
-  </pluginRepositories>
   <!-- // [START os-maven-plugin] -->
   <build>
     <!-- // [END os-maven-plugin] -->
     <plugins>
-      <plugin>
-        <artifactId>maven-compiler-plugin</artifactId>
-        <version>${maven-compiler-plugin-version}</version>
-        <configuration>
-          <source>${jdk.version}</source>
-          <target>${jdk.version}</target>
-          <showWarnings>true</showWarnings>
-          <showDeprecation>false</showDeprecation>
-          <compilerArgument>-Xlint:-options</compilerArgument>
-        </configuration>
-      </plugin>
       <plugin>
         <artifactId>maven-assembly-plugin</artifactId>
         <configuration>

diff --git a/speech/grpc/src/main/java/com/examples/cloud/speech/StreamingRecognizeClient.java b/speech/grpc/src/main/java/com/examples/cloud/speech/StreamingRecognizeClient.java
@@ -124,7 +124,7 @@ private TargetDataLine getAudioInputLine() {
     DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
     if (!AudioSystem.isLineSupported(info)) {
       throw new RuntimeException(String.format(
-            "Device doesn't support LINEAR16 mono raw audio format at {}Hz", samplingRate));
+            "Device doesn't support LINEAR16 mono raw audio format at {%d}Hz", samplingRate));
     }
     try {
       TargetDataLine line = (TargetDataLine) AudioSystem.getLine(info);

diff --git a/storage/storage-transfer/README.md b/storage/storage-transfer/README.md
@@ -54,18 +54,21 @@ Creating a one-time transfer from Amazon S3 to Google Cloud Storage.
   1. Set the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables.
 1. In AwsRequester.java, fill in the user-provided constants.
 1. Compile the package with
-   ```
-   mvn compile
-   ```
+
+```bash
+mvn compile
+```
+
 1. Run the transfer job with
-   ```
-   mvn exec:java \
-       -Dexec.mainClass="com.google.cloud.storage.storagetransfer.samples.AwsRequester" \
-       -DprojectId=your-google-cloud-project-id \
-       -DjobDescription="Sample transfer job from S3 to GCS." \
-       -DawsSourceBucket=your-s3-bucket-name \
-       -DgcsSinkBucket=your-gcs-bucket-name
-   ```
+
+```bash
+mvn exec:java \
+   -Dexec.mainClass="com.google.cloud.storage.storagetransfer.samples.AwsRequester" \
+   -DprojectId=your-google-cloud-project-id \
+   -DjobDescription="Sample transfer job from S3 to GCS." \
+   -DawsSourceBucket=your-s3-bucket-name \
+   -DgcsSinkBucket=your-gcs-bucket-name
+```
   1. Note the job ID in the returned Transfer Job.
 
 ## Transfer data from a standard Cloud Storage bucket to a Cloud Storage Nearline bucket

diff --git a/travis.sh b/travis.sh
@@ -64,7 +64,7 @@ common_travis_dir="$(travis_changed_files_parent)"
 
 [ -z "$common_travis_dir" ] || pushd "$common_travis_dir"
 
-./mvnw --batch-mode clean verify -DskipTests=$SKIP_TESTS | egrep -v "(^\[INFO\] Download|^\[INFO\].*skipping)"
+./mvnw  --batch-mode clean verify -e -DskipTests=$SKIP_TESTS | egrep -v "(^\[INFO\] Download|^\[INFO\].*skipping)"
 
 [ -z "$common_travis_dir" ] || popd