Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to new client library and fixes broken link in README #427

Merged
merged 5 commits into from
Dec 1, 2016
Merged

Updates to new client library and fixes broken link in README #427

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
81d550f
Updates to new client library and fixes broken link in README
gguuss Nov 28, 2016
4baae60
Fixes error on test
gguuss Nov 28, 2016
a5f7fc0
Fixes type in test assertion
gguuss Nov 28, 2016
bd07872
Removes extraneous createService method, reduces line length in integ…
gguuss Nov 30, 2016
dad9f17
Merge branch 'master' into language-client
lesv Dec 1, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion language/analysis/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
This sample demonstrates the use of the [Google Cloud Natural Language API][NL-Docs]
for entity recognition.

[NL-Docs]: https://cloud.google.com/language/docs/
[NL-Docs]: https://cloud.google.com/natural-language/docs/

## Java Version

Expand Down
11 changes: 3 additions & 8 deletions language/analysis/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,9 @@ limitations under the License.
<dependencies>
<!-- [START dependencies] -->
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-language</artifactId>
<version>v1-rev1-1.22.0</version>
</dependency>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.22.0</version>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-language</artifactId>
<version>0.7.0</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
Expand Down
118 changes: 40 additions & 78 deletions language/analysis/src/main/java/com/google/cloud/language/samples/Analyze.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,21 @@

package com.google.cloud.language.samples;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.language.v1.CloudNaturalLanguage;
import com.google.api.services.language.v1.CloudNaturalLanguageScopes;
import com.google.api.services.language.v1.model.AnalyzeEntitiesRequest;
import com.google.api.services.language.v1.model.AnalyzeEntitiesResponse;
import com.google.api.services.language.v1.model.AnalyzeSentimentRequest;
import com.google.api.services.language.v1.model.AnalyzeSentimentResponse;
import com.google.api.services.language.v1.model.AnalyzeSyntaxRequest;
import com.google.api.services.language.v1.model.AnalyzeSyntaxResponse;
import com.google.api.services.language.v1.model.AnnotateTextRequest;
import com.google.api.services.language.v1.model.AnnotateTextResponse;
import com.google.api.services.language.v1.model.Document;
import com.google.api.services.language.v1.model.Entity;
import com.google.api.services.language.v1.model.EntityMention;
import com.google.api.services.language.v1.model.Features;
import com.google.api.services.language.v1.model.Sentiment;
import com.google.api.services.language.v1.model.Token;
import com.google.cloud.language.spi.v1.LanguageServiceClient;

import com.google.cloud.language.v1.AnalyzeEntitiesRequest;
import com.google.cloud.language.v1.AnalyzeEntitiesResponse;
import com.google.cloud.language.v1.AnalyzeSentimentResponse;
import com.google.cloud.language.v1.AnalyzeSyntaxRequest;
import com.google.cloud.language.v1.AnalyzeSyntaxResponse;
import com.google.cloud.language.v1.Document;
import com.google.cloud.language.v1.Document.Type;
import com.google.cloud.language.v1.EncodingType;
import com.google.cloud.language.v1.Entity;
import com.google.cloud.language.v1.EntityMention;
import com.google.cloud.language.v1.Sentiment;
import com.google.cloud.language.v1.Token;
import com.google.protobuf.Descriptors;

import java.io.IOException;
import java.io.PrintStream;
Expand All @@ -49,16 +42,7 @@
* A sample application that uses the Natural Language API to perform
* entity, sentiment and syntax analysis.
*/
@SuppressWarnings("serial")
public class Analyze {
/**
* Be sure to specify the name of your application. If the application name is {@code null} or
* blank, the application will log a warning. Suggested format is "MyCompany-ProductName/1.0".
*/
private static final String APPLICATION_NAME = "Google-LanguagAPISample/1.0";

private static final int MAX_RESULTS = 4;

/**
* Detects entities,sentiment and syntax in a document using the Natural Language API.
*/
Expand All @@ -73,7 +57,7 @@ public static void main(String[] args) throws IOException, GeneralSecurityExcept
String command = args[0];
String text = args[1];

Analyze app = new Analyze(getLanguageService());
Analyze app = new Analyze(createLanguageService());

if (command.equals("entities")) {
printEntities(System.out, app.analyzeEntities(text));
Expand All @@ -97,15 +81,17 @@ public static void printEntities(PrintStream out, List<Entity> entities) {
out.printf("%s\n", entity.getName());
out.printf("\tSalience: %.3f\n", entity.getSalience());
out.printf("\tType: %s\n", entity.getType());
if (entity.getMetadata() != null) {
for (Map.Entry<String, String> metadata : entity.getMetadata().entrySet()) {
if (entity.getMetadataMap() != null) {
for (Map.Entry<String, String> metadata : entity.getMetadataMap().entrySet()) {
out.printf("\tMetadata: %s = %s\n", metadata.getKey(), metadata.getValue());
}
}
if (entity.getMentions() != null) {
for (EntityMention mention : entity.getMentions()) {
for (Map.Entry<String, Object> mentionSetMember : mention.entrySet()) {
out.printf("\tMention: %s = %s\n", mentionSetMember.getKey(), mentionSetMember.getValue());
if (entity.getMentionsList() != null) {
for (EntityMention mention : entity.getMentionsList()) {
for (Map.Entry<Descriptors.FieldDescriptor, Object> mentionSetMember :
mention.getAllFields().entrySet()) {
out.printf("\tMention: %s = %s\n", mentionSetMember.getKey(),
mentionSetMember.getValue());
}
}
}
Expand Down Expand Up @@ -157,29 +143,16 @@ public static void printSyntax(PrintStream out, List<Token> tokens) {
/**
* Connects to the Natural Language API using Application Default Credentials.
*/
public static CloudNaturalLanguage getLanguageService()
throws IOException, GeneralSecurityException {
GoogleCredential credential =
GoogleCredential.getApplicationDefault().createScoped(CloudNaturalLanguageScopes.all());
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
return new CloudNaturalLanguage.Builder(
GoogleNetHttpTransport.newTrustedTransport(),
jsonFactory, new HttpRequestInitializer() {
@Override
public void initialize(HttpRequest request) throws IOException {
credential.initialize(request);
}
})
.setApplicationName(APPLICATION_NAME)
.build();
public static LanguageServiceClient createLanguageService() throws IOException{
return LanguageServiceClient.create();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems a little pedantic, no? Might be clearer to just use LanguageServiceClient.create() directly and just delete this method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, change added to integration test as well.

}

private final CloudNaturalLanguage languageApi;
private final LanguageServiceClient languageApi;

/**
* Constructs a {@link Analyze} which connects to the Cloud Natural Language API.
*/
public Analyze(CloudNaturalLanguage languageApi) {
public Analyze(LanguageServiceClient languageApi) {
this.languageApi = languageApi;
}

Expand All @@ -188,41 +161,30 @@ public Analyze(CloudNaturalLanguage languageApi) {
*/
public List<Entity> analyzeEntities(String text) throws IOException {
AnalyzeEntitiesRequest request =
new AnalyzeEntitiesRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"))
.setEncodingType("UTF16");
CloudNaturalLanguage.Documents.AnalyzeEntities analyze =
languageApi.documents().analyzeEntities(request);

AnalyzeEntitiesResponse response = analyze.execute();
return response.getEntities();
AnalyzeEntitiesRequest.newBuilder()
.setDocument(Document.newBuilder().setContent(text).setType(Type.PLAIN_TEXT))
.setEncodingType(EncodingType.UTF16).build();
AnalyzeEntitiesResponse response = languageApi.analyzeEntities(request);
return response.getEntitiesList();
}

/**
* Gets {@link Sentiment} from the string {@code text}.
*/
public Sentiment analyzeSentiment(String text) throws IOException {
AnalyzeSentimentRequest request =
new AnalyzeSentimentRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"));
CloudNaturalLanguage.Documents.AnalyzeSentiment analyze =
languageApi.documents().analyzeSentiment(request);

AnalyzeSentimentResponse response = analyze.execute();
AnalyzeSentimentResponse response = languageApi.analyzeSentiment(
Document.newBuilder().setContent(text).setType(Type.PLAIN_TEXT).build());
return response.getDocumentSentiment();
}

/**
* Gets {@link Token}s from the string {@code text}.
*/
public List<Token> analyzeSyntax(String text) throws IOException {
AnalyzeSyntaxRequest request =
new AnalyzeSyntaxRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"))
.setEncodingType("UTF16");
CloudNaturalLanguage.Documents.AnalyzeSyntax analyze =
languageApi.documents().analyzeSyntax(request);
AnalyzeSyntaxResponse response = analyze.execute();
return response.getTokens();
AnalyzeSyntaxRequest request = AnalyzeSyntaxRequest.newBuilder()
.setDocument(Document.newBuilder().setContent(text).setType(Type.PLAIN_TEXT).build())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line is way too long.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revised this line and adjacent lines to build document separate from request - makes all the lines a bit less lengthy.

.setEncodingType(EncodingType.UTF16).build();
AnalyzeSyntaxResponse response = languageApi.analyzeSyntax(request);
return response.getTokensList();
}
}
15 changes: 8 additions & 7 deletions language/analysis/src/test/java/com/google/cloud/language/samples/AnalyzeIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,10 @@

import static com.google.common.truth.Truth.assertThat;

import com.google.api.services.language.v1.model.Entity;
import com.google.api.services.language.v1.model.Sentiment;
import com.google.api.services.language.v1.model.Token;
import com.google.cloud.language.v1.Entity;
import com.google.cloud.language.v1.PartOfSpeech.Tag;
import com.google.cloud.language.v1.Sentiment;
import com.google.cloud.language.v1.Token;

import org.junit.Before;
import org.junit.Test;
Expand All @@ -40,7 +41,7 @@ public class AnalyzeIT {
private Analyze analyzeApp;

@Before public void setup() throws Exception {
analyzeApp = new Analyze(Analyze.getLanguageService());
analyzeApp = new Analyze(Analyze.createLanguageService());
}

@Test public void analyzeEntities_withEntities_returnsLarryPage() throws Exception {
Expand Down Expand Up @@ -85,11 +86,11 @@ public class AnalyzeIT {
analyzeApp.analyzeSyntax(
"President Obama was elected for the second term");

List<String> got = token.stream().map(e -> e.getPartOfSpeech().getTag())
List<Tag> got = token.stream().map(e -> e.getPartOfSpeech().getTag())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea - Java 8. Good to see, but we don't typically use this in samples. Note - I'm Ok w/ this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Leaving as-is.

.collect(Collectors.toList());

// Assert
assertThat(got).containsExactly("NOUN", "NOUN", "VERB",
"VERB", "ADP", "DET", "ADJ", "NOUN").inOrder();
assertThat(got).containsExactly(Tag.NOUN, Tag.NOUN, Tag.VERB,
Tag.VERB, Tag.ADP, Tag.DET, Tag.ADJ, Tag.NOUN).inOrder();
}
}
27 changes: 19 additions & 8 deletions language/analysis/src/test/java/com/google/cloud/language/samples/AnalyzeTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@

import static com.google.common.truth.Truth.assertThat;

import com.google.api.services.language.v1.model.Entity;
import com.google.cloud.language.v1.Entity;
import com.google.cloud.language.v1.Entity.Builder;
import com.google.cloud.language.v1.Entity.Type;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;

Expand Down Expand Up @@ -69,13 +71,22 @@ public class AnalyzeTest {
PrintStream out = new PrintStream(bout);
ImmutableList<Entity> entities =
ImmutableList.of(
new Entity().setName("Larry Page").setSalience(0.426f).setType("PERSON").setMetadata(
ImmutableMap.<String, String>builder()
.put("knowledge_graph_mid", "/m/0gjpq")
.put("wikipedia_url", "http://en.wikipedia.org/wiki/index.html?curid=60903")
.build()),
new Entity().setName("search engine").setSalience(0.188f).setType("CONSUMER_GOOD"),
new Entity().setName("something"));
Entity.newBuilder().setName("Larry Page")
.setSalience(0.426f)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems really specific, should you explain it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added verbose comments calling out this is mock data based on actual data.

.setType(Type.PERSON)
.putAllMetadata(
ImmutableMap.<String, String>builder()
.put("knowledge_graph_mid", "/m/0gjpq")
.put("wikipedia_url",
"http://en.wikipedia.org/wiki/index.html?curid=60903")
.build())
.build(),
Entity.newBuilder()
.setName("search engine")
.setSalience(0.188f)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also seems very specific - does it deserve a comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Unless, you mean to comment on the inner members, e.g.

Entity.newBuilder()
    .setName("search engine") // The proper name for the mock entity
    .setSalience(0.188f) // The prominence of this entity relative to others in the document

I'm not sure how much detail is appropriate here in the mocks. For example, AFAICT the salience of entities sums up to 1. This means that if you sorted the entities by salience, descending, the most prominent entities in the target text will appear first: the actual salience values just tell you how important this particular entity is within the scope of the document.

.setType(Type.CONSUMER_GOOD)
.build(),
Entity.newBuilder().setName("something").build());

// Act
Analyze.printEntities(out, entities);
Expand Down