Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Text Analytics] Adding analyze and healthcare APIs #11375

Merged
merged 13 commits into from
Nov 23, 2020
Merged

[Text Analytics] Adding analyze and healthcare APIs #11375

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
cdec7a2
adding analyze and health APIs
deyaaeldeen Nov 6, 2020
cfafeb2
address feedback and streamline analyze
deyaaeldeen Nov 13, 2020
2c2e35f
Adding a know issues section to README
deyaaeldeen Nov 18, 2020
4a12db1
improve samples
deyaaeldeen Nov 19, 2020
61e8b83
adding recordings
deyaaeldeen Nov 19, 2020
8e573f3
fix link errors
deyaaeldeen Nov 19, 2020
8fdecef
fix lint issues
deyaaeldeen Nov 19, 2020
2a5f012
no karma timeout
deyaaeldeen Nov 19, 2020
85bf1a0
fix release date in changelog
deyaaeldeen Nov 19, 2020
b6fbcf7
increase karma timeout
deyaaeldeen Nov 19, 2020
6315104
Recorded browser #analyze tests
witemple-msft Nov 20, 2020
a8dc5e0
Recorded even more browser tests that were missing
witemple-msft Nov 20, 2020
2abcf62
merge master
deyaaeldeen Nov 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions sdk/textanalytics/ai-text-analytics/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Release History

## 5.1.0-beta.3 (Unreleased)

## 5.2.0-beta.1 (Unreleased)
deyaaeldeen marked this conversation as resolved.
Show resolved Hide resolved

## 5.1.0-beta.2 (2020-10-06)

Expand Down
4 changes: 3 additions & 1 deletion sdk/textanalytics/ai-text-analytics/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"sdk-type": "client",
"author": "Microsoft Corporation",
"description": "An isomorphic client library for the Azure Text Analytics service.",
"version": "5.1.0-beta.3",
"version": "5.2.0-beta.1",
"keywords": [
"node",
"azure",
Expand Down Expand Up @@ -78,6 +78,8 @@
"dependencies": {
"@azure/core-auth": "^1.1.3",
"@azure/core-http": "^1.2.0",
"@azure/core-lro": "^1.0.2",
"@azure/core-paging": "^1.1.1",
"@azure/core-tracing": "1.0.0-preview.9",
"@azure/logger": "^1.0.0",
"@opentelemetry/api": "^0.10.2",
Expand Down
188 changes: 185 additions & 3 deletions sdk/textanalytics/ai-text-analytics/review/ai-text-analytics.api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,50 @@
import { AzureKeyCredential } from '@azure/core-auth';
import { KeyCredential } from '@azure/core-auth';
import { OperationOptions } from '@azure/core-http';
import { PagedAsyncIterableIterator } from '@azure/core-paging';
import { PipelineOptions } from '@azure/core-http';
import { PollerLike } from '@azure/core-lro';
import { PollOperationState } from '@azure/core-lro';
import { TokenCredential } from '@azure/core-auth';

// @public
export interface AnalyzeEntitiesResult extends RecognizeCategorizedEntitiesSuccessResult {
// (undocumented)
type: "Entities";
}

// @public
export interface AnalyzeErrorResult extends TextAnalyticsErrorResult {
// (undocumented)
type: "Error";
}

// @public
export interface AnalyzeJobOptions extends TextAnalyticsOperationOptions {
}

// @public (undocumented)
deyaaeldeen marked this conversation as resolved.
Show resolved Hide resolved
export interface AnalyzeKeyPhrasesResult extends ExtractKeyPhrasesSuccessResult {
// (undocumented)
type: "KeyPhrases";
}

// @public (undocumented)
export interface AnalyzePiiEntitiesResult extends RecognizePiiEntitiesSuccessResult {
// (undocumented)
type: "PiiEntities";
}

// @public
export type AnalyzePollerLike = PollerLike<BeginAnalyzeOperationState, PaginatedAnalyzeResults>;

// @public
export type AnalyzeResult = AnalyzeEntitiesResult | AnalyzePiiEntitiesResult | AnalyzeKeyPhrasesResult | AnalyzeErrorResult;

// @public
export interface AnalyzeResultsArray extends Array<AnalyzeResult> {
}

// @public
export type AnalyzeSentimentErrorResult = TextAnalyticsErrorResult;

Expand Down Expand Up @@ -52,6 +93,28 @@ export interface AspectSentiment {

export { AzureKeyCredential }

// @public
export type BeginAnalyzeHealthcareOperationState = PollOperationState<PaginatedHealthcareEntities>;

// @public
export interface BeginAnalyzeHealthcareOptions {
// (undocumented)
deyaaeldeen marked this conversation as resolved.
Show resolved Hide resolved
health?: HealthcareJobOptions;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand BeginAnalyzeHealthcareOptions and BeginAnalyzeOptions having custom subkeys that are just copies of TextAnalyticsOperationOptions with no customization.

I would expect this to instead look like

export interface BeginAnalyzeHealthcareOptions  extends TextAnalyticsOperationOptions, PollingOptions { }

or possibly

export type BeginAnalyzeHealthcareOptions = TextAnalyticsOperationOptions &  PollingOptions

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are discussing two things here:

  • analyze key is typed as AnalyzeJobOptions which is just a synonym to TextAnalyticsOperationOptions, so why not type it as TextAnalyticsOperationOptions instead? I think having a distinct relevant name makes a better user experience with intellisense and it makes it easier to extend this type in the future.
  • Why not flatten BeginAnalyzeOptions? I think keeping the options type as is makes it clear to the user which options are used to configure the analyze job itself and which are used to configure the poller.

// (undocumented)
polling?: PollingOptions;
}

// @public
export type BeginAnalyzeOperationState = PollOperationState<PaginatedAnalyzeResults>;

// @public
export interface BeginAnalyzeOptions {
// (undocumented)
analyze?: AnalyzeJobOptions;
// (undocumented)
polling?: PollingOptions;
}

// @public
export interface CategorizedEntity extends Entity {
}
Expand Down Expand Up @@ -95,6 +158,17 @@ export interface DetectLanguageSuccessResult extends TextAnalyticsSuccessResult
// @public
export type DocumentSentimentLabel = "positive" | "neutral" | "negative" | "mixed";

// @public (undocumented)
export type EntitiesTask = {
parameters?: EntitiesTaskParameters;
deyaaeldeen marked this conversation as resolved.
Show resolved Hide resolved
};

// @public (undocumented)
export interface EntitiesTaskParameters {
// (undocumented)
modelVersion?: string;
}

// @public
export interface Entity {
category: string;
Expand All @@ -108,7 +182,7 @@ export interface Entity {
export type ErrorCode = ErrorCodeValue | InnerErrorCodeValue;

// @public
export type ErrorCodeValue = "InvalidRequest" | "InvalidArgument" | "InternalServerError" | "ServiceUnavailable";
export type ErrorCodeValue = "InvalidRequest" | "InvalidArgument" | "InternalServerError" | "ServiceUnavailable" | "NotFound";

// @public
export type ExtractKeyPhrasesErrorResult = TextAnalyticsErrorResult;
Expand All @@ -131,7 +205,72 @@ export interface ExtractKeyPhrasesSuccessResult extends TextAnalyticsSuccessResu
}

// @public
export type InnerErrorCodeValue = "InvalidParameterValue" | "InvalidRequestBodyFormat" | "EmptyRequest" | "MissingInputRecords" | "InvalidDocument" | "ModelVersionIncorrect" | "InvalidDocumentBatch" | "UnsupportedLanguageCode" | "InvalidCountryHint";
export interface HealthcareEntitiesArray extends Array<HealthcareResult> {
}

// @public (undocumented)
export type HealthcareEntity = Entity & {
isNegated: boolean;
links?: HealthcareEntityLink[];
};

// @public (undocumented)
export interface HealthcareEntityLink {
dataSource: string;
id: string;
}

// @public
export type HealthcareErrorResult = TextAnalyticsErrorResult;

// @public
export interface HealthcareJobOptions extends TextAnalyticsOperationOptions {
}

// @public (undocumented)
export interface HealthcareRelation {
bidirectional: boolean;
relationType: string;
source: string;
target: string;
}

// @public
export type HealthcareResult = HealthcareSuccessResult | HealthcareErrorResult;

// @public
export interface HealthcareSuccessResult extends TextAnalyticsSuccessResult {
entities: HealthcareEntity[];
relations: HealthcareRelation[];
}

// @public
export type HealthPollerLike = PollerLike<BeginAnalyzeHealthcareOperationState, PaginatedHealthcareEntities>;

// @public
export type InnerErrorCodeValue = "InvalidParameterValue" | "InvalidRequestBodyFormat" | "EmptyRequest" | "MissingInputRecords" | "InvalidDocument" | "ModelVersionIncorrect" | "InvalidDocumentBatch" | "UnsupportedLanguageCode" | "InvalidCountryHint" | string;
deyaaeldeen marked this conversation as resolved.
Show resolved Hide resolved

// @public (undocumented)
export interface JobManifestTasks {
// (undocumented)
entityRecognitionPiiTasks?: PiiTask[];
// (undocumented)
entityRecognitionTasks?: EntitiesTask[];
// (undocumented)
keyPhraseExtractionTasks?: KeyPhrasesTask[];
}

// @public (undocumented)
export interface KeyPhrasesTask {
// (undocumented)
parameters?: KeyPhrasesTaskParameters;
}

// @public (undocumented)
export interface KeyPhrasesTaskParameters {
// (undocumented)
modelVersion?: string;
}

// @public
export interface LinkedEntity {
Expand Down Expand Up @@ -161,6 +300,23 @@ export interface MinedOpinion {
export interface OpinionSentiment extends SentenceOpinion {
}

// @public
export type PagedAsyncIterableAnalyzeResults = PagedAsyncIterableIterator<AnalyzeResult, AnalyzeResultsArray>;

// @public
export type PagedAsyncIterableHealthEntities = PagedAsyncIterableIterator<HealthcareResult, HealthcareEntitiesArray>;

// @public
export interface PaginatedAnalyzeResults extends PagedAsyncIterableAnalyzeResults {
statistics?: TextDocumentBatchStatistics;
}

// @public
export interface PaginatedHealthcareEntities extends PagedAsyncIterableHealthEntities {
modelVersion: string;
statistics?: TextDocumentBatchStatistics;
}

// @public
export interface PiiEntity extends Entity {
}
Expand All @@ -170,6 +326,28 @@ export enum PiiEntityDomainType {
PROTECTED_HEALTH_INFORMATION = "PHI"
}

// @public (undocumented)
export type PiiTask = {
parameters?: PiiTaskParameters;
};

// @public (undocumented)
export interface PiiTaskParameters {
// (undocumented)
domain?: PiiTaskParametersDomain;
// (undocumented)
modelVersion?: string;
}

// @public
export type PiiTaskParametersDomain = "phi" | "none" | string;

// @public
export interface PollingOptions {
resumeFrom?: string;
updateIntervalInMs?: number;
}

// @public
export type RecognizeCategorizedEntitiesErrorResult = TextAnalyticsErrorResult;

Expand Down Expand Up @@ -269,6 +447,10 @@ export class TextAnalyticsClient {
constructor(endpointUrl: string, credential: TokenCredential | KeyCredential, options?: TextAnalyticsClientOptions);
analyzeSentiment(documents: string[], language?: string, options?: AnalyzeSentimentOptions): Promise<AnalyzeSentimentResultArray>;
analyzeSentiment(documents: TextDocumentInput[], options?: AnalyzeSentimentOptions): Promise<AnalyzeSentimentResultArray>;
beginAnalyze(documents: string[], tasks: JobManifestTasks, language?: string, options?: BeginAnalyzeOptions): Promise<AnalyzePollerLike>;
beginAnalyze(documents: TextDocumentInput[], tasks: JobManifestTasks, options?: BeginAnalyzeOptions): Promise<AnalyzePollerLike>;
beginAnalyzeHealthcare(documents: string[], language?: string, options?: BeginAnalyzeHealthcareOptions): Promise<HealthPollerLike>;
beginAnalyzeHealthcare(documents: TextDocumentInput[], options?: BeginAnalyzeHealthcareOptions): Promise<HealthPollerLike>;
defaultCountryHint: string;
defaultLanguage: string;
detectLanguage(documents: string[], countryHint?: string, options?: DetectLanguageOptions): Promise<DetectLanguageResultArray>;
Expand Down Expand Up @@ -351,7 +533,7 @@ export interface TextDocumentStatistics {
export type TokenSentimentValue = "positive" | "mixed" | "negative";

// @public
export type WarningCode = "LongWordsInDocument" | "DocumentTruncated";
export type WarningCode = "LongWordsInDocument" | "DocumentTruncated" | string;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this for open-ended extensibility? Should we instead make this string and have KnownWarningCodes or something?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @joheredi, is this related to a recent change in the code generator? This is the swagger definition: https://github.com/Azure/azure-rest-api-specs/blob/28d37f8847253787a56d39814e7ccd5194aeeabe/specification/cognitiveservices/data-plane/TextAnalytics/preview/v3.1-preview.3/TextAnalytics.json#L815

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is the way the generator is doing "extensible" enums. I don't think we have reach a formal agreement on how to generate these.

@xirzec your proposal is to instead of generating this:

export type WarningCode = "LongWordsInDocument" | "DocumentTruncated" | string;
export interface TextAnalyticsWarning {
    code: WarningCode;
    message: string;
}

Do this?

export type KnownWarningCode = "LongWordsInDocument" | "DocumentTruncated";
export interface TextAnalyticsWarning {
    code: string;
    message: string;
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue with this way of modeling extensible enums is that it is exactly the same as type WarningCode = string;. The information about known values is only contained in the source code text and is fundamentally erased from the type when it comes down to set theory. Even if we added a KnownWarningCodes enum or type, the problem then is how to associate that type with the parameter in a discoverable way. It's not going to show up when you invoke completion in an IDE, for example.



// (No @packageDocumentation comment for this package)
Expand Down
69 changes: 69 additions & 0 deletions sdk/textanalytics/ai-text-analytics/src/analyzeResult.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

import { PagedAsyncIterableIterator } from "@azure/core-paging";
import { ExtractKeyPhrasesSuccessResult } from "./extractKeyPhrasesResult";
import { TextDocumentBatchStatistics } from "./generated/models";
import { RecognizeCategorizedEntitiesSuccessResult } from "./recognizeCategorizedEntitiesResult";
import { RecognizePiiEntitiesSuccessResult } from "./recognizePiiEntitiesResult";
import { TextAnalyticsErrorResult } from "./textAnalyticsResult";

/**
* The results of a successful analyze entities job on a single document.
*/
export interface AnalyzeEntitiesResult extends RecognizeCategorizedEntitiesSuccessResult {
type: "Entities";
}

export interface AnalyzePiiEntitiesResult extends RecognizePiiEntitiesSuccessResult {
type: "PiiEntities";
}

export interface AnalyzeKeyPhrasesResult extends ExtractKeyPhrasesSuccessResult {
type: "KeyPhrases";
}

/**
* An error result from the analyze operation on a single document.
*/
export interface AnalyzeErrorResult extends TextAnalyticsErrorResult {
type: "Error";
}

/**
* The results of a successful analyze job for a single document.
*/
export type AnalyzeResult =
| AnalyzeEntitiesResult
| AnalyzePiiEntitiesResult
| AnalyzeKeyPhrasesResult
| AnalyzeErrorResult;

/**
* Array of {@link AnalyzeResult}
*/
export interface AnalyzeResultsArray extends Array<AnalyzeResult> {}

/**
* The results of an analyze job represented as a paginated iterator that can
* either iterate over the results on a document-by-document basis or, by
* byPage(), can iterate over pages of documents.
*/
export type PagedAsyncIterableAnalyzeResults = PagedAsyncIterableIterator<
AnalyzeResult,
AnalyzeResultsArray
>;

/**
* The results of an analyze job represented as a paginated iterator that can
* either iterate over the results on a document-by-document basis or, by
* byPage(), can iterate over pages of documents.
*/
export interface PaginatedAnalyzeResults extends PagedAsyncIterableAnalyzeResults {
/**
* Statistics about the input document batch and how it was processed
* by the service. This property will have a value when includeStatistics is set to true
* in the client call.
*/
statistics?: TextDocumentBatchStatistics;
}
2 changes: 1 addition & 1 deletion sdk/textanalytics/ai-text-analytics/src/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

export const SDK_VERSION: string = "5.1.0-beta.3";
export const SDK_VERSION: string = "5.2.0-beta.1";
Loading