Skip to content

Commit

Permalink
address feedback and streamline analyze
Browse files Browse the repository at this point in the history
  • Loading branch information
deyaaeldeen committed Nov 11, 2020
1 parent cdec7a2 commit e67ae4d
Show file tree
Hide file tree
Showing 30 changed files with 1,293 additions and 404 deletions.
8 changes: 6 additions & 2 deletions sdk/textanalytics/ai-text-analytics/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Release History

## 5.2.0-beta.1 (Unreleased)
## 5.1.0-beta.3 (Unreleased)

- We are now targeting the service's v3.1-preview.3 API as the default instead of v3.1-preview.2.
- We now have added support for performing multiple analyses at once with the introduction of the `beginAnalyze` API.
- We now have added support for the recognition of healthcare entities with the introduction of the `beginAnalyzeHealthcare` API.

## 5.1.0-beta.2 (2020-10-06)

Expand All @@ -17,7 +21,7 @@

## 5.0.1 (2020-08-18)

- Handles REST exceptions with code InvalidDocumentBatch in a way that maintains backward compatability.
- Handles REST exceptions with code InvalidDocumentBatch in a way that maintains backward compatibility.

## 5.0.0 (2020-07-27)

Expand Down
131 changes: 121 additions & 10 deletions sdk/textanalytics/ai-text-analytics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,24 @@

[Azure TextAnalytics](https://azure.microsoft.com/services/cognitive-services/text-analytics/) is a cloud-based service that provides advanced natural language processing over raw text, and includes six main functions:

__Note:__ This SDK targets Azure Text Analytics service API version 3.1.0-preview.2.
**Note:** This SDK targets Azure Text Analytics service API version 3.1.0-preview.2.

- Language Detection
- Sentiment Analysis
- Key Phrase Extraction
- Named Entity Recognition
- Recognition of Personally Identifiable Information
- Linked Entity Recognition
- Healthcare Analysis
- Batch Analysis

Use the client library to:

- Detect what language input text is written in.
- Determine what customers think of your brand or topic by analyzing raw text for clues about positive or negative sentiment.
- Automatically extract key phrases to quickly identify the main points.
- Identify and categorize entities in your text as people, places, organizations, date/time, quantities, percentages, currencies, and more.
- Identify and categorize entities in your text as people, places, organizations, date/time, quantities, percentages, currencies, healthcare specific, and more.
- Perform multiple of the above tasks at once.

[Source code](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/textanalytics/ai-text-analytics/) |
[Package (NPM)](https://www.npmjs.com/package/@azure/ai-text-analytics) |
Expand Down Expand Up @@ -113,7 +116,7 @@ For example, each document can be passed as a string in an array, e.g.
const documents = [
"I hated the movie. It was so slow!",
"The movie made it into my top ten favorites.",
"What a great movie!",
"What a great movie!"
];
```

Expand All @@ -123,7 +126,7 @@ or, if you wish to pass in a per-item document `id` or `language`/`countryHint`,
const textDocumentInputs = [
{ id: "1", language: "en", text: "I hated the movie. It was so slow!" },
{ id: "2", language: "en", text: "The movie made it into my top ten favorites." },
{ id: "3", language: "en", text: "What a great movie!" },
{ id: "3", language: "en", text: "What a great movie!" }
];
```

Expand Down Expand Up @@ -186,7 +189,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"I did not like the restaurant. The food was too spicy.",
"The restaurant was decorated beautifully. The atmosphere was unlike any other restaurant I've been to.",
"The food was yummy. :)",
"The food was yummy. :)"
];

async function main() {
Expand Down Expand Up @@ -221,7 +224,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"Microsoft was founded by Bill Gates and Paul Allen.",
"Redmond is a city in King County, Washington, United States, located 15 miles east of Seattle.",
"Jeff bought three dozen eggs because there was a 50% discount.",
"Jeff bought three dozen eggs because there was a 50% discount."
];

async function main() {
Expand Down Expand Up @@ -284,7 +287,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"Microsoft was founded by Bill Gates and Paul Allen.",
"Easter Island, a Chilean territory, is a remote volcanic island in Polynesia.",
"I use Azure Functions to develop my product.",
"I use Azure Functions to develop my product."
];

async function main() {
Expand All @@ -296,7 +299,13 @@ async function main() {
for (const entity of result.entities) {
console.log(entity.name, "(URL:", entity.url, ", Source:", entity.dataSource, ")");
for (const match of entity.matches) {
console.log(" Occurrence:", "\"" + match.text + "\"", "(Score:", match.confidenceScore, ")");
console.log(
" Occurrence:",
'"' + match.text + '"',
"(Score:",
match.confidenceScore,
")"
);
}
}
} else {
Expand All @@ -320,7 +329,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"Redmond is a city in King County, Washington, United States, located 15 miles east of Seattle.",
"I need to take my cat to the veterinarian.",
"I will travel to South America in the summer.",
"I will travel to South America in the summer."
];

async function main() {
Expand Down Expand Up @@ -382,6 +391,108 @@ async function main() {
main();
```

### Analyze Healthcare Entities

Healthcare analysis identifies healthcare entities. For example, given input text "Prescribed 100mg ibuprofen, taken twice daily", the service returns "100mg" categorized as Dosage, "ibuprofen" as MedicationName, and "twice daily" as Frequency.

```javascript
const { TextAnalyticsClient, AzureKeyCredential } = require("@azure/ai-text-analytics");

const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<API key>"));

const documents = [
"Prescribed 100mg ibuprofen, taken twice daily.",
"Patient does not suffer from high blood pressure."
];

async function main() {
const poller = await client.beginAnalyzeHealthcare(documents);
const results = await poller.pollUntilDone();

for await (const result of results) {
console.log(`- Document ${result.id}`);
if (!result.error) {
console.log(" Recognized Entities:");
for (const entity of result.entities) {
console.log(` - Entity ${entity.text} of type ${entity.category}`);
}
} else console.error(" Error:", result.error);
}
}

main();
```

### Analyze

Analyze enables the application of multiple analyses at once.

```javascript
const { TextAnalyticsClient, AzureKeyCredential } = require("@azure/ai-text-analytics");

const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<API key>"));

const documents = [
"Microsoft was founded by Bill Gates and Paul Allen.",
"The employee's SSN is 555-55-5555.",
"Easter Island, a Chilean territory, is a remote volcanic island in Polynesia.",
"I use Azure Functions to develop my product."
];

async function main() {
const tasks = {
entityRecognitionTasks: [{ modelVersion: "latest" }],
entityRecognitionPiiTasks: [{ modelVersion: "latest" }],
keyPhraseExtractionTasks: [{ modelVersion: "latest" }]
};
const poller = await client.beginAnalyze(documents, tasks);
const resultPages = await poller.pollUntilDone();

for await (const page of resultPages) {
const keyPhrasesResults = page.keyPhrasesExtractionResults[0];
for (const doc of keyPhrasesResults) {
console.log(`- Document ${doc.id}`);
if (!doc.error) {
console.log(" Key phrases:");
for (const phrase of doc.keyPhrases) {
console.log(` ${phrase}`);
}
} else {
console.error(" Error:", doc.error);
}
}

const entitiesResults = page.entitiesRecognitionResults[0];
for (const doc of entitiesResults) {
console.log(`- Document ${doc.id}`);
if (!doc.error) {
console.log(" Entities:");
for (const entity of doc.entities) {
console.log(` ${entity}`);
}
} else {
console.error(" Error:", doc.error);
}
}

const piiEntitiesResults = page.piiEntitiesRecognitionResults[0];
for (const doc of piiEntitiesResults) {
console.log(`- Document ${doc.id}`);
if (!doc.error) {
console.log(" Pii Entities:");
for (const entity of doc.entities) {
console.log(` ${entity}`);
}
} else {
console.error(" Error:", doc.error);
}
}
}
}

main();
```

## Troubleshooting

### Enable logs
Expand Down Expand Up @@ -421,4 +532,4 @@ If you'd like to contribute to this library, please read the [contributing guide
[register_aad_app]: https://docs.microsoft.com/azure/cognitive-services/authentication#assign-a-role-to-a-service-principal
[defaultazurecredential]: https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/identity/identity#defaultazurecredential
[data_limits]: https://docs.microsoft.com/azure/cognitive-services/text-analytics/overview#data-limits
[analyze_sentiment_opinion_mining_sample]: https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/textanalytics/ai-text-analytics/samples/typescript/src/analyzeSentimentWithOpinionMining.ts
[analyze_sentiment_opinion_mining_sample]: https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/textanalytics/ai-text-analytics/samples/typescript/src/analyzeSentimentWithOpinionMining.ts
2 changes: 1 addition & 1 deletion sdk/textanalytics/ai-text-analytics/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"sdk-type": "client",
"author": "Microsoft Corporation",
"description": "An isomorphic client library for the Azure Text Analytics service.",
"version": "5.2.0-beta.1",
"version": "5.1.0-beta.3",
"keywords": [
"node",
"azure",
Expand Down
76 changes: 13 additions & 63 deletions sdk/textanalytics/ai-text-analytics/review/ai-text-analytics.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,41 +14,18 @@ import { PollOperationState } from '@azure/core-lro';
import { TokenCredential } from '@azure/core-auth';

// @public
export interface AnalyzeEntitiesResult extends RecognizeCategorizedEntitiesSuccessResult {
// (undocumented)
type: "Entities";
}

// @public
export interface AnalyzeErrorResult extends TextAnalyticsErrorResult {
// (undocumented)
type: "Error";
}

// @public
export interface AnalyzeJobOptions extends TextAnalyticsOperationOptions {
}

// @public (undocumented)
export interface AnalyzeKeyPhrasesResult extends ExtractKeyPhrasesSuccessResult {
// (undocumented)
type: "KeyPhrases";
}

// @public (undocumented)
export interface AnalyzePiiEntitiesResult extends RecognizePiiEntitiesSuccessResult {
// (undocumented)
type: "PiiEntities";
export interface AnalyzeJobOptions extends OperationOptions {
includeStatistics?: boolean;
}

// @public
export type AnalyzePollerLike = PollerLike<BeginAnalyzeOperationState, PaginatedAnalyzeResults>;

// @public
export type AnalyzeResult = AnalyzeEntitiesResult | AnalyzePiiEntitiesResult | AnalyzeKeyPhrasesResult | AnalyzeErrorResult;

// @public
export interface AnalyzeResultsArray extends Array<AnalyzeResult> {
export interface AnalyzeResult {
entitiesRecognitionResults?: RecognizeCategorizedEntitiesResultArray[];
keyPhrasesExtractionResults?: ExtractKeyPhrasesResultArray[];
piiEntitiesRecognitionResults?: RecognizePiiEntitiesResultArray[];
}

// @public
Expand Down Expand Up @@ -98,9 +75,7 @@ export type BeginAnalyzeHealthcareOperationState = PollOperationState<PaginatedH

// @public
export interface BeginAnalyzeHealthcareOptions {
// (undocumented)
health?: HealthcareJobOptions;
// (undocumented)
polling?: PollingOptions;
}

Expand All @@ -109,9 +84,7 @@ export type BeginAnalyzeOperationState = PollOperationState<PaginatedAnalyzeResu

// @public
export interface BeginAnalyzeOptions {
// (undocumented)
analyze?: AnalyzeJobOptions;
// (undocumented)
polling?: PollingOptions;
}

Expand Down Expand Up @@ -158,16 +131,10 @@ export interface DetectLanguageSuccessResult extends TextAnalyticsSuccessResult
// @public
export type DocumentSentimentLabel = "positive" | "neutral" | "negative" | "mixed";

// @public (undocumented)
// @public
export type EntitiesTask = {
parameters?: EntitiesTaskParameters;
};

// @public (undocumented)
export interface EntitiesTaskParameters {
// (undocumented)
modelVersion?: string;
}
};

// @public
export interface Entity {
Expand Down Expand Up @@ -250,25 +217,15 @@ export type HealthPollerLike = PollerLike<BeginAnalyzeHealthcareOperationState,
// @public
export type InnerErrorCodeValue = "InvalidParameterValue" | "InvalidRequestBodyFormat" | "EmptyRequest" | "MissingInputRecords" | "InvalidDocument" | "ModelVersionIncorrect" | "InvalidDocumentBatch" | "UnsupportedLanguageCode" | "InvalidCountryHint" | string;

// @public (undocumented)
// @public
export interface JobManifestTasks {
// (undocumented)
entityRecognitionPiiTasks?: PiiTask[];
// (undocumented)
entityRecognitionTasks?: EntitiesTask[];
// (undocumented)
keyPhraseExtractionTasks?: KeyPhrasesTask[];
}

// @public (undocumented)
// @public
export interface KeyPhrasesTask {
// (undocumented)
parameters?: KeyPhrasesTaskParameters;
}

// @public (undocumented)
export interface KeyPhrasesTaskParameters {
// (undocumented)
modelVersion?: string;
}

Expand Down Expand Up @@ -301,7 +258,7 @@ export interface OpinionSentiment extends SentenceOpinion {
}

// @public
export type PagedAsyncIterableAnalyzeResults = PagedAsyncIterableIterator<AnalyzeResult, AnalyzeResultsArray>;
export type PagedAsyncIterableAnalyzeResults = PagedAsyncIterableIterator<AnalyzeResult, AnalyzeResult>;

// @public
export type PagedAsyncIterableHealthEntities = PagedAsyncIterableIterator<HealthcareResult, HealthcareEntitiesArray>;
Expand All @@ -326,18 +283,11 @@ export enum PiiEntityDomainType {
PROTECTED_HEALTH_INFORMATION = "PHI"
}

// @public (undocumented)
// @public
export type PiiTask = {
parameters?: PiiTaskParameters;
};

// @public (undocumented)
export interface PiiTaskParameters {
// (undocumented)
domain?: PiiTaskParametersDomain;
// (undocumented)
modelVersion?: string;
}
};

// @public
export type PiiTaskParametersDomain = "phi" | "none" | string;
Expand Down
Loading

0 comments on commit e67ae4d

Please sign in to comment.