Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Text Analytics] Adding analyze and healthcare APIs #11375

Merged
merged 13 commits into from
Nov 23, 2020
7 changes: 5 additions & 2 deletions sdk/textanalytics/ai-text-analytics/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# Release History

## 5.1.0-beta.3 (Unreleased)
## 5.1.0-beta.3 (2020-11-20)

- We are now targeting the service's v3.1-preview.3 API as the default instead of v3.1-preview.2.
- We now have added support for performing multiple analyses at once with the introduction of the `beginAnalyze` API.
- We now have added support for the recognition of healthcare entities with the introduction of the `beginAnalyzeHealthcare` API.

## 5.1.0-beta.2 (2020-10-06)

Expand All @@ -18,7 +21,7 @@

## 5.0.1 (2020-08-18)

- Handles REST exceptions with code InvalidDocumentBatch in a way that maintains backward compatability.
- Handles REST exceptions with code InvalidDocumentBatch in a way that maintains backward compatibility.

## 5.0.0 (2020-07-27)

Expand Down
136 changes: 126 additions & 10 deletions sdk/textanalytics/ai-text-analytics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,24 @@

[Azure TextAnalytics](https://azure.microsoft.com/services/cognitive-services/text-analytics/) is a cloud-based service that provides advanced natural language processing over raw text, and includes six main functions:

__Note:__ This SDK targets Azure Text Analytics service API version 3.1.0-preview.2.
**Note:** This SDK targets Azure Text Analytics service API version 3.1.0-preview.2.

- Language Detection
- Sentiment Analysis
- Key Phrase Extraction
- Named Entity Recognition
- Recognition of Personally Identifiable Information
- Linked Entity Recognition
- Healthcare Analysis
- Batch Processing

Use the client library to:

- Detect what language input text is written in.
- Determine what customers think of your brand or topic by analyzing raw text for clues about positive or negative sentiment.
- Automatically extract key phrases to quickly identify the main points.
- Identify and categorize entities in your text as people, places, organizations, date/time, quantities, percentages, currencies, and more.
- Identify and categorize entities in your text as people, places, organizations, date/time, quantities, percentages, currencies, healthcare specific, and more.
- Perform multiple of the above tasks at once.

[Source code](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/textanalytics/ai-text-analytics/) |
[Package (NPM)](https://www.npmjs.com/package/@azure/ai-text-analytics) |
Expand Down Expand Up @@ -113,7 +116,7 @@ For example, each document can be passed as a string in an array, e.g.
const documents = [
"I hated the movie. It was so slow!",
"The movie made it into my top ten favorites.",
"What a great movie!",
"What a great movie!"
];
```

Expand All @@ -123,7 +126,7 @@ or, if you wish to pass in a per-item document `id` or `language`/`countryHint`,
const textDocumentInputs = [
{ id: "1", language: "en", text: "I hated the movie. It was so slow!" },
{ id: "2", language: "en", text: "The movie made it into my top ten favorites." },
{ id: "3", language: "en", text: "What a great movie!" },
{ id: "3", language: "en", text: "What a great movie!" }
];
```

Expand Down Expand Up @@ -186,7 +189,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"I did not like the restaurant. The food was too spicy.",
"The restaurant was decorated beautifully. The atmosphere was unlike any other restaurant I've been to.",
"The food was yummy. :)",
"The food was yummy. :)"
];

async function main() {
Expand Down Expand Up @@ -221,7 +224,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"Microsoft was founded by Bill Gates and Paul Allen.",
"Redmond is a city in King County, Washington, United States, located 15 miles east of Seattle.",
"Jeff bought three dozen eggs because there was a 50% discount.",
"Jeff bought three dozen eggs because there was a 50% discount."
];

async function main() {
Expand Down Expand Up @@ -284,7 +287,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"Microsoft was founded by Bill Gates and Paul Allen.",
"Easter Island, a Chilean territory, is a remote volcanic island in Polynesia.",
"I use Azure Functions to develop my product.",
"I use Azure Functions to develop my product."
];

async function main() {
Expand All @@ -296,7 +299,13 @@ async function main() {
for (const entity of result.entities) {
console.log(entity.name, "(URL:", entity.url, ", Source:", entity.dataSource, ")");
for (const match of entity.matches) {
console.log(" Occurrence:", "\"" + match.text + "\"", "(Score:", match.confidenceScore, ")");
console.log(
" Occurrence:",
'"' + match.text + '"',
"(Score:",
match.confidenceScore,
")"
);
}
}
} else {
Expand All @@ -320,7 +329,7 @@ const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<AP
const documents = [
"Redmond is a city in King County, Washington, United States, located 15 miles east of Seattle.",
"I need to take my cat to the veterinarian.",
"I will travel to South America in the summer.",
"I will travel to South America in the summer."
];

async function main() {
Expand Down Expand Up @@ -382,6 +391,113 @@ async function main() {
main();
```

### Analyze Healthcare Entities

Healthcare analysis identifies healthcare entities. For example, given input text "Prescribed 100mg ibuprofen, taken twice daily", the service returns "100mg" categorized as Dosage, "ibuprofen" as MedicationName, and "twice daily" as Frequency.

```javascript
const { TextAnalyticsClient, AzureKeyCredential } = require("@azure/ai-text-analytics");

const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<API key>"));

const documents = [
"Prescribed 100mg ibuprofen, taken twice daily.",
"Patient does not suffer from high blood pressure."
];

async function main() {
const poller = await client.beginAnalyzeHealthcare(documents);
const results = await poller.pollUntilDone();

for await (const result of results) {
console.log(`- Document ${result.id}`);
if (!result.error) {
console.log("\tRecognized Entities:");
for (const entity of result.entities) {
console.log(`\t- Entity ${entity.text} of type ${entity.category}`);
}
} else console.error("\tError:", result.error);
}
}

main();
```

### Analyze

Analyze enables the application of multiple analyses at once.

```javascript
const { TextAnalyticsClient, AzureKeyCredential } = require("@azure/ai-text-analytics");

const client = new TextAnalyticsClient("<endpoint>", new AzureKeyCredential("<API key>"));

const documents = [
"Microsoft was founded by Bill Gates and Paul Allen.",
"The employee's SSN is 555-55-5555.",
"Easter Island, a Chilean territory, is a remote volcanic island in Polynesia.",
"I use Azure Functions to develop my product."
];

async function main() {
const tasks = {
entityRecognitionTasks: [{ modelVersion: "latest" }],
entityRecognitionPiiTasks: [{ modelVersion: "latest" }],
keyPhraseExtractionTasks: [{ modelVersion: "latest" }]
};
const poller = await client.beginAnalyze(documents, tasks);
const resultPages = await poller.pollUntilDone();

for await (const page of resultPages) {
const keyPhrasesResults = page.keyPhrasesExtractionResults[0];
for (const doc of keyPhrasesResults) {
console.log(`- Document ${doc.id}`);
if (!doc.error) {
console.log("\tKey phrases:");
for (const phrase of doc.keyPhrases) {
console.log(`\t- ${phrase}`);
}
} else {
console.error("\tError:", doc.error);
}
}

const entitiesResults = page.entitiesRecognitionResults[0];
for (const doc of entitiesResults) {
console.log(`- Document ${doc.id}`);
if (!doc.error) {
console.log("\tEntities:");
for (const entity of doc.entities) {
console.log(`\t- Entity ${entity.text} of type ${entity.category}`);
}
} else {
console.error("\tError:", doc.error);
}
}

const piiEntitiesResults = page.piiEntitiesRecognitionResults[0];
for (const doc of piiEntitiesResults) {
console.log(`- Document ${doc.id}`);
if (!doc.error) {
console.log("\tPii Entities:");
for (const entity of doc.entities) {
console.log(`\t- Entity ${entity.text} of type ${entity.category}`);
}
} else {
console.error("\tError:", doc.error);
}
}
}
}

main();
```

## Known Issues

- Currently, the `beginAnalyze` API accepts `includeStatistics` in its options bag, a feature that was not yet supported by the service at the time of the current release. This feature is expected to be supported soon after the release.
- `beginAnalyzeHealthcare` is still in gated preview and can not be used with AAD credentials.

## Troubleshooting

### Enable logs
Expand Down Expand Up @@ -421,4 +537,4 @@ If you'd like to contribute to this library, please read the [contributing guide
[register_aad_app]: https://docs.microsoft.com/azure/cognitive-services/authentication#assign-a-role-to-a-service-principal
[defaultazurecredential]: https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/identity/identity#defaultazurecredential
[data_limits]: https://docs.microsoft.com/azure/cognitive-services/text-analytics/overview#data-limits
[analyze_sentiment_opinion_mining_sample]: https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/textanalytics/ai-text-analytics/samples/typescript/src/analyzeSentimentWithOpinionMining.ts
[analyze_sentiment_opinion_mining_sample]: https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/textanalytics/ai-text-analytics/samples/typescript/src/analyzeSentimentWithOpinionMining.ts
4 changes: 2 additions & 2 deletions sdk/textanalytics/ai-text-analytics/karma.conf.js
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ module.exports = function(config) {
// how many browser should be started simultaneous
concurrency: 1,

browserNoActivityTimeout: 600000,
browserNoActivityTimeout: 60000000,
browserDisconnectTimeout: 10000,
browserDisconnectTolerance: 3,
browserConsoleLogOptions: {
Expand All @@ -143,7 +143,7 @@ module.exports = function(config) {
mocha: {
// change Karma's debug.html to the mocha web reporter
reporter: "html",
timeout: "600000"
timeout: 0
}
}
});
Expand Down
5 changes: 5 additions & 0 deletions sdk/textanalytics/ai-text-analytics/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@
"//smokeTestConfiguration": {
"skipFolder": true
},
"browser": {
"./dist-esm/src/utils/url.js": "./dist-esm/src/utils/url.browser.js"
},
"scripts": {
"audit": "node ../../../common/scripts/rush-audit.js && rimraf node_modules package-lock.json && npm i --package-lock-only 2>&1 && npm audit",
"build:browser": "tsc -p . && cross-env ONLY_BROWSER=true rollup -c 2>&1",
Expand Down Expand Up @@ -78,6 +81,8 @@
"dependencies": {
"@azure/core-auth": "^1.1.3",
"@azure/core-http": "^1.2.0",
"@azure/core-lro": "^1.0.2",
"@azure/core-paging": "^1.1.1",
"@azure/core-tracing": "1.0.0-preview.9",
"@azure/logger": "^1.0.0",
"@opentelemetry/api": "^0.10.2",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading