Skip to content

Commit

Permalink
[Text Analytics] Add piiCategories to recognizePiiEntities (Azure#14043)
Browse files Browse the repository at this point in the history
* [Text Analytics] Add piiCategories to recognizePiiEntities

* address feedback

* update sample

* update param name in changelog

* add phi back to samples

* fix syntax
  • Loading branch information
deyaaeldeen authored Mar 2, 2021
1 parent 8bd2ca0 commit a7b7403
Show file tree
Hide file tree
Showing 11 changed files with 375 additions and 26 deletions.
1 change: 1 addition & 0 deletions sdk/textanalytics/ai-text-analytics/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- `beginAnalyzeHealthcareEntities` returns `entityRelations` per document, a list of relations between healthcare entities.
- `beginAnalyzeHealthcareEntities` entities now include `assertions` instead of `isNegated` which gives more context about the respective entity.
- [Breaking] `beginAnalyzeHealthcareEntities` no longer returns `relatedEntities`.
- `recognizePiiEntities` takes a new option, `categoriesFilter`, that specifies a list of Pii categories to return.

## 5.1.0-beta.4 (2021-02-10)

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -333,13 +333,13 @@ export type PagedAsyncIterableAnalyzeBatchActionsResult = PagedAsyncIterableIter
// @public
export type PagedAsyncIterableAnalyzeHealthcareEntitiesResult = PagedAsyncIterableIterator<AnalyzeHealthcareEntitiesResult, AnalyzeHealthcareEntitiesResultArray>;

// @public
export type PiiCategory = string;

// @public
export interface PiiEntity extends Entity {
}

// @public
export type PiiEntityCategory = string;

// @public
export enum PiiEntityDomainType {
PROTECTED_HEALTH_INFORMATION = "PHI"
Expand Down Expand Up @@ -428,7 +428,7 @@ export type RecognizePiiEntitiesAction = {
domain?: PiiEntityDomainType;
modelVersion?: string;
stringIndexType?: StringIndexType;
piiCategories?: PiiCategory[];
categoriesFilter?: PiiEntityCategory[];
};

// @public
Expand All @@ -447,6 +447,7 @@ export type RecognizePiiEntitiesErrorResult = TextAnalyticsErrorResult;

// @public
export interface RecognizePiiEntitiesOptions extends TextAnalyticsOperationOptions {
categoriesFilter?: PiiEntityCategory[];
domainFilter?: PiiEntityDomainType;
stringIndexType?: StringIndexType;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,47 @@ async function main() {

const client = new TextAnalyticsClient(endpoint, new AzureKeyCredential(apiKey));

const [result] = await client.recognizePiiEntities(["My phone number is 555-5555"]);
const textOnePii = "My phone number is 555-5555";
const textNoPHI =
"FIFA is a non-profit organization which describes itself as an international governing body of association football.";
const textMultiplePIIs = "Patient name is Joe and SSN is 859-98-0987";

const [result] = await client.recognizePiiEntities([textOnePii]);

if (!result.error) {
console.log(`The redacted text: ${result.redactedText}`);
console.log(
`The redacted text is "${result.redactedText}" and found the following PII entities`
);
for (const entity of result.entities) {
console.log(`Found PII entity ${entity.text} of type ${entity.category}`);
console.log(`\t- "${entity.text}" of type ${entity.category}`);
}
}

const textNoPHI =
"FIFA is a non-profit organization which describes itself as an international governing body of association football.";
console.log(`There are no PHI entities in this text: ${textNoPHI}`);
const [resultWithPHI] = await client.recognizePiiEntities(
[{ id: "0", text: textNoPHI, language: "en" }],
{ domainFilter: PiiEntityDomainType.PROTECTED_HEALTH_INFORMATION }
);
if (!resultWithPHI.error) {
console.log(`Also there is nothing to redact: ${resultWithPHI.redactedText}`);
assert(resultWithPHI.entities.length === 0);
assert(resultWithPHI.entities.length === 0, "did not expect any entities but got some");
}

console.log(`But there are other entities in that text`);
console.log(`But there are other PII entities in that text:`);
const [resultWithoutPHI] = await client.recognizePiiEntities([textNoPHI]);
if (!resultWithoutPHI.error) {
for (const entity of resultWithoutPHI.entities) {
console.log(`Found PII entity ${entity.text} of type ${entity.category}`);
console.log(`\t- "${entity.text}" of type ${entity.category}`);
}
}
const [resultWithSSNPII] = await client.recognizePiiEntities([textMultiplePIIs], "en", {
categoriesFilter: ["USSocialSecurityNumber"]
});
if (!resultWithSSNPII.error) {
console.log("The service was asked to return just SSN entities:");
for (const entity of resultWithSSNPII.entities) {
console.log(`\t- "${entity.text}"`);
assert(entity.category === "USSocialSecurityNumber");
}
}
}
Expand Down
Loading

0 comments on commit a7b7403

Please sign in to comment.