Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chat Copilot Security Improvements #126

Merged
merged 41 commits into from
Aug 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
183792b
Begin porting security fixes - does not build
gitri-ms Aug 1, 2023
b61abe1
Security updates to backend webapi
gitri-ms Aug 1, 2023
8232749
Update ChatParticipantsController
gitri-ms Aug 1, 2023
d81c067
Update routes on FE
gitri-ms Aug 2, 2023
0ee5c96
Merge branch 'main' into security
gitri-ms Aug 2, 2023
65de3b9
Code cleanup
gitri-ms Aug 2, 2023
59aaeb0
Fix claim names, remove api key references
gitri-ms Aug 2, 2023
369f60c
Fix policy and unblock patch requests
gitri-ms Aug 2, 2023
045784b
Merge branch 'main' into security
gitri-ms Aug 2, 2023
c2535f9
Merge branch 'main' into security
gitri-ms Aug 3, 2023
2079c04
Update deploy scripts to use AAD auth by default
gitri-ms Aug 3, 2023
cf65b19
Merge branch 'main' into security
gitri-ms Aug 3, 2023
cb5bb18
Update deployment pipeline
gitri-ms Aug 3, 2023
a637834
Update deploy scripts to set API scopes in env, fix bug in policy
gitri-ms Aug 4, 2023
7e9f5e5
Merge branch 'main' into security
gitri-ms Aug 4, 2023
43bf249
Fix bicep lint error, include tenant id in user id on server side
gitri-ms Aug 4, 2023
64f7b15
change AAD auth to opt-in in webapp
dehoward Aug 7, 2023
85d40bf
Merge remote-tracking branch 'gitri-ms/security' into feature-better-…
dehoward Aug 7, 2023
408f580
Merge branch 'no-auth-webapp' into feature-better-auth
dehoward Aug 7, 2023
f644b37
Merge branch 'main' into feature-better-auth
dehoward Aug 7, 2023
fe1e093
Update deploy scripts to set REACT_APP_AUTH_TYPE in .env
gitri-ms Aug 7, 2023
57c6b47
Update READMEs, enforce tenant ID in deploy scripts
gitri-ms Aug 8, 2023
a1fea64
resolve comments from bugbash
dehoward Aug 8, 2023
ab69bc4
Merge branch 'main' into feature-better-auth
gitri-ms Aug 8, 2023
358b41b
Merge branch 'feature-better-auth' of https://github.com/microsoft/ch…
gitri-ms Aug 8, 2023
cd7857e
Merge branch 'main' into feature-better-auth
gitri-ms Aug 8, 2023
75291fe
deploy-azure.sh: update usage and mandatory args check
gitri-ms Aug 8, 2023
751d990
run `yarn format:fix`
dehoward Aug 8, 2023
c312a22
Change routes back
gitri-ms Aug 9, 2023
cbf915d
bug fixes
gitri-ms Aug 10, 2023
9527350
Rename input to variable
gitri-ms Aug 10, 2023
ddfbdf6
Merge branch 'main' into feature-better-auth
gitri-ms Aug 10, 2023
7517156
Update local scripts to make auth optional
gitri-ms Aug 10, 2023
3419e87
bug fixes
gitri-ms Aug 10, 2023
82dbb02
address pr comments
gitri-ms Aug 12, 2023
8e9e055
Update scripts and bug fixes
gitri-ms Aug 12, 2023
56db1d3
Merge branch 'main' into feature-better-auth
gitri-ms Aug 17, 2023
36dd3e2
Script and workflow updates
gitri-ms Aug 17, 2023
17b2e8f
PR updates
gitri-ms Aug 18, 2023
c10f28e
bug fixes
gitri-ms Aug 18, 2023
598165d
Merge branch 'main' into feature-better-auth
gitri-ms Aug 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .github/workflows/copilot-deploy-environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,8 @@ on:
required: true
AZURE_SUBSCRIPTION_ID:
required: true
WEB_API_KEY:
required: true
AZURE_OPENAI_ENDPOINT:
required: true
APPLICATION_AUTHORITY:
required: true

permissions:
contents: read
Expand All @@ -36,7 +32,6 @@ jobs:
AZURE_CLIENT_ID: ${{secrets.AZURE_CLIENT_ID}}
AZURE_TENANT_ID: ${{secrets.AZURE_TENANT_ID}}
AZURE_SUBSCRIPTION_ID: ${{secrets.AZURE_SUBSCRIPTION_ID}}
WEB_API_KEY: ${{secrets.WEB_API_KEY}}
AZURE_OPENAI_ENDPOINT: ${{secrets.AZURE_OPENAI_ENDPOINT}}

deploy-backend:
Expand All @@ -61,4 +56,3 @@ jobs:
AZURE_CLIENT_ID: ${{secrets.AZURE_CLIENT_ID}}
AZURE_TENANT_ID: ${{secrets.AZURE_TENANT_ID}}
AZURE_SUBSCRIPTION_ID: ${{secrets.AZURE_SUBSCRIPTION_ID}}
APPLICATION_AUTHORITY: ${{secrets.APPLICATION_AUTHORITY}}
4 changes: 1 addition & 3 deletions .github/workflows/copilot-deploy-frontend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,6 @@ on:
required: true
AZURE_SUBSCRIPTION_ID:
required: true
APPLICATION_AUTHORITY:
required: true

permissions:
contents: read
Expand Down Expand Up @@ -90,4 +88,4 @@ jobs:

- name: Deploy SWA
run: |
scripts/deploy/deploy-webapp.sh --subscription ${{secrets.AZURE_SUBSCRIPTION_ID}} --resource-group ${{vars.CC_DEPLOYMENT_GROUP_NAME}} --deployment-name ${{inputs.DEPLOYMENT_NAME}} --application-id ${{vars.APPLICATION_CLIENT_ID}} --authority ${{secrets.APPLICATION_AUTHORITY}} --no-redirect --version ${{ steps.versiontag.outputs.versiontag }} --version-info "Built from commit ${{ steps.gitversion.outputs.ShortSha }} on $(date +"%Y-%m-%d")"
scripts/deploy/deploy-webapp.sh --subscription ${{secrets.AZURE_SUBSCRIPTION_ID}} --resource-group ${{vars.CC_DEPLOYMENT_GROUP_NAME}} --deployment-name ${{inputs.DEPLOYMENT_NAME}} --client-id ${{vars.FRONTEND_CLIENT_ID}} --no-redirect --version ${{ steps.versiontag.outputs.versiontag }} --version-info "Built from commit ${{ steps.gitversion.outputs.ShortSha }} on $(date +"%Y-%m-%d")"
4 changes: 1 addition & 3 deletions .github/workflows/copilot-deploy-infra.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ on:
required: true
AZURE_SUBSCRIPTION_ID:
required: true
WEB_API_KEY:
required: true
AZURE_OPENAI_ENDPOINT:
required: true
outputs:
Expand Down Expand Up @@ -61,4 +59,4 @@ jobs:
inlineScript: |
AI_SERVICE_KEY=$(az cognitiveservices account keys list --name ${{vars.AZUREOPENAI__NAME}} --resource-group ${{vars.AZUREOPENAI_DEPLOYMENT_GROUP_NAME}} | jq -r '.key1')
echo "::add-mask::$AI_SERVICE_KEY"
scripts/deploy/deploy-azure.sh --subscription ${{secrets.AZURE_SUBSCRIPTION_ID}} --web-api-key ${{secrets.WEB_API_KEY}} --resource-group ${{vars.CC_DEPLOYMENT_GROUP_NAME}} --deployment-name ${{steps.deployment-id.outputs.deployment_name}} --region ${{vars.CC_DEPLOYMENT_REGION}} --ai-service AzureOpenAI --ai-endpoint ${{secrets.AZURE_OPENAI_ENDPOINT}} --ai-service-key $AI_SERVICE_KEY --app-service-sku ${{vars.WEBAPP_API_SKU}} --no-deploy-package --debug-deployment
scripts/deploy/deploy-azure.sh --subscription ${{secrets.AZURE_SUBSCRIPTION_ID}} --resource-group ${{vars.CC_DEPLOYMENT_GROUP_NAME}} --deployment-name ${{steps.deployment-id.outputs.deployment_name}} --region ${{vars.CC_DEPLOYMENT_REGION}} --client-id ${{vars.BACKEND_CLIENT_ID}} --tenant-id ${{secrets.AZURE_TENANT_ID}} --instance ${{vars.AZURE_INSTANCE}} --ai-service AzureOpenAI --ai-endpoint ${{secrets.AZURE_OPENAI_ENDPOINT}} --ai-service-key $AI_SERVICE_KEY --app-service-sku ${{vars.WEBAPP_API_SKU}} --no-deploy-package --debug-deployment
4 changes: 0 additions & 4 deletions .github/workflows/copilot-deploy-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,7 @@ jobs:
AZURE_CLIENT_ID: ${{secrets.AZURE_CLIENT_ID}}
AZURE_TENANT_ID: ${{secrets.AZURE_TENANT_ID}}
AZURE_SUBSCRIPTION_ID: ${{secrets.AZURE_SUBSCRIPTION_ID}}
WEB_API_KEY: ${{secrets.WEB_API_KEY}}
AZURE_OPENAI_ENDPOINT: ${{secrets.AZURE_OPENAI_ENDPOINT}}
APPLICATION_AUTHORITY: ${{secrets.APPLICATION_AUTHORITY}}

stable:
uses: ./.github/workflows/copilot-deploy-environment.yml
Expand All @@ -41,6 +39,4 @@ jobs:
AZURE_CLIENT_ID: ${{secrets.AZURE_CLIENT_ID}}
AZURE_TENANT_ID: ${{secrets.AZURE_TENANT_ID}}
AZURE_SUBSCRIPTION_ID: ${{secrets.AZURE_SUBSCRIPTION_ID}}
WEB_API_KEY: ${{secrets.WEB_API_KEY}}
AZURE_OPENAI_ENDPOINT: ${{secrets.AZURE_OPENAI_ENDPOINT}}
APPLICATION_AUTHORITY: ${{secrets.APPLICATION_AUTHORITY}}
3 changes: 2 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
"**/.hg": true,
"**/CVS": true,
"**/.DS_Store": true,
"**/Thumbs.db": true
"**/Thumbs.db": true,
"**.lock": true,
},
}
86 changes: 48 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,6 @@ You will need the following items to run the sample:
- [.NET 7.0 SDK](https://dotnet.microsoft.com/download/dotnet/7.0) _(via Setup script)_
- [Node.js](https://nodejs.org/en/download) _(via Setup script)_
- [Yarn](https://classic.yarnpkg.com/docs/install) _(via Setup script)_
- [Azure account](https://azure.microsoft.com/free)
- [Azure AD Tenant](https://learn.microsoft.com/azure/active-directory/develop/quickstart-create-new-tenant)
- AI Service

| AI Service | Requirement |
Expand All @@ -28,13 +26,6 @@ You will need the following items to run the sample:

# Instructions

## Register an application

1. Follow [these instructions](https://learn.microsoft.com/azure/active-directory/develop/quickstart-register-app) and use the values below:
- `Supported account types`: "_Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)_"
- `Redirect URI (optional)`: _Single-page application (SPA)_ and use _http://localhost:3000_.
2. Take note of the `Application (client) ID`. Chat Copilot will use this ID for authentication.

## Windows

1. Open PowerShell as an administrator.
Expand All @@ -52,19 +43,12 @@ You will need the following items to run the sample:
3. Configure Chat Copilot.

```powershell
.\Configure.ps1 -AIService {AI_SERVICE} -APIKey {API_KEY} -Endpoint {AZURE_OPENAI_ENDPOINT} -ClientId {AZURE_APPLICATION_ID}
.\Configure.ps1 -AIService {AI_SERVICE} -APIKey {API_KEY} -Endpoint {AZURE_OPENAI_ENDPOINT}
```

- `AI_SERVICE`: `AzureOpenAI` or `OpenAI`.
- `API_KEY`: The `API key` for Azure OpenAI or for OpenAI.
- `AZURE_OPENAI_ENDPOINT`: The Azure OpenAI resource `Endpoint` address. Omit `-Endpoint` if using OpenAI.
- `AZURE_APPLICATION_ID`: The `Application (client) ID` associated with the registered application.

- (Optional): To set a specific Tenant Id, use the parameter:

```powershell
-TenantId {TENANT_ID}
```

- > **IMPORTANT:** For `AzureOpenAI`, if you deployed models `gpt-35-turbo` and `text-embedding-ada-002` with custom names (instead of each own's given name), also use the parameters:

Expand Down Expand Up @@ -111,19 +95,13 @@ You will need the following items to run the sample:
3. Configure Chat Copilot.

```bash
./Configure.sh --aiservice {AI_SERVICE} --apikey {API_KEY} --endpoint {AZURE_OPENAI_ENDPOINT} --clientid {AZURE_APPLICATION_ID}
./Configure.sh --aiservice {AI_SERVICE} --apikey {API_KEY} --endpoint {AZURE_OPENAI_ENDPOINT}
```

- `AI_SERVICE`: `AzureOpenAI` or `OpenAI`.
- `API_KEY`: The `API key` for Azure OpenAI or for OpenAI.
- `AZURE_OPENAI_ENDPOINT`: The Azure OpenAI resource `Endpoint` address. Omit `--endpoint` if using OpenAI.
- `AZURE_APPLICATION_ID`: The `Application (client) ID` associated with the registered application.

- (Optional): To set a specific Tenant Id, use the parameter:

```bash
--tenantid {TENANT_ID}
```

- > **IMPORTANT:** For `AzureOpenAI`, if you deployed models `gpt-35-turbo` and `text-embedding-ada-002` with custom names (instead of each own's given name), also use the parameters:

Expand All @@ -141,27 +119,36 @@ You will need the following items to run the sample:

> NOTE: Confirm pop-ups are not blocked and you are logged in with the same account used to register the application.

## (Optional) Enable backend authorization via Azure AD
## (Optional) Enable backend authentication via Azure AD

By default, Chat Copilot runs locally without authentication, using a guest user profile. If you want to enable authentication with Azure Active Directory, follow the steps below.

### Requirements

1. Ensure you created the required application registration mentioned in [Register an application](#register-an-application)
- [Azure account](https://azure.microsoft.com/free)
- [Azure AD Tenant](https://learn.microsoft.com/azure/active-directory/develop/quickstart-create-new-tenant)

2. Create a second application registration to represent the web api
### Instructions

> For more details on creating an application registration, go [here](https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app).
1. Create an [application registration](https://learn.microsoft.com/azure/active-directory/develop/quickstart-register-app) for the frontend web app, using the values below
- `Supported account types`: "_Accounts in this organizational directory only ({YOUR TENANT} only - Single tenant)_"
- `Redirect URI (optional)`: _Single-page application (SPA)_ and use _http://localhost:3000_.

1. Give the app registration a name
2. Create a second [application registration](https://learn.microsoft.com/azure/active-directory/develop/quickstart-register-app) for the backend web api, using the values below:
- `Supported account types`: "_Accounts in this organizational directory only ({YOUR TENANT} only - Single tenant)_"
- Do **not** configure a `Redirect URI (optional)`

2. As _Supported account type_ choose `Accounts in any organizational directory and personal Microsoft Accounts`
> NOTE: Other account types can be used to allow multitenant and personal Microsoft accounts to use your application if you desire. Doing so may result in more users and therefore higher costs.

3. Do not configure a _Redirect Uri_
> Take note of the `Application (client) ID` for both app registrations as you will need them in future steps.

3. Expose an API within the second app registration

1. Select _Expose an API_ from the menu

2. Add an _Application ID URI_

1. This will generate an `api://` URI with a generated for you
1. This will generate an `api://` URI

2. Click _Save_ to store the generated URI

Expand Down Expand Up @@ -193,17 +180,40 @@ You will need the following items to run the sample:

7. Click _Add permissions_

5. Update frontend web app configuration
5. Run the Configure script with additional parameters to set up authentication.

1. Open _.env_ file
**Powershell**

2. Set the value of `REACT_APP_AAD_API_SCOPE` to your application ID URI followed by the scope `access_as_user`, e.g. `api://12341234-1234-1234-1234-123412341234/access_as_user`
```powershell
.\Configure.ps1 -AiService {AI_SERVICE} -APIKey {API_KEY} -Endpoint {AZURE_OPENAI_ENDPOINT} -FrontendClientId {FRONTEND_APPLICATION_ID} -BackendClientId {BACKEND_APPLICATION_ID} -TenantId {TENANT_ID} -Instance {AZURE_AD_INSTANCE}
```

6. Update backend web api configuration
**Bash**
```bash
./Configure.sh --aiservice {AI_SERVICE} --apikey {API_KEY} --endpoint {AZURE_OPENAI_ENDPOINT} --frontend-clientid {FRONTEND_APPLICATION_ID} --backend-clientid {BACKEND_APPLICATION_ID} --tenantid {TENANT_ID} --instance {AZURE_AD_INSTANCE}
```

1. Open _appsettings.json_
- `AI_SERVICE`: `AzureOpenAI` or `OpenAI`.
- `API_KEY`: The `API key` for Azure OpenAI or for OpenAI.
- `AZURE_OPENAI_ENDPOINT`: The Azure OpenAI resource `Endpoint` address. Omit `-Endpoint` if using OpenAI.
- `FRONTEND_APPLICATION_ID`: The `Application (client) ID` associated with the application registration for the frontend.
- `BACKEND_APPLICATION_ID`: The `Application (client) ID` associated with the application registration for the backend.
- `TENANT_ID` : Your Azure AD tenant ID
- `AZURE_AD_INSTANCE` _(optional)_: The Azure AD cloud instance for the authenticating users. Defaults to `https://login.microsoftonline.com`.

2. Set the value of `Authorization:AzureAd:Audience` to your application ID URI
6. Run Chat Copilot locally. This step starts both the backend API and frontend application.

**Powershell**

```powershell
.\Start.ps1
```

**Bash**

```bash
./Start.sh
```

# Troubleshooting

Expand Down
5 changes: 4 additions & 1 deletion scripts/.env
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@ ENV_PLANNER_MODEL_OPEN_AI="gpt-3.5-turbo"
ENV_COMPLETION_MODEL_AZURE_OPEN_AI="gpt-35-turbo"
ENV_PLANNER_MODEL_AZURE_OPEN_AI="gpt-35-turbo"
ENV_EMBEDDING_MODEL="text-embedding-ada-002"
ENV_TENANT_ID="common"
ENV_ASPNETCORE="Development"
ENV_INSTANCE="https://login.microsoftonline.com"

# Constants
ENV_AZURE_OPEN_AI="AzureOpenAI"
ENV_OPEN_AI="OpenAI"
ENV_AZURE_AD="AzureAd"
ENV_NONE="None"
ENV_SCOPES="access_as_user"
Loading