Skip to content

Commit

Permalink
Merge pull request backstage#4017 from viavarejo/feature/techdocs-azu…
Browse files Browse the repository at this point in the history
…re-storage
  • Loading branch information
OrkoHunter authored Jan 31, 2021
2 parents 9a5244a + ee271bf commit 8eb862c
Show file tree
Hide file tree
Showing 15 changed files with 747 additions and 9 deletions.
6 changes: 6 additions & 0 deletions .changeset/chilly-dodos-drop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@backstage/techdocs-common': patch
'@backstage/plugin-techdocs-backend': patch
---

1. Added option to use Azure Blob Storage as a choice to store the static generated files for TechDocs.
2 changes: 1 addition & 1 deletion app-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ techdocs:
generators:
techdocs: 'docker' # Alternatives - 'local'
publisher:
type: 'local' # Alternatives - 'googleGcs' or 'awsS3'. Read documentation for using alternatives.
type: 'local' # Alternatives - 'googleGcs' or 'awsS3' or 'azureBlobStorage'. Read documentation for using alternatives.

sentry:
organization: my-company
Expand Down
12 changes: 6 additions & 6 deletions docs/features/techdocs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,12 +108,12 @@ providers are used.
| GitLab | Yes ✅ |
| GitLab Enterprise | Yes ✅ |

| File Storage Provider | Support Status |
| --------------------------------- | ----------------------------------------------------------------- |
| Local Filesystem of Backstage app | Yes ✅ |
| Google Cloud Storage (GCS) | Yes ✅ |
| Amazon Web Services (AWS) S3 | Yes ✅ |
| Azure Storage | No ❌ [#3938](https://github.com/backstage/backstage/issues/3938) |
| File Storage Provider | Support Status |
| --------------------------------- | -------------- |
| Local Filesystem of Backstage app | Yes ✅ |
| Google Cloud Storage (GCS) | Yes ✅ |
| Amazon Web Services (AWS) S3 | Yes ✅ |
| Azure Blob Storage | Yes ✅ |

[Reach out to us](#feedback) if you want to request more platforms.

Expand Down
19 changes: 18 additions & 1 deletion docs/features/techdocs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ techdocs:
# or you want to use External storage providers like Google Cloud Storage, AWS S3, etc.

publisher:
# techdocs.publisher.type can be - 'local' or 'googleGcs' or 'awsS3' (azureStorage to be available in future).
# techdocs.publisher.type can be - 'local' or 'googleGcs' or 'awsS3' or 'azureBlobStorage'.
# When set to 'local', techdocs-backend will create a 'static' directory at its root to store generated documentation files.
# When set to 'googleGcs', techdocs-backend will use a Google Cloud Storage Bucket to store generated documentation files.
# When set to 'awsS3', techdocs-backend will use an Amazon Web Service (AWS) S3 bucket to store generated documentation files.
Expand Down Expand Up @@ -84,4 +84,21 @@ techdocs:
# https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-region.html
region:
$env: AWS_REGION

# Required when techdocs.publisher.type is set to 'azureBlobStorage'. Skip otherwise.

azureBlobStorage:
# (Required) Azure Blob Storage Container Name
containerName: 'techdocs-storage'

# (Required) An account name is required to write to a storage blob container.
# https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key
credentials:
accountName:
$env: TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_NAME
# (Optional) An account key is required to write to a storage container.
# If missing,AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET environment variable will be used.
# https://docs.microsoft.com/en-us/azure/storage/common/storage-auth?toc=/azure/storage/blobs/toc.json
accountKey:
$env: TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_KEY
```
91 changes: 91 additions & 0 deletions docs/features/techdocs/using-cloud-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,3 +195,94 @@ Your Backstage app is now ready to use AWS S3 for TechDocs, to store and read
the static generated documentation files. When you start the backend of the app,
you should be able to see
`techdocs info Successfully connected to the AWS S3 bucket` in the logs.

## Configuring Azure Blob Storage Container with TechDocs

Follow the
[official Azure Blob Storage documentation](https://docs.microsoft.com/en-us/azure/storage/common/storage-auth?toc=/azure/storage/blobs/toc.json)
for the latest instructions on the following steps involving Azure Blob Storage.

**1. Set `techdocs.publisher.type` config in your `app-config.yaml`**

Set `techdocs.publisher.type` to `'azureBlobStorage'`.

```yaml
techdocs:
publisher:
type: 'azureBlobStorage'
```

**2. Create an Azure Blob Storage Container**

Create a dedicated container for TechDocs sites.
[Refer to the official documentation](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal).

TechDocs will publish documentation to this container and will fetch files from
here to serve documentation in Backstage. Note that the container names are
globally unique.

Set the config `techdocs.publisher.azureBlobStorage.containerName` in your
`app-config.yaml` to the name of the container you just created.

```yaml
techdocs:
publisher:
type: 'azureBlobStorage'
azureBlobStorage:
containerName: 'name-of-techdocs-storage-container'
```

**3a. (Recommended) Authentication using environment variable**

Set the config `techdocs.publisher.azureBlobStorage.credentials.accountName` in
your `app-config.yaml` to the your account name.

The storage blob client will automatically use the environment variable
`AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET` to authenticate with
Azure Blob Storage.
[Steps to create the service where the variables can be retrieved from](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal).

https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad for more
details.

```yaml
techdocs:
publisher:
type: 'azureBlobStorage'
azureBlobStorage:
containerName: 'name-of-techdocs-storage-bucket'
credentials:
accountName:
$env: TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_NAME
```

**3b. Authentication using app-config.yaml**

If you do not prefer (3a) and optionally like to use a service account, you can
follow these steps.

To get credentials, access the Azure Portal and go to "Settings > Access Keys",
and get your Storage account name and Primary Key.
https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key
for more details.

```yaml
techdocs:
publisher:
type: 'azureBlobStorage'
azureBlobStorage:
containerName: 'name-of-techdocs-storage-bucket'
credentials:
accountName:
$env: TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_NAME
accountKey:
$env: TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_KEY
```

**4. That's it!**

Your Backstage app is now ready to use Azure Blob Storage for TechDocs, to store
and read the static generated documentation files. When you start the backend of
the app, you should be able to see
`techdocs info Successfully connected to the Azure Blob Storage container` in
the logs.
20 changes: 20 additions & 0 deletions packages/techdocs-common/__mocks__/@azure/identity.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/*
* Copyright 2020 Spotify AB
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
export class DefaultAzureCredential {
/**
* Creates an instance of the DefaultAzureCredential class.
*/
}
86 changes: 86 additions & 0 deletions packages/techdocs-common/__mocks__/@azure/storage-blob.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
/*
* Copyright 2020 Spotify AB
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import fs from 'fs';

export class BlockBlobClient {
private readonly blobName;

constructor(blobName: string) {
this.blobName = blobName;
}

uploadFile(source: string) {
return new Promise((resolve, reject) => {
if (!fs.existsSync(source)) {
reject('');
} else {
resolve('');
}
});
}

exists() {
return new Promise((resolve, reject) => {
if (fs.existsSync(this.blobName)) {
resolve(true);
} else {
reject({ message: 'The object doest not exist !' });
}
});
}
}

export class ContainerClient {
private readonly containerName;

constructor(containerName: string) {
this.containerName = containerName;
}

getProperties() {
return new Promise(resolve => {
resolve('');
});
}

getBlockBlobClient(blobName: string) {
return new BlockBlobClient(blobName);
}
}

export class BlobServiceClient {
private readonly url;
private readonly credential;

constructor(url: string, credential?: StorageSharedKeyCredential) {
this.url = url;
this.credential = credential;
}

getContainerClient(containerName: string) {
return new ContainerClient(containerName);
}
}

export class StorageSharedKeyCredential {
private readonly accountName;
private readonly accountKey;

constructor(accountName: string, accountKey: string) {
this.accountName = accountName;
this.accountKey = accountKey;
}
}
2 changes: 2 additions & 0 deletions packages/techdocs-common/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
},
"dependencies": {
"@aws-sdk/client-s3": "^3.1.0",
"@azure/identity": "^1.2.2",
"@azure/storage-blob": "^12.4.0",
"@backstage/backend-common": "^0.5.1",
"@backstage/catalog-model": "^0.7.0",
"@backstage/config": "^0.1.2",
Expand Down
Loading

0 comments on commit 8eb862c

Please sign in to comment.