Skip to content

Replace Docker with Docker Compose for running the DB to be targeted by integration tests #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 193 commits into from
Sep 3, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
193 commits
Select commit Hold shift + click to select a range
5bdc1e6
Prepare the grounds for using Docker Compose when running integration…
satrapu May 3, 2020
b1a7071
Improve script used for checking container health state
satrapu May 7, 2020
cd7c21b
Add support for enabling printing info for debugging purposes
satrapu May 7, 2020
8695e45
Merge branch 'master' into feature/use-docker-compose-for-it
satrapu Jun 11, 2020
aff84ce
Integration tests are now targeting a database run via Docker Compose
satrapu Jun 11, 2020
3b28b38
Fix incorrect YAML property
satrapu Jun 11, 2020
cc05fcf
Remove comments from YAML fragment as Azure DevOps complains about them
satrapu Jun 11, 2020
5dd5361
Fix path pointing to Docker Compose file
satrapu Jun 11, 2020
692d3eb
Add pipe symbol as requested by Docker Compose task
satrapu Jun 11, 2020
bb87704
Enable debugging for Docker Compose services
satrapu Jun 11, 2020
83a6a77
Use hard-coded sleep to ensure compose services are up when running IT
satrapu Jun 11, 2020
d50cbba
Fix YAML indentation
satrapu Jun 11, 2020
12db4b6
Give enough time to PostgreSQL DB to start before running IT
satrapu Jun 11, 2020
4cf02b6
Tweak Docker Compose task parameters
satrapu Jun 11, 2020
d4e6abc
Ensure compose services are healthy before running IT
satrapu Jun 12, 2020
3aa6cff
Fix Docker Compose command used for checking service status
satrapu Jun 12, 2020
05634a3
Use PowerShell Core to interact with Docker Compose
satrapu Jun 12, 2020
fcabdeb
Fix passing environment variables to PowerShell Core script
satrapu Jun 12, 2020
8d7c01e
Fix PowerShell Core script syntax
satrapu Jun 12, 2020
081e40b
Fix Azure DevOps logging command syntax used for displaying info mess…
satrapu Jun 12, 2020
c9d9b0f
Temporarily hard-code host port for the db compose service
satrapu Jun 12, 2020
ba65bc8
Provide a default database port for the template job file
satrapu Jun 12, 2020
f2a7f42
Log database host port for debugging purposes
satrapu Jun 12, 2020
56f442a
Identify the Docker host port associated with the 'db' compose service
satrapu Jun 12, 2020
4e5ccde
Publish test log file for debugging purposes
satrapu Jun 12, 2020
4b8832f
Publish test log file for debugging purposes
satrapu Jun 12, 2020
aa1f86b
Temporarily log IT connection string for debugging purposes
satrapu Jun 12, 2020
f7ab1c2
Fix typo in parameter reference
satrapu Jun 12, 2020
aeb3476
Remove logging connection string as CI pipeline has been fixed
satrapu Jun 12, 2020
bc3d13c
Fix typo
satrapu Jun 24, 2020
b3fd5f6
Use PowerShell script to start compose services used by IT
satrapu Jul 26, 2020
2227025
Remove no longer needed build step
satrapu Jul 26, 2020
766826f
Wrap environment variable values with single quotes
satrapu Jul 26, 2020
adda69d
Remove > symbol since PowerShell was misinterpreting it
satrapu Jul 26, 2020
ec25482
Split PowerShell long line via tick symbol
satrapu Jul 26, 2020
e62359a
Un-ingore .env file to be used by Docker Compose
satrapu Jul 26, 2020
dc86f6a
Avoid breaking build due to Docker Compose writing to standard error
satrapu Jul 26, 2020
7a96ecb
Fix referencing variables storing connection strings to be used when …
satrapu Jul 26, 2020
a91808b
Fix variable name by adding missing dot symbol
satrapu Jul 26, 2020
09967da
Fix referencing variables in steps related to running IT
satrapu Jul 26, 2020
959d1f3
Move PowerShell script used for starting compose service to a better …
satrapu Jul 26, 2020
87b17f7
Move PowerShell script used for starting compose service to a better …
satrapu Jul 26, 2020
600161c
Move PowerShell script used for starting compose service to a better …
satrapu Jul 26, 2020
63fc7c7
Move PowerShell script used for starting compose service to a better …
satrapu Jul 26, 2020
67af97b
Remove invalid string
satrapu Jul 27, 2020
7c4a746
Add comments and validation rules to PowerShell script parameters
satrapu Jul 27, 2020
bcdc459
Fix displaying message about mapping container port to host one
satrapu Jul 27, 2020
1d3d14b
Fix displaying variable value after mapping container port to host one
satrapu Jul 27, 2020
00f3e96
Made Docker Compose service names more explicit
satrapu Jul 27, 2020
e33385b
Add macOS agent
satrapu Aug 2, 2020
70f0896
Fail script if Docker Compose writes anything to standard error stream
satrapu Aug 2, 2020
170cc31
Fail build in case compose services fail to start
satrapu Aug 2, 2020
2c26f3f
Fix handling Docker Compose command standard streams
satrapu Aug 2, 2020
3239399
Fix redirecting all streams to null
satrapu Aug 2, 2020
43fb16a
Fix redirecting success,warning and error streams to null
satrapu Aug 2, 2020
c249f4e
Temporarily disable build on macOS agents till the pipeline is workin…
satrapu Aug 2, 2020
c274dc4
Replace redirecting Docker Compose streams with --log-level flag
satrapu Aug 2, 2020
29f9124
Replace redirecting Docker Compose streams with --log-level flag, take 2
satrapu Aug 2, 2020
ab05993
Replace redirecting Docker Compose streams with --log-level flag, take 3
satrapu Aug 2, 2020
a1f26de
Replace redirecting Docker Compose streams with --log-level flag, take 4
satrapu Aug 2, 2020
fa25349
Detect errors while running Docker Compose via $LASTEXITCODE
satrapu Aug 2, 2020
63ee4ec
Fix exit condition when everything it's OK
satrapu Aug 2, 2020
46e0931
Re-enable macOS agent
satrapu Aug 2, 2020
74cb8a9
Install Docker Compose via brew on macOS agent
satrapu Aug 2, 2020
f02f9cc
Revert commit since compose issue occurring on macOS agent is caused …
satrapu Aug 2, 2020
1e61cc3
Use compose file v3.7 since macOS agent cannot run a newer version of…
satrapu Aug 2, 2020
c85ced1
Fix incrementing the counter used when checking whether Docker servic…
satrapu Aug 2, 2020
2331abe
Enable Windows-based agent too
satrapu Aug 2, 2020
5d30020
Fix validating input parameter
satrapu Aug 2, 2020
c1dafb2
Parameterize the name of the Docker image used for running the DB to …
satrapu Aug 2, 2020
54e7fc4
Fix passing arguments to PowerShell script
satrapu Aug 2, 2020
158cf24
Explicitly set string properties to avoid issues due to invalid types
satrapu Aug 2, 2020
c83687f
Fix reference to compose variable
satrapu Aug 2, 2020
0dab18c
Log more info for debugging purposes
satrapu Aug 2, 2020
0cc3e24
Log more info for debugging purposes, take 2
satrapu Aug 2, 2020
274075a
Wrap parameter values with single quotes
satrapu Aug 2, 2020
c0ac113
Explicitly set job parameters to avoid pipeline considering them not …
satrapu Aug 2, 2020
ef69136
Remove no longer needed logging messages
satrapu Aug 2, 2020
222106f
Fix parameter name in XML doc
satrapu Aug 4, 2020
0bd05a1
Disable ReSharper warning about unused variable since it will be used…
satrapu Aug 4, 2020
9b01fa5
Add new word to project dictionary
satrapu Aug 4, 2020
afaa2f8
Avoid possible NullReferenceException when running integration tests
satrapu Aug 4, 2020
7f7ba34
Disable ReSharper warning since I'm testing whether the constructor w…
satrapu Aug 4, 2020
a44a6cb
Merge latest from master branch
satrapu Aug 4, 2020
3f49b5a
Since compose services are run as part of CI pipeline, no need for at…
satrapu Aug 4, 2020
332ae14
Add extra link to article describing the meaning of 'xmin' property i…
satrapu Aug 4, 2020
4d68fd5
Remove redundant environment variables
satrapu Aug 4, 2020
0bfc713
Cache NuGet packages when requested only
satrapu Aug 4, 2020
21fc61d
Remove unused 'databasePort' job parameter
satrapu Aug 4, 2020
d40ac84
Remove extra whitespaces
satrapu Aug 4, 2020
1ea2976
Restoring NuGet packages happens only in case there is no cache for them
satrapu Aug 4, 2020
2fee0e6
Fix condition by adding missing round bracket
satrapu Aug 4, 2020
05be958
Fix condition used for evaluating whether restoring NuGet packages is…
satrapu Aug 4, 2020
3158351
Remove whitespaces since they break the commands
satrapu Aug 4, 2020
32e74bd
Remove job parameters default values since they bring no real value r…
satrapu Aug 4, 2020
238f16b
Update .NET Core SDK to v3.1.302
satrapu Aug 4, 2020
1856e18
Update ReportGenerator to v4.6.4
satrapu Aug 4, 2020
10c92d4
Rollback updating ReportGenerator to v4.6.4 since Azure DevOps feed d…
satrapu Aug 4, 2020
32dc88c
Try installing latest available version of Docker via homebrew
satrapu Aug 4, 2020
d48570d
Rollback installing latest version of Docker since macOS based agent …
satrapu Aug 4, 2020
77d52b1
Replace hard-coded value with variable
satrapu Aug 4, 2020
b8313e9
Set default value for job parameter to avoid code duplication
satrapu Aug 5, 2020
04d922e
Log connection string pointing to test database for debugging purposes
satrapu Aug 5, 2020
82d726a
The databaseHost job parameter must be explicitly set
satrapu Aug 5, 2020
788277f
Experiment with specifying volume for the DB to be targeted by integr…
satrapu Aug 5, 2020
4832127
Experiment with specifying volume for the DB to be targeted by integr…
satrapu Aug 5, 2020
2fda052
Experiment with specifying volume for the DB to be targeted by integr…
satrapu Aug 5, 2020
657ef01
Fix Docker Compose volume related error occurring on Windows based ag…
satrapu Aug 5, 2020
b44c291
Add missing key-value pair separator
satrapu Aug 5, 2020
fbfdf1e
Wrap environment variable in single quotes hoping
satrapu Aug 5, 2020
7b28bb8
Experiment with specifying volume for the DB to be targeted by integr…
satrapu Aug 5, 2020
91fe0ca
Set volume path according to the Docker container (Linux or Windows)
satrapu Aug 5, 2020
2c607e0
Make script starting compose services more robust
satrapu Aug 5, 2020
8869657
Fix splitting long line in PowerShell script
satrapu Aug 5, 2020
94194ff
Make script starting compose services more robust, take 2
satrapu Aug 5, 2020
b4c96c0
Fix creating a custom object in PowerShell
satrapu Aug 5, 2020
ce1773e
Fix Docker Compose command broken due to formatting code
satrapu Aug 5, 2020
5d9c98f
Fix referencing true and false boolean values
satrapu Aug 5, 2020
e5f25f7
Fix referencing container id when fetching port mappings
satrapu Aug 5, 2020
f71e9ef
Wrap container port with double quotes hoping port mappings will be f…
satrapu Aug 5, 2020
d3c2ddf
Specify host port to avoid Windows-based Azure DevOps agents not fetc…
satrapu Aug 5, 2020
762e2cd
Let again Docker assign a dynamic host port
satrapu Aug 5, 2020
f99368c
Replace identifying ports via 'docker port' with 'docker inspect'
satrapu Aug 5, 2020
c822924
Fix variable name
satrapu Aug 5, 2020
0a9edf4
Fix fetching all JSON properties
satrapu Aug 5, 2020
7b70584
Create Docker volume before starting compose services
satrapu Aug 8, 2020
53a5ddd
Fix declaring named volume
satrapu Aug 8, 2020
9cae3da
Explicitly set durations used by Docker Compose
satrapu Aug 8, 2020
f8a0597
Wrap strings in double quotes
satrapu Aug 8, 2020
e5cfeec
Rename job parameter to increase code readability
satrapu Aug 8, 2020
3bc86ce
Display message about creating Docker volume to make the build step m…
satrapu Aug 8, 2020
bdd8f78
Fix passing job parameter as PowerShell script parameter value
satrapu Aug 8, 2020
4fe6b18
Ignore comments and empty lines found in Docker Compose environment file
satrapu Aug 8, 2020
7146daa
Expose Docker Compose logs as build artifact for debugging purposes
satrapu Aug 8, 2020
c3c3d25
Let the PowerShell script handle errors
satrapu Aug 8, 2020
d4993b0
Publish Docker Compose logs as build artifact
satrapu Aug 8, 2020
b577f2a
Publish Docker Compose logs as build artifact, take 2
satrapu Aug 8, 2020
fedebd5
Display Docker Compose logs
satrapu Aug 8, 2020
274f068
Publish Docker Compose logs via build step
satrapu Aug 8, 2020
93aaf70
Fix splitting long line into multiple shorter ones
satrapu Aug 8, 2020
eeccb4b
Fix publishing Docker Compose logs
satrapu Aug 8, 2020
ca6f6e8
Create missing folders if needed when writing Docker Compose log file…
satrapu Aug 8, 2020
1b2114e
Create missing folders if needed when writing Docker Compose log file…
satrapu Aug 8, 2020
a473436
Ensure Docker Compose writing to standard error stream does not break…
satrapu Aug 8, 2020
8a577fc
Remove no longer needed Azure DevOps command
satrapu Aug 8, 2020
e4894ee
Add compose project name to the name of the compose service log file …
satrapu Aug 8, 2020
f60a316
Add missing parameter when creating Docker volumes
satrapu Aug 8, 2020
166e9ed
Print Azure DevOps-style errors when Docker commands fail
satrapu Aug 8, 2020
fe15705
Add timestamps to the published Docker Compose logs
satrapu Aug 8, 2020
56a8f2d
Avoid checking whether commands have ended with success since they wr…
satrapu Aug 8, 2020
f7406be
Fix container path passed as job parameter value
satrapu Aug 8, 2020
6fe2b86
Publish compose service log files in a folder per Azure DevOps agent OS
satrapu Aug 8, 2020
9cb8392
Fix container path
satrapu Aug 8, 2020
82fcefc
Fixcommand used for creating compose service volume
satrapu Aug 8, 2020
7c563c8
Make use of PGDATA env var in order to fix Windows-based agent error
satrapu Aug 8, 2020
eafa359
Fix container path
satrapu Aug 8, 2020
7532605
Fix path pointing to test related artifacts to be published
satrapu Aug 8, 2020
03c9b0c
Fix container path, take 2
satrapu Aug 8, 2020
04e3bef
MOve .artifactignore file to repo root, as recommended by official doc
satrapu Aug 8, 2020
0ab6ab7
Fix container path, take 3
satrapu Aug 8, 2020
4e91300
Simplify folder path storing published compose service log file
satrapu Aug 8, 2020
0149118
Tune a little bit the health check durations
satrapu Aug 8, 2020
ecaca6f
Print message for debugging purposes
satrapu Aug 8, 2020
22dbab7
Rephrase message to include compose project name
satrapu Aug 8, 2020
a256bd6
Tune a little bit printing message for debugging purposes
satrapu Aug 8, 2020
293a8fb
Fix printing message
satrapu Aug 8, 2020
b97c279
Fix printing another message
satrapu Aug 8, 2020
36c9404
Ensure no PowerShell errors goe unnoticed
satrapu Aug 8, 2020
cd26923
Need to set errorActionPreference to Continue and failOnStderr to Fal…
satrapu Aug 8, 2020
52f8516
Remove db4it volume - let's see what happens
satrapu Aug 8, 2020
6e1404c
Remove any trace of Docker volume since PostgreSQL compose service do…
satrapu Aug 8, 2020
a3533fa
Polish messages printed for debugging purposes
satrapu Aug 8, 2020
8d92646
Refine publish artifact exclusion and inclusion rules
satrapu Aug 9, 2020
77e23a0
Reduce number of if-statements
satrapu Aug 9, 2020
b55dd5e
Move .artifactignore file under Tests folder to limit what gets publi…
satrapu Aug 9, 2020
47f3240
Change build step id and name to better reflect its purpose
satrapu Aug 9, 2020
64d127f
Ensure compose project name is unique across each builds
satrapu Aug 9, 2020
859c87e
Use minimum amount of info to ensure unique compose project names acr…
satrapu Aug 9, 2020
33befbc
Remove no longer used variable
satrapu Aug 9, 2020
63e3a4a
Rename compose project name to follow the Azure DevOps user-defined v…
satrapu Aug 9, 2020
7d3fea6
Make compose project name even shorter
satrapu Aug 9, 2020
c091c27
Ensure compose project name contains letters only, as using numbers (…
satrapu Aug 9, 2020
2444a91
Remove job parameter duplication
satrapu Aug 9, 2020
ae68f17
Ensure job parameter is set outside job template file to avoid consid…
satrapu Aug 9, 2020
8f6f736
Revert compose project name to last working value
satrapu Aug 9, 2020
62c9904
Specify job parameter value outside job template to ensure it will no…
satrapu Aug 9, 2020
e50c2ed
Fix variable name used as example
satrapu Aug 24, 2020
2fdfacb
Change variable name to correctly state its purpose
satrapu Aug 29, 2020
2c72868
Temporarily expose more ports for testing purposes
satrapu Aug 30, 2020
b430814
Revert multiple ports to just one
satrapu Aug 30, 2020
a40a1e3
Make use of 'docker container <VERB>' commands
satrapu Aug 31, 2020
8a114ee
Declare external volume for the compose service
satrapu Sep 3, 2020
2210232
Remove volume since build fails on Windows
satrapu Sep 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 0 additions & 94 deletions Build/Provision-Docker-container-using-healthcheck.ps1

This file was deleted.

97 changes: 0 additions & 97 deletions Build/Provision-Docker-container-using-log-polling.ps1

This file was deleted.

69 changes: 69 additions & 0 deletions Build/PublishComposeServiceLogs.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# This script saves the compose service log files to a given folder.
Param(
# Docker Compose project name.
# See more here: https://docs.docker.com/compose/reference/overview/#use--p-to-specify-a-project-name.
[String]
$ComposeProjectName = 'aspnet-core-logging-it',

# The path to the folder where Docker Compose will write its logs.
[String]
$LogsOutputFolder
)

$InfoMessage = "The log files of compose services from project: $ComposeProjectName " `
+ "will be written to folder: `"$LogsOutputFolder`""
Write-Output "$InfoMessage"

$LsCommandOutput = docker container ls -a `
--filter "label=com.docker.compose.project=$ComposeProjectName" `
--format "{{ .ID }}" `
| Out-String

if ((!$?) -or ($LsCommandOutput.Length -eq 0))
{
Write-Output "##vso[task.LogIssue type=error;]Failed to identify compose services for project: $ComposeProjectName"
Write-Output "##vso[task.complete result=Failed;]"
exit 4;
}

Write-Output "Found the following container(s) under compose project $($ComposeProjectName): $LsCommandOutput"

$LsCommandOutput.Split([System.Environment]::NewLine, [System.StringSplitOptions]::RemoveEmptyEntries) | ForEach-Object {
$ContainerId = $_
$ComposeServiceLabelsAsJson = docker inspect --format '{{ json .Config.Labels }}' `
"$ContainerId" `
| Out-String `
| ConvertFrom-Json

if (!$?)
{
Write-Output "##vso[task.LogIssue type=error;]Failed to inspect container with ID: $ContainerId"
Write-Output "##vso[task.complete result=Failed;]"
exit 2;
}

$ComposeServiceNameLabel = 'com.docker.compose.service'
$ComposeServiceName = $ComposeServiceLabelsAsJson.$ComposeServiceNameLabel
$LogFileName = "$ComposeProjectName--$ComposeServiceName--$ContainerId.log"
$LogFilePath = Join-Path -Path $LogsOutputFolder $LogFileName

$PublishLogsInfoMessage = "About to publish logs for compose service with container id: " `
+ "`"$ContainerId`" and service name: " `
+ "`"$ComposeServiceName`" to file: `"$LogFilePath`" ..."
Write-Output $PublishLogsInfoMessage

if (![System.IO.File]::Exists($LogsOutputFolder))
{
New-Item -Path "$LogsOutputFolder" -ItemType "Directory"
}

# Do not check whether this command has ended successfully since it's writing to
# standard error stream, thus tricking runtime into thinking it failed.
docker logs --tail "all" `
--details `
--timestamps `
"$ContainerId" `
| Out-File -FilePath "$LogFilePath"
}

exit 0;
Loading