Skip to content

Commit

Permalink
Dotnet6 agent merge master (#4243)
Browse files Browse the repository at this point in the history
* Catch updating exception (#4082)

* Node16 version updated to 16.17.1" (#4084)

* Bump ubuntu image version to 20.04 (#4079)

* Prevent creating host container network (#3788)

* Prevent creating host container network

* Remove redundant empty lines

* Fix invalid knob value fetching

* Update ContainerOperationProvider.cs

Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com>
Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>
Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>

* Allow env.sh to accept additional env var names (#3339)

The hard-coded list is not necessarily enough. This allows the caller
to specify additional environment variables to write to .env.

Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>
Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* Revert "Fix execution of scripts in variables (#4016)" (#4088)

This reverts commit af5328b.

* Add Ubuntu 18.04 (#3943)

Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>
Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>

* Make agent update list of systems supporting .NET 6 periodically (#4081)

* implemented logic to fetch net6.json from server every hour

* added await keyword

* Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async

* fixed issue with condition in Equals method of OperatingSystem class

* small refactoring

* net6.json must be read only if it was not fetched from server

* refactored code of method "GetNet6SupportedSystems"

* restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour

Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* Retries to lock Services database (#4085)

* retries to lock database

* Decrease timeout + Add retry message

Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com>
Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>

* Fixed name of DockerAdditionalNetworkOptions Knob (#3888)

Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>

* Format agent source code (#4101)

* Run dotnet format

* Remove extra semicolons

* Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404)

* Add script to predict v2->v3 upgrade compatibility (#4092)

* Add script to predict v2->v3 upgrade compatibility

* Fix typo

* Fixed another typo: frpom

Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* Added strict boolean knob values (#4103)

* Add ConvertToBooleanStrict

* Add AsBooleanStrict for agent knobs

* update build-job (#4110)

* update build-job
- added UseDotNet task for linux os

* Switch EsrpCodeSigning from V1 to V2
returned back UseDotNet task condition

* Set UseDotNet version to 6 on osx

* added dotnet 3.1.x before unit tests on win x86

* fix dotnet host search (#4124)

* fix dotnet host search
- added performMultiLevelLookup

* Added runtime to unitTests step

* Changed patch version in semver of runtime

* remove sdk and runtime from unitTests script

* Fixed problem with .NET Core 3.1 installation

* Added UseDotNet task in functionalTests

Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com>

* Add option to skip rhel6 in pipeline and ci (#4121)

* skipRhelRelease parameter added

* skipRhelRelease parameter fixed

* releaseRhelOnly parameter added

* Logic of removing not filling lines from releaseNote

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Fix pipeline with test releaseNote job

* Removed test logs

* Added clear empty lines logic

* Commented code for test. Test changes in pipeline

* Removed test changes

* skipRhelRelease parametr added to ci pipeline

* Update dotnet to 3.1.32 (#4112)

* DockerLogin retries (#4100)

* DockerLogin retries
- added changes from PR #3840
- fixed incorrect variables

* DockerLogin retries
- added step UseDotNet@2

* DockerLogin retries
- added linux to UseDotNet task

* Docker Login retries
Implement retry into DockerLogin method

* Implement feature flag for docker login retries

* revert build-job back

* Added retry for docker start

* fix delay

* update knob description

* Update agent Docker methods
- reworked Docker methods with retries behavior
- changed InvokeWithRetryIfNonZero on to incapsulated logic method

* - renamed local variable as is in ADO

* Added docker retry behavior to docker version method

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>
Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com>

* Add the exact operation system version and environment to the telemetry (#4104)

* Update TaskRunner.cs

* Update build-job.yml

* Added new variables to telemetry AgentName, MachineName, IsSelfHosted

Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* Mingit version upgraded to 2.39.1 (#4126)

* Fix agent lint errors (#4117)

* Fix CA2000 error with secrets masker

* Fix CA1711 for ServiceBootFlag

* Fix CA2000 for JobRunner

* Resolve CA2000 in StepHost

* Resolve CA2000 in Windows service

* Resolve CA2000 for WorkerCommandManager

* Resove CA2000 for CodeCoverageCommands

* Remove unused namespaces

* Add fixture for SecretMasker tests

* Change Fixtures -> Disposable

* Use dispose pattern to fix errors

* Formatting

* Revert "Allow env.sh to accept additional env var names (#3339)" (#4143)

This reverts commit 27e3d4e.

* Bump azure pipelines task lib to 4.2.0 (#4155)

* Add warning for deprecated runners (#4162)

* Add warning for deprecated runners

* Update strings.json

* fix typo

* Update NodeHandler.cs

* FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage (#4166)

* FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage

* Re-write try-catch

* Update TaskRunner.cs

* cleanup

* Distinguish AzureInstanceMetadata detection to a separate class

---------

Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* Added legacy lib source for support on Ubuntu 22.04 (#4161)

* Moved logic to drain queues after each task under knob "AGENT_DRAIN_QUEUES_AFTER_TASK" (#4176)

* Added knob "DrainQueuesAfterTask" with default "false"

* move logic to drain queues behind knob DrainQueuesAfterTask

* added mocking of GetScopedEnvironment method in step execution context

* added mocking of GetScopedEnvironment method in step execution context

* changed default value for knob "AGENT_DRAIN_QUEUES_AFTER_TASK" to true

* implementd AsBooleanStrict and check in JobRunner

* Revert "implementd AsBooleanStrict and check in JobRunner"

This reverts commit a1c0c67.

* Reversed the knob

* Add issue templates (#4154)

* Add issue templates

* Update config.yml

* update templates

* Update config.yml

* Added knob to break pipeline unless user opt-in (#4165)

* Added knob to force agent fail if user doesn't opt-in updates on unsupported OS

* Resolved PR comments

---------

Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* Localization update (#4187)

* Added files for onelocbuild

* Update localize-pipeline.yml for Azure Pipelines

* Fixed typo in resource.resx path

* Removed en-US

* Update localize-pipeline.yml for Azure Pipelines

* LEGO: check in for Localization to temporary branch. (#3246)

* LEGO: check in for Localization to temporary branch. (#3268)

* LEGO: check in for Localization to temporary branch. (#3265)

* LEGO: check in for Localization to temporary branch. (#3279)

* Temporary renamed localization folders

* Renamed localization folders

* Set up schedule and notifications for the localization pipeline (#3269)

* [localization] Fixed localization pipeline issue with already localized strings replaced (#3332)

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14646614

Localized file check-in by OneLocBuild Task

* LEGO: check in for Localization to temporary branch. (#3363)

Co-authored-by: csigs <csigs@outlook.com>

* LEGO: check in for Localization to temporary branch. (#3364)

Co-authored-by: csigs <csigs@outlook.com>

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14919763

Localized file check-in by OneLocBuild Task

* Create PR in OneLocBuild task only on third week of sprint (#3374)

* Fix localization pipeline

* Add missed change

* Added option to disable PR creation

* update localize pipeline (#3637)

Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com>

* Localization update (#3612)

* Removing Localize folder

* RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)

* Revert "RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)"

This reverts commit 629921e.

* Revert "Removing Localize folder"

This reverts commit 33cb0ae.

Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com>
Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com>
Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com>

* Removed OneLocBuild folder

* Move notifications about Agent Localization PR from Slack to MS Teams - Part 1 (#3744)

* Include strings from Misc/src/layoutbin to LocProject file (#4021)

* Update LocProject file

* Update CopyOption for LocProject

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)

* Revert "Include strings from Misc/src/layoutbin to LocProject file (#4021)"

This reverts commit 8f5c0c0.

* Revert "Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)" (#4023)

This reverts commit b9c714b.

* Include layoutbin into locproject

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18680899 (#4028)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221104175414554. (#4030)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221105085259861. (#4031)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221106085521421. (#4032)

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18697046 (#4033)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221107143113520. (#4039)

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18706620 (#4044)

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19415018 (#4163)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230223095051399. (#4168)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230224095040576. (#4170)

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476421 (#4179)

* Update loc strings

* Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476736 (#4181)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230305095015478. (#4183)

* Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230306095225210. (#4184)

* Removing Localize and OneLocBuild folder

---------

Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com>
Co-authored-by: csigs <csigs@users.noreply.github.com>
Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com>
Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com>
Co-authored-by: csigs <csigs@outlook.com>
Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com>
Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com>
Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com>
Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com>
Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com>
Co-authored-by: KonstantinTyukalov <v-ktyukalov@microsoft.com>

* Add "Mariner" to list of .NET 6 supporting systems (#4197)

* Cover unsupported OS failure with FF (#4196)

* Cover unsupported OS failure with FF

* Update strings.json

* Update JobRunner.cs

* Update JobRunner.cs

* Add knob to enable fetching net6.json file from GitHub (#4200)

* Added knob "AGENT_ENABLE_FETCHING_NET6_LIST"

* implemented knob EnableFetchingNet6List in function GetNet6SupportedSystems

* Removed RuntimeKnobSource for EnableFetchingNet6List

* removed logic which fails pipeline if there is some error occurred during checking if system supports .NET 6

* fixed typo

* Update autoAssignABTT.yml (#4201)

* Fixed typos in strings.json (#3523)

* Added trimmed user secret to masker dictionary (#4195)

* Add trimmed secret value to masker dictionary

* [Refactor] Update MinSecretLength property. (#4137)

* [Refactor] Update MinSecretLength property.

Removed exception throwing, Added MinSecretLengthLimit public property

* Update test MinValue test

* Update test

* add ability to disable publishing tests metadata (#4209)

* moved logic of checking if system supports .NET 6 to JobExtension (#4207)

* Increase limit for AZP_IGNORE_SECRETS_SHORTER_THAN knob to 6 (#4214)

* Update secret masker length limit

* Make property static

* Decrease limit to 6

* Update tests

* Update tests

* Update comment

---------

Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* Refactor logic to drain queues after each task (#4213)

* move draining queues before calling method "Complete"

* Revert "BUG 1972388: vsbuild task in YAML build pipeline hangs forever in ADO even though the task has already logged completion (#3979)"

This reverts commit b3cf2c0.

* implemented logic to drain web console and timeline queues after each task

* corrected typo

Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>

* renamed variable "drain" to "shouldDrain"

---------

Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>
Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com>

* git lfs version upgraded (#4218)

* Add noderunner.md (#4202)

* Add noderunner.md

* Minor fixes

* Update noderunner.md

* Update noderunner.md

* Resolve comments

* Minor fixes in noderunner doc

* minor fixes

* Bug 2030875 - azure-pipelines-task-lib repo logs passwords in plain text (#4223)

* added logic to unescape vso command in case there is encoded symbols

* Added "TaskCommandHelper" class with method "AddSecret" which checks if decoding percents disabled and adds secret with decoded percents as well

* fixed issue with test SetEndpointAuthParameter

* moved logic to add decoded secret under check if value is empty

* Update AzureInstanceMetadataProvider.cs

* Update agentversion

---------

Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com>
Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com>
Co-authored-by: Steve <hez2010@outlook.com>
Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com>
Co-authored-by: Rick Brown <ricksbrown73@gmail.com>
Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com>
Co-authored-by: Eric van Wijk <eric@van-wijk.com>
Co-authored-by: Sergey Koryshev <sergey.koryshev@akvelon.com>
Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com>
Co-authored-by: Rami <icnocop@users.noreply.github.com>
Co-authored-by: Martin <modermatt@tuta.io>
Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com>
Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com>
Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com>
Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com>
Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com>
Co-authored-by: csigs <csigs@users.noreply.github.com>
Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com>
Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com>
Co-authored-by: csigs <csigs@outlook.com>
Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com>
Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com>
Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com>
Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com>
Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com>
Co-authored-by: Maxim Zaytsev <v-mazayt@microsoft.com>
Co-authored-by: Rob Hensley <hensleyrob@gmail.com>
  • Loading branch information
1 parent b66c6e7 commit 4836fd2
Show file tree
Hide file tree
Showing 19 changed files with 258 additions and 162 deletions.
60 changes: 60 additions & 0 deletions docs/noderunner.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Node 6 support

Agent tasks can be implemented in PowerShell or Node. The agent currently ships with three versions of Node that tasks can target: 6, 10 & 16.

Since Node 6 has long passed out of the upstream maintenance window, and all officially supported tasks are migrated from Node 6 to Node 10, Node 6 soon will be removed from the agent package.
It's also highly recommended to third-party task maintainers migrate tasks to Node 10 or Node 16.

However, to support backward compatibility with the Node 6 tasks we provide self-service methods to install the designated Node runner manually.

## Install Node 6 runner manually

To support the execution of Node 6 tasks agent should be provided with the latest Node 6 version - `6.17.1.0`.

Despite that Node 6 is officially reached the End-of-Life, please, notice that it still can have maintenance updates, so it is required for the agent to get the latest binaries. You can check the currently existing Node versions [here](https://nodejs.org/dist/).

Please use the following steps to manually install the required runner:

1. Download the latest available version of Node 6 binaries for your operating system from the official Node [registry](https://nodejs.org/dist/).

1. Create a folder named `node` under the `agent/externals` directory, and extract downloaded Node binaries into that folder.

You can also use the following commands to install the Node 6 runner via the Powershell or Bash:

Windows:
```powershell
$agentFolder = "" // Specify the Azure DevOps Agent folder, e.g. C:\agents\my_agent
$osArch = "" // Specify the OS architecture, e.g. x64 / x86
New-Item -Type Directory -Path "${agentFolder}\externals\node"
Invoke-WebRequest -Uri "https://nodejs.org/dist/v6.17.1/win-${osArch}/node.exe" -OutFile "${agentFolder}\externals\node\node.exe"
Invoke-WebRequest -Uri "https://nodejs.org/dist/v6.17.1/win-${osArch}/node.lib" -OutFile "${agentFolder}\externals\node\node.lib"
```

Linux / macOS:
```bash
agent_folder="" // Specify the Azure DevOps Agent folder, e.g. /home/user/agents/my_agent
os_platform="" // Specify the OS platform, e.g. linux / darwin
os_arch="" // Specify the OS architecture, e.g. x64 / x86

mkdir "${agent_folder}/externals/node"

wget -O "/tmp/node-v6.17.1-${os_platform}-${os_arch}.tar.gz" "https://nodejs.org/dist/v6.17.1/node-v6.17.1-${os_platform}-${os_arch}.tar.gz"

tar -xvf "/tmp/node-v6.17.1-${os_platform}-${os_arch}.tar.gz" -C "${agent_folder}/externals/node/"
```

## Install Node runner via NodeTaskRunnerInstaller

You can also use the Azure DevOps task [NodeTaskRunnerInstaller](https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/NodeTaskRunnerInstallerV0) to install the required runner version via Azure DevOps CI.

Use the following pipeline task sample to install the latest version of Node 6 runner:

```yaml
- task: NodeTaskRunnerInstaller@0
inputs:
runnerVersion: 6
```
Please, check more details in [NodeTaskRunnerInstaller task]() documentation [TODO: FIX LINK].
7 changes: 7 additions & 0 deletions src/Agent.Sdk/Knob/AgentKnobs.cs
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,13 @@ public class AgentKnobs
new EnvironmentKnobSource("VSTSAGENT_DUMP_JOB_EVENT_LOGS"),
new BuiltInDefaultKnobSource("false"));

public static readonly Knob DisableTestsMetadata = new Knob(
nameof(DisableTestsMetadata),
"If true, publishing tests metadata to evidence store will be disabled.",
new RuntimeKnobSource("AZP_AGENT_DISABLE_TESTS_METADATA"),
new EnvironmentKnobSource("AZP_AGENT_DISABLE_TESTS_METADATA"),
new BuiltInDefaultKnobSource("false"));

// Diag logging
public static readonly Knob AgentDiagLogPath = new Knob(
nameof(AgentDiagLogPath),
Expand Down
2 changes: 2 additions & 0 deletions src/Agent.Sdk/Util/ILoggedSecretMasker.cs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ namespace Agent.Sdk.Util
/// </summary>
public interface ILoggedSecretMasker : ISecretMasker
{
static int MinSecretLengthLimit { get; }

void AddRegex(String pattern, string origin);
void AddValue(String value, string origin);
void AddValueEncoder(ValueEncoder encoder, string origin);
Expand Down
19 changes: 10 additions & 9 deletions src/Agent.Sdk/Util/LoggedSecretMasker.cs
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,6 @@ public class LoggedSecretMasker : ILoggedSecretMasker
private ISecretMasker _secretMasker;
private ITraceWriter _trace;

// We don't allow to skip secrets longer than 4 characters.
private readonly short _maxMinSecretLength = 4;

private void Trace(string msg)
{
this._trace?.Info(msg);
Expand Down Expand Up @@ -73,6 +70,10 @@ public void AddRegex(string pattern, string origin)
AddRegex(pattern);
}

// We don't allow to skip secrets longer than 5 characters.
// Note: the secret that will be ignored is of length n-1.
public static int MinSecretLengthLimit => 6;

public int MinSecretLength
{
get
Expand All @@ -81,14 +82,14 @@ public int MinSecretLength
}
set
{
if (value > _maxMinSecretLength)
if (value > MinSecretLengthLimit)
{
_secretMasker.MinSecretLength = _maxMinSecretLength;

throw new ArgumentException($"Not allowed minimum secret length. Set max value: {_maxMinSecretLength}");
_secretMasker.MinSecretLength = MinSecretLengthLimit;
}
else
{
_secretMasker.MinSecretLength = value;
}

_secretMasker.MinSecretLength = value;
}
}

Expand Down
3 changes: 2 additions & 1 deletion src/Agent.Sdk/Util/PlatformUtil.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Text;
Expand All @@ -16,7 +18,6 @@
using Newtonsoft.Json;
using System.ServiceProcess;
using Agent.Sdk.Util;
using System.Net.Http;

namespace Agent.Sdk
{
Expand Down
29 changes: 16 additions & 13 deletions src/Agent.Worker/ExecutionContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,12 @@ public TaskResult Complete(TaskResult? result = null, string currentOperation =
this.Warning(StringUtil.Loc("TotalThrottlingDelay", TimeSpan.FromMilliseconds(_totalThrottlingDelayInMilliseconds).TotalSeconds));
}

if (!AgentKnobs.DisableDrainQueuesAfterTask.GetValue(this).AsBoolean())
{
_jobServerQueue.ForceDrainWebConsoleQueue = true;
_jobServerQueue.ForceDrainTimelineQueue = true;
}

_record.CurrentOperation = currentOperation ?? _record.CurrentOperation;
_record.ResultCode = resultCode ?? _record.ResultCode;
_record.FinishTime = DateTime.UtcNow;
Expand Down Expand Up @@ -518,26 +524,23 @@ public void InitializeJob(Pipelines.AgentJobRequestMessage message, Cancellation
// Prepend Path
PrependPath = new List<string>();

// Docker (JobContainer)
string imageName = Variables.Get("_PREVIEW_VSTS_DOCKER_IMAGE");
if (string.IsNullOrEmpty(imageName))
{
imageName = Environment.GetEnvironmentVariable("_PREVIEW_VSTS_DOCKER_IMAGE");
}

var minSecretLen = AgentKnobs.MaskedSecretMinLength.GetValue(this).AsInt();
HostContext.SecretMasker.MinSecretLength = minSecretLen;

try
if (HostContext.SecretMasker.MinSecretLength < minSecretLen)
{
this.HostContext.SecretMasker.MinSecretLength = minSecretLen;
warnings.Add(StringUtil.Loc("MinSecretsLengtLimitWarning", HostContext.SecretMasker.MinSecretLength));
}
catch (ArgumentException ex)

HostContext.SecretMasker.RemoveShortSecretsFromDictionary();

// Docker (JobContainer)
string imageName = Variables.Get("_PREVIEW_VSTS_DOCKER_IMAGE");
if (string.IsNullOrEmpty(imageName))
{
warnings.Add(ex.Message);
imageName = Environment.GetEnvironmentVariable("_PREVIEW_VSTS_DOCKER_IMAGE");
}

this.HostContext.SecretMasker.RemoveShortSecretsFromDictionary();

Containers = new List<ContainerInfo>();
_defaultStepTarget = null;
_currentStepTarget = null;
Expand Down
50 changes: 50 additions & 0 deletions src/Agent.Worker/JobExtension.cs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,52 @@ public async Task<List<IStep>> InitializeJob(IExecutionContext jobContext, Pipel
context.Start();
context.Section(StringUtil.Loc("StepStarting", StringUtil.Loc("InitializeJob")));

// Check if a system supports .NET 6
PackageVersion agentVersion = new PackageVersion(BuildConstants.AgentPackage.Version);
if (agentVersion.Major < 3)
{
try
{
Trace.Verbose("Checking if your system supports .NET 6");

string systemId = PlatformUtil.GetSystemId();
SystemVersion systemVersion = PlatformUtil.GetSystemVersion();
string notSupportNet6Message = null;

if (await PlatformUtil.DoesSystemPersistsInNet6Whitelist())
{
// Check version of the system
if (!await PlatformUtil.IsNet6Supported())
{
notSupportNet6Message = $"The operating system the agent is running on is \"{systemId}\" ({systemVersion}), which will not be supported by the .NET 6 based v3 agent. Please upgrade the operating system of this host to ensure compatibility with the v3 agent. See https://aka.ms/azdo-pipeline-agent-version";
if (AgentKnobs.AgentFailOnIncompatibleOS.GetValue(jobContext).AsBoolean() &&
!AgentKnobs.AcknowledgeNoUpdates.GetValue(jobContext).AsBoolean())
{
throw new UnsupportedOsException(StringUtil.Loc("FailAgentOnUnsupportedOs"));
}
}
}
else
{
notSupportNet6Message = $"The operating system the agent is running on is \"{systemId}\" ({systemVersion}), which has not been tested with the .NET 6 based v3 agent. The v2 agent wil not automatically upgrade to the v3 agent. You can manually download the .NET 6 based v3 agent from https://github.com/microsoft/azure-pipelines-agent/releases. See https://aka.ms/azdo-pipeline-agent-version";
}

if (!string.IsNullOrWhiteSpace(notSupportNet6Message))
{
context.Warning(notSupportNet6Message);
}
}
catch (UnsupportedOsException)
{
throw;
}
catch (Exception ex)
{
Trace.Error($"Error has occurred while checking if system supports .NET 6: {ex}");
context.Warning(ex.Message);
}
}

// Set agent version variable.
context.SetVariable(Constants.Variables.Agent.Version, BuildConstants.AgentPackage.Version);
context.Output(StringUtil.Loc("AgentNameLog", context.Variables.Get(Constants.Variables.Agent.Name)));
Expand Down Expand Up @@ -592,4 +638,8 @@ private void OutputSetupInfo(IExecutionContext context)
}
}
}

public class UnsupportedOsException : Exception {
public UnsupportedOsException(string message) : base(message) { }
}
}
46 changes: 0 additions & 46 deletions src/Agent.Worker/JobRunner.cs
Original file line number Diff line number Diff line change
Expand Up @@ -103,52 +103,6 @@ public async Task<TaskResult> RunAsync(Pipelines.AgentJobRequestMessage message,
jobContext = HostContext.CreateService<IExecutionContext>();
jobContext.InitializeJob(message, jobRequestCancellationToken);

// Check if a system supports .NET 6
PackageVersion agentVersion = new PackageVersion(BuildConstants.AgentPackage.Version);
if (agentVersion.Major < 3)
{
try
{
Trace.Verbose("Checking if your system supports .NET 6");

string systemId = PlatformUtil.GetSystemId();
SystemVersion systemVersion = PlatformUtil.GetSystemVersion();
string notSupportNet6Message = null;

if (await PlatformUtil.DoesSystemPersistsInNet6Whitelist())
{
// Check version of the system
if (!await PlatformUtil.IsNet6Supported())
{
notSupportNet6Message = $"The operating system the agent is running on is \"{systemId}\" ({systemVersion}), which will not be supported by the .NET 6 based v3 agent. Please upgrade the operating system of this host to ensure compatibility with the v3 agent. See https://aka.ms/azdo-pipeline-agent-version";
if (AgentKnobs.AgentFailOnIncompatibleOS.GetValue(jobContext).AsBoolean() &&
!AgentKnobs.AcknowledgeNoUpdates.GetValue(jobContext).AsBoolean())
{
jobContext.Error(StringUtil.Loc("FailAgentOnUnsupportedOs"));
return await CompleteJobAsync(jobServer, jobContext, message, TaskResult.Failed);
}
}
}
else
{
notSupportNet6Message = $"The operating system the agent is running on is \"{systemId}\" ({systemVersion}), which has not been tested with the .NET 6 based v3 agent. The v2 agent wil not automatically upgrade to the v3 agent. You can manually download the .NET 6 based v3 agent from https://github.com/microsoft/azure-pipelines-agent/releases. See https://aka.ms/azdo-pipeline-agent-version";
}

if (!string.IsNullOrWhiteSpace(notSupportNet6Message))
{


jobContext.AddIssue(new Issue() { Type = IssueType.Warning, Message = notSupportNet6Message });
}
}
catch (Exception ex)
{
jobContext.Warning($"Error has occurred while checking if system supports .NET 6: {ex.Message}");
Trace.Error($"Error has occurred while checking if system supports .NET 6: {ex}");

}
}

Trace.Info("Starting the job execution context.");
jobContext.Start();
jobContext.Section(StringUtil.Loc("StepStarting", message.JobDisplayName));
Expand Down
20 changes: 0 additions & 20 deletions src/Agent.Worker/StepsRunner.cs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
using System.Threading.Tasks;
using Microsoft.TeamFoundation.DistributedTask.Expressions;
using Pipelines = Microsoft.TeamFoundation.DistributedTask.Pipelines;
using Microsoft.VisualStudio.Services.CircuitBreaker;
using Agent.Sdk.Knob;

namespace Microsoft.VisualStudio.Services.Agent.Worker
Expand All @@ -35,10 +34,6 @@ public interface IStepsRunner : IAgentService

public sealed class StepsRunner : AgentService, IStepsRunner
{
private IJobServerQueue _jobServerQueue;

private IJobServerQueue JobServerQueue => _jobServerQueue ??= HostContext.GetService<IJobServerQueue>();

// StepsRunner should never throw exception to caller
public async Task RunAsync(IExecutionContext jobContext, IList<IStep> steps)
{
Expand Down Expand Up @@ -309,21 +304,6 @@ private async Task RunStepAsync(IStep step, CancellationToken jobCancellationTok
// Complete the step context.
step.ExecutionContext.Section(StringUtil.Loc("StepFinishing", step.DisplayName));
step.ExecutionContext.Complete();

if (!AgentKnobs.DisableDrainQueuesAfterTask.GetValue(step.ExecutionContext).AsBoolean())
{
try
{
// We need to drain the queues after a task just in case if
// there are a lot of items since it can cause some UI hangs.
await JobServerQueue.DrainQueues();
}
catch (Exception ex)
{
Trace.Error($"Error has occurred while draining queues, it can cause some UI glitches but it doesn't affect a pipeline execution itself: {ex}");
step.ExecutionContext.Error(ex);
}
}
}

private async Task SwitchToUtf8Codepage(IStep step)
Expand Down
Loading

0 comments on commit 4836fd2

Please sign in to comment.