Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upd dotnet6 agent before merge with master #4252

Merged
merged 51 commits into from
Apr 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
8dd8075
Catch updating exception (#4082)
Roman-Shchukin Dec 12, 2022
b12a109
Node16 version updated to 16.17.1" (#4084)
Roman-Shchukin Dec 12, 2022
2ed3a2e
Bump ubuntu image version to 20.04 (#4079)
KonstantinTyukalov Dec 12, 2022
6d0bf0d
Prevent creating host container network (#3788)
hez2010 Dec 12, 2022
27e3d4e
Allow env.sh to accept additional env var names (#3339)
ricksbrown Dec 12, 2022
7e2a3a1
Revert "Fix execution of scripts in variables (#4016)" (#4088)
LiliaSabitova Dec 13, 2022
0f3401a
Add Ubuntu 18.04 (#3943)
geekzter Dec 15, 2022
82f092c
Make agent update list of systems supporting .NET 6 periodically (#4081)
sergey-koryshev Dec 16, 2022
c42fd0c
Retries to lock Services database (#4085)
DenisRumyantsev Dec 16, 2022
e76a890
Fixed name of DockerAdditionalNetworkOptions Knob (#3888)
icnocop Dec 16, 2022
df73f63
Format agent source code (#4101)
KonstantinTyukalov Dec 26, 2022
24d4035
Fix SonarQube's "'Any()' should be used to test for emptiness" / Code…
marodev Jan 4, 2023
866c696
Add script to predict v2->v3 upgrade compatibility (#4092)
geekzter Jan 5, 2023
dbdb0bf
Added strict boolean knob values (#4103)
KonstantinTyukalov Jan 16, 2023
96a6922
update build-job (#4110)
ivanduplenskikh Jan 17, 2023
92e289b
fix dotnet host search (#4124)
ivanduplenskikh Jan 23, 2023
18080a2
Add option to skip rhel6 in pipeline and ci (#4121)
Roman-Shchukin Jan 23, 2023
f310843
Update dotnet to 3.1.32 (#4112)
kirill-ivlev Jan 23, 2023
1055f4c
DockerLogin retries (#4100)
ivanduplenskikh Jan 24, 2023
d37241e
Add the exact operation system version and environment to the telemet…
LiliaSabitova Jan 24, 2023
d503197
Mingit version upgraded to 2.39.1 (#4126)
ismayilov-ismayil Jan 26, 2023
37bb9cf
Fix agent lint errors (#4117)
KonstantinTyukalov Feb 1, 2023
4b66e90
Revert "Allow env.sh to accept additional env var names (#3339)" (#4143)
KonstantinTyukalov Feb 2, 2023
c21b0c6
Bump azure pipelines task lib to 4.2.0 (#4155)
kirill-ivlev Feb 14, 2023
1781c92
Add warning for deprecated runners (#4162)
kirill-ivlev Feb 23, 2023
df3255d
FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container …
LiliaSabitova Feb 23, 2023
c0bd968
Added legacy lib source for support on Ubuntu 22.04 (#4161)
ismayilov-ismayil Mar 2, 2023
84a6f86
Moved logic to drain queues after each task under knob "AGENT_DRAIN_Q…
sergey-koryshev Mar 7, 2023
b4dd146
Add issue templates (#4154)
kirill-ivlev Mar 7, 2023
1bc1fd1
Added knob to break pipeline unless user opt-in (#4165)
ismayilov-ismayil Mar 7, 2023
39797c2
Localization update (#4187)
vmapetr Mar 9, 2023
443adfd
Add "Mariner" to list of .NET 6 supporting systems (#4197)
sergey-koryshev Mar 13, 2023
3936c12
Cover unsupported OS failure with FF (#4196)
kirill-ivlev Mar 13, 2023
ad2467b
Add knob to enable fetching net6.json file from GitHub (#4200)
sergey-koryshev Mar 13, 2023
55be4b7
Update autoAssignABTT.yml (#4201)
max-zaytsev Mar 15, 2023
fa4e296
Fixed typos in strings.json (#3523)
hensleyrob Mar 16, 2023
8114868
Added trimmed user secret to masker dictionary (#4195)
KonstantinTyukalov Mar 20, 2023
cc4b2b0
[Refactor] Update MinSecretLength property. (#4137)
KonstantinTyukalov Mar 23, 2023
7f7c16b
add ability to disable publishing tests metadata (#4209)
DenisRumyantsev Mar 24, 2023
e5c6c57
moved logic of checking if system supports .NET 6 to JobExtension (#4…
sergey-koryshev Mar 27, 2023
4e4f00c
Increase limit for AZP_IGNORE_SECRETS_SHORTER_THAN knob to 6 (#4214)
KonstantinTyukalov Mar 28, 2023
857cbef
Refactor logic to drain queues after each task (#4213)
sergey-koryshev Apr 5, 2023
61391d3
git lfs version upgraded (#4218)
ismayilov-ismayil Apr 6, 2023
6f8b10d
Add noderunner.md (#4202)
vmapetr Apr 6, 2023
c4e9eec
Bug 2030875 - azure-pipelines-task-lib repo logs passwords in plain t…
sergey-koryshev Apr 11, 2023
240cd92
Fix: Remove invalid chars from filename (#3831)
mihaiandreiratoiu Apr 18, 2023
e54db88
Add force of _tasks creation by knob (#4236)
KonstantinTyukalov Apr 21, 2023
7778d90
Upgrade vss api netcore version, has BuildXL changes (#4220)
fadnavistanmay Apr 24, 2023
d7b4f6c
Merge branch 'master' into upd-dotnet6-agent-before-merge-with-master
kirill-ivlev Apr 26, 2023
4ac5b78
update packages
kirill-ivlev Apr 26, 2023
0b8b5ed
Update agentversion
kirill-ivlev Apr 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/Agent.Listener/Agent.Listener.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

<PropertyGroup>
<OutputType>Exe</OutputType>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
Expand Down
1 change: 1 addition & 0 deletions src/Agent.PluginHost/Agent.PluginHost.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

<PropertyGroup>
<OutputType>Exe</OutputType>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
Expand Down
1 change: 1 addition & 0 deletions src/Agent.Plugins/Agent.Plugins.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

<PropertyGroup>
<OutputType>Library</OutputType>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
Expand Down
4 changes: 2 additions & 2 deletions src/Agent.Sdk/Agent.Sdk.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@

<PropertyGroup>
<OutputType>Library</OutputType>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.9.8" Condition="$(CodeAnalysis)=='true'" />
<PackageReference Include="Microsoft.Win32.Registry" Version="4.7.0" />
<PackageReference Include="System.Security.Cryptography.ProtectedData" Version="4.7.0" />
<PackageReference Include="System.ServiceProcess.ServiceController" Version="4.4.0" />
<PackageReference Include="System.Security.Principal.Windows" Version="6.0.0-preview.5.21301.5" />
<PackageReference Include="System.Text.Encoding.CodePages" Version="4.4.0" />
<PackageReference Include="System.Security.Cryptography.Cng" Version="4.5.0" />
<PackageReference Include="vss-api-netcore" Version="$(VssApiVersion)" />
</ItemGroup>
</Project>
7 changes: 7 additions & 0 deletions src/Agent.Sdk/Knob/AgentKnobs.cs
Original file line number Diff line number Diff line change
Expand Up @@ -451,5 +451,12 @@ public class AgentKnobs
"Forces the agent to fetch list of .NET 6 supporting systems from server",
new EnvironmentKnobSource("AGENT_ENABLE_FETCHING_NET6_LIST"),
new BuiltInDefaultKnobSource("false"));

public static readonly Knob ForceCreateTasksDirectory = new Knob(
nameof(ForceCreateTasksDirectory),
"Forces the agent to create _tasks folder for tasks.",
new RuntimeKnobSource("AGENT_FORCE_CREATE_TASKS_DIRECTORY"),
new EnvironmentKnobSource("AGENT_FORCE_CREATE_TASKS_DIRECTORY"),
new BuiltInDefaultKnobSource("false"));
}
}
4 changes: 2 additions & 2 deletions src/Agent.Sdk/Util/PlatformUtil.cs
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@ public static bool DetectDockerContainer()
}
}
}
catch (Exception ex)
catch (Exception)
{
// Logging exception will be handled by JobRunner
throw;
Expand All @@ -411,7 +411,7 @@ public static bool DetectAzureVM()
if (metadataProvider.HasMetadata())
isAzureVM = true;
}
catch (Exception ex)
catch (Exception)
{
// Logging exception will be handled by JobRunner
throw;
Expand Down
2 changes: 1 addition & 1 deletion src/Agent.Worker/Agent.Worker.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

<PropertyGroup>
<OutputType>Exe</OutputType>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
Expand All @@ -14,7 +15,6 @@
<ItemGroup>
<PackageReference Include="Azure.Identity" Version="1.6.1" />
<PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.9.8" Condition="$(CodeAnalysis)=='true'" />
<PackageReference Include="System.Security.Cryptography.ProtectedData" Version="4.7.0" />
<PackageReference Include="System.ServiceProcess.ServiceController" Version="4.4.0" />
<PackageReference Include="vss-api-netcore" Version="$(VssApiVersion)" />
</ItemGroup>
Expand Down
6 changes: 5 additions & 1 deletion src/Agent.Worker/ExecutionContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
using System.Collections.Specialized;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using System.Web;
Expand Down Expand Up @@ -262,8 +263,11 @@ public void Start(string currentOperation = null)
{
var buildLogsJobFolder = Path.Combine(_buildLogsFolderPath, _mainTimelineId.ToString());
Directory.CreateDirectory(buildLogsJobFolder);
string pattern = new string(Path.GetInvalidFileNameChars()) + new string(Path.GetInvalidPathChars());
Regex regex = new Regex(string.Format("[{0}]", Regex.Escape(pattern)));
var recordName = regex.Replace(_record.Name, string.Empty);

_buildLogsFile = Path.Combine(buildLogsJobFolder, $"{_record.Name}-{_record.Id.ToString()}.log");
_buildLogsFile = Path.Combine(buildLogsJobFolder, $"{recordName}-{_record.Id.ToString()}.log");
_buildLogsData = new FileStream(_buildLogsFile, FileMode.CreateNew);
_buildLogsWriter = new StreamWriter(_buildLogsData, System.Text.Encoding.UTF8);

Expand Down
19 changes: 18 additions & 1 deletion src/Agent.Worker/JobExtension.cs
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,22 @@ public async Task<List<IStep>> InitializeJob(IExecutionContext jobContext, Pipel
Trace.Info($"Run initial step from extension {this.GetType().Name}.");
InitializeJobExtension(context, message?.Steps, message?.Workspace);

if (AgentKnobs.ForceCreateTasksDirectory.GetValue(context).AsBoolean())
{
var tasksDir = HostContext.GetDirectory(WellKnownDirectory.Tasks);
try
{
Trace.Info($"Pre-creating {tasksDir} directory");
Directory.CreateDirectory(tasksDir);
IOUtil.ValidateExecutePermission(tasksDir);
}
catch (Exception ex)
{
Trace.Error(ex);
context.Error(ex);
}
}

// Download tasks if not already in the cache
Trace.Info("Downloading task definitions.");
var taskManager = HostContext.GetService<ITaskManager>();
Expand Down Expand Up @@ -639,7 +655,8 @@ private void OutputSetupInfo(IExecutionContext context)
}
}

public class UnsupportedOsException : Exception {
public class UnsupportedOsException : Exception
{
public UnsupportedOsException(string message) : base(message) { }
}
}
2 changes: 1 addition & 1 deletion src/Common.props
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
<OSPlatform>OS_UNKNOWN</OSPlatform>
<OSArchitecture>ARCH_UNKNOWN</OSArchitecture>
<DebugConstant></DebugConstant>
<VssApiVersion>0.5.181-private</VssApiVersion>
<VssApiVersion>0.5.183-private</VssApiVersion>
<CodeAnalysis>$(CodeAnalysis)</CodeAnalysis>
<InvariantGlobalization>false</InvariantGlobalization>
</PropertyGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ public DedupIdentifier DedupId
{
get
{
return Node.GetDedupIdentifier(HashType.Dedup64K);
return Node.GetDedupIdentifier();
}
}
public bool Success { get; set; }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ public static class BlobStoreUtils
foreach (var file in fileNodes.Where(x => x.Success))
{
// ChunkHelper uses 64k block default size
var dedupId = file.Node.GetDedupIdentifier(HashType.Dedup64K);
var dedupId = file.Node.GetDedupIdentifier();
fileDedupIds[dedupId] = file.Path;
}

Expand Down Expand Up @@ -180,7 +180,7 @@ private static DedupNode CreateNodeToUpload(IEnumerable<DedupNode> nodes)
var chunk = await ChunkerHelper.CreateFromFileAsync(FileSystem.Instance, itemPath, cancellationToken, false);
var rootNode = new DedupNode(new[] { chunk });
// ChunkHelper uses 64k block default size
var dedupId = rootNode.GetDedupIdentifier(HashType.Dedup64K);
var dedupId = rootNode.GetDedupIdentifier();

// Setup upload session to keep file for at mimimum one day
// Blobs will need to be associated with the server with an ID ref otherwise they will be
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

<PropertyGroup>
<OutputType>Library</OutputType>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
Expand All @@ -13,8 +14,7 @@
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.9.8" Condition="$(CodeAnalysis)=='true'" />
<PackageReference Include="Microsoft.Win32.Registry" Version="4.7.0" />
<PackageReference Include="System.Security.Cryptography.Cng" Version="4.5.0" />
<PackageReference Include="System.Security.Cryptography.ProtectedData" Version="4.7.0" />
<PackageReference Include="System.Security.Principal.Windows" Version="6.0.0-preview.5.21301.5" />
<PackageReference Include="System.Text.Encoding.CodePages" Version="4.4.0" />
<PackageReference Include="vss-api-netcore" Version="$(VssApiVersion)" />
</ItemGroup>
Expand Down
2 changes: 1 addition & 1 deletion src/Test/L1/Mock/FakeJobServer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ public Task<BlobIdentifierWithBlocks> UploadLogToBlobStore(Stream blob, string h
UploadedAttachmentBlobFiles.Add(itemPath);
var chunk = await ChunkerHelper.CreateFromFileAsync(FileSystem.Instance, itemPath, cancellationToken, false);
var rootNode = new DedupNode(new[] { chunk });
var dedupId = rootNode.GetDedupIdentifier(HashType.Dedup64K);
var dedupId = rootNode.GetDedupIdentifier();

return (dedupId, rootNode.TransitiveContentBytes);
}
Expand Down
5 changes: 4 additions & 1 deletion src/Test/Test.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,16 @@
<ProjectReference Include="..\Agent.Sdk\Agent.Sdk.csproj" />
</ItemGroup>

<PropertyGroup>
<SuppressTfmSupportBuildWarnings>true</SuppressTfmSupportBuildWarnings>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="azuredevops-testresultparser" Version="1.0.2" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0" />
<PackageReference Include="Microsoft.CodeCoverage" Version="16.4.0" />
<PackageReference Include="Microsoft.TestPlatform.TestHost" Version="16.4.0" GeneratePathProperty="true" />
<PackageReference Include="Moq" Version="4.6.36-alpha" />
<PackageReference Include="System.Buffers" Version="4.3.0" />
<PackageReference Include="System.Reflection.TypeExtensions" Version="4.4.0" />
<PackageReference Include="System.Threading.ThreadPool" Version="4.3.0" />
<PackageReference Include="vss-api-netcore" Version="$(VssApiVersion)" />
Expand Down
2 changes: 1 addition & 1 deletion src/agentversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.220.0
2.999.999