Skip to content
This repository was archived by the owner on Nov 1, 2023. It is now read-only.

Commit ec5cec2

Browse files
chkeitachkeita
authored
Merge branch 'main' into bug_fix
2 parents 887a057 + 7091c54 commit ec5cec2

File tree

18 files changed

+467
-89
lines changed

18 files changed

+467
-89
lines changed

‎CHANGELOG.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,47 @@ All notable changes to this project will be documented in this file.
44
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
55
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

7+
## 6.0.0
8+
9+
## BREAKING CHANGES
10+
11+
### Manual Deployment Step
12+
When upgrading from version 5.20 a manual step is required. Before deploying 6.0 delete both Azure App Functions and the Azure App Service plan before upgrading. This is required because we have migrated the service from `python` to `C#`.
13+
14+
After deployment, there will be two App Functions deployed, one with the name of the deployment and a second one with the same name and a `-net` suffix. This is a temporary situation and the `-net` app function will be removed in a following release.
15+
16+
If you have not used the deployment parameters to deploy C# functions in 5.20, you can manually delete the `-net` app function immediately. Deploying the C# functions was not a default action in 5.20, for most deployments deleting the `-net` app function immediately is ok.
17+
18+
### Deprecation of jinja templates
19+
With this release we are moving from jinja templates to [scriban](https://github.com/scriban/scriban) templates. See the documentation for [scriban here](https://github.com/scriban/scriban/tree/master/doc).
20+
21+
Version 6.0 will convert jinja templates on-the-fly for a short period of time. We do **_not_** guarantee that this will be successful for all jinja template options. These on-the-fly conversions are not persisted in the notifications table in this release. They will be in a following release. This will allow time for conversions of templates that are not handled by the current automatic conversion process.
22+
23+
### CLI
24+
The default value for the `--container_type` parameter to the `container` command has been removed. The `container_type` parameter is still required for the command. This change removes the ambiguity of the container information being returned.
25+
26+
### Added
27+
* Agent: Added `machine_id` a parameter of the agent config. [#2649](https://github.com/microsoft/onefuzz/pull/2649)
28+
* Agent: Pass the `machine_id` from the Agent to the Task. [#2662](https://github.com/microsoft/onefuzz/pull/2662)
29+
### Changed
30+
* Service: Deployment enables refactored C# App Function. [#2650](https://github.com/microsoft/onefuzz/pull/2650)
31+
* CLI: Attempt to use broker or browser login instead of device flow for authentication. Canceling the attempt with `Ctrl-C` will fall back to using the device flow. [#2612](https://github.com/microsoft/onefuzz/pull/2612)
32+
* Service: Update to .NET 7. [#2615](https://github.com/microsoft/onefuzz/pull/2615)
33+
* Service: Make Proxy `TelemetryKey` optional. [#2619](https://github.com/microsoft/onefuzz/pull/2619)
34+
* Service: Update OMI to 1.6.10.2 on Ubuntu VMs. [#2629](https://github.com/microsoft/onefuzz/pull/2629)
35+
* CLI: Make the `--container_type` parameter required when using the `containers` command. [#2631](https://github.com/microsoft/onefuzz/pull/2631)
36+
* Service: Improve logging around notification failures. [#2653](https://github.com/microsoft/onefuzz/pull/2653)
37+
* Service: Standardize HTTP Error Results. Better Rejection Message When Parsing Validated Strings. [#2663](https://github.com/microsoft/onefuzz/pull/2663)
38+
* CLI: Retry on Connection Errors when acquiring auth token. [#2668](https://github.com/microsoft/onefuzz/pull/2668)
39+
### Fixed
40+
* Service: Notification Template `targetUrl` parameter fix. Only use the filename instead of the absolute path in the URL. The makes the links created in ADO bugs work as expected. [#2625](https://github.com/microsoft/onefuzz/pull/2625)
41+
* CLI: Fixed SignalR client code not reading responses correctly. [#2626](https://github.com/microsoft/onefuzz/pull/2626)
42+
* Service: Fix a logic bug in the notification hook. [#2627](https://github.com/microsoft/onefuzz/pull/2627)
43+
* Service: Bug fixes related to the unmanaged nodes (an unreleased feature). [#2632](https://github.com/microsoft/onefuzz/pull/2632)
44+
* Service: Fix invocation of `functionapp` in the deployment script. Where the wrong value/parameter pair were used. [#2645](https://github.com/microsoft/onefuzz/pull/2645)
45+
* Service: Fixing .NET crash report no-repro. [#2642](https://github.com/microsoft/onefuzz/pull/2642)
46+
* Service: Check Extensions Status Before Transitioning to `running` state during VM setup. [#2667](https://github.com/microsoft/onefuzz/pull/2667)
47+
748
## 5.20.0
849
### Added
950
* Service: Added endpoint to download agent binaries to support the unmanaged node scenario. [#2600](https://github.com/microsoft/onefuzz/pull/2600)

‎CURRENT_VERSION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
5.20.0
1+
6.0.0

‎src/ApiService/ApiService/ApiService.csproj‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@
1111
<PackageReference Include="Microsoft.Azure.Functions.Worker.ApplicationInsights" Version="1.0.0-preview3" />
1212
<PackageReference Include="Semver" Version="2.1.0" />
1313
<PackageReference Include="Azure.Security.KeyVault.Secrets" Version="4.3.0" />
14+
<PackageReference Include="Microsoft.Extensions.Configuration.AzureAppConfiguration" Version="5.1.0" />
15+
<PackageReference Include="Microsoft.FeatureManagement" Version="2.5.1" />
16+
<PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.1.0" />
1417
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Storage" Version="5.0.0" />
1518
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.EventGrid" Version="2.1.0" />
1619
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.0.13" />

‎src/ApiService/ApiService/FeatureFlags.cs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
namespace Microsoft.OneFuzz.Service;
2+
3+
public static class FeatureFlagConstants {
4+
public const string EnableScribanOnly = "EnableScribanOnly";
5+
}

‎src/ApiService/ApiService/Functions/AgentCanSchedule.cs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ public class AgentCanSchedule {
88
private readonly IEndpointAuthorization _auth;
99
private readonly IOnefuzzContext _context;
1010

11+
1112
public AgentCanSchedule(ILogTracer log, IEndpointAuthorization auth, IOnefuzzContext context) {
1213
_log = log;
1314
_auth = auth;

‎src/ApiService/ApiService/Program.cs‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,10 @@
1313
using Microsoft.ApplicationInsights.DependencyCollector;
1414
using Microsoft.Azure.Functions.Worker;
1515
using Microsoft.Azure.Functions.Worker.Middleware;
16+
using Microsoft.Extensions.Configuration;
1617
using Microsoft.Extensions.DependencyInjection;
1718
using Microsoft.Extensions.Hosting;
19+
using Microsoft.FeatureManagement;
1820
using Microsoft.Graph;
1921
using Microsoft.OneFuzz.Service.OneFuzzLib.Orm;
2022

@@ -47,13 +49,22 @@ public static async Async.Task Main() {
4749

4850
using var host =
4951
new HostBuilder()
52+
.ConfigureAppConfiguration(builder => {
53+
var _ = builder.AddAzureAppConfiguration(options => {
54+
var _ = options
55+
.Connect(new Uri(configuration.AppConfigurationEndpoint!), new DefaultAzureCredential())
56+
.UseFeatureFlags(ffOptions => ffOptions.CacheExpirationInterval = TimeSpan.FromMinutes(1));
57+
});
58+
})
5059
.ConfigureFunctionsWorkerDefaults(builder => {
5160
builder.UseMiddleware<LoggingMiddleware>();
5261
builder.AddApplicationInsights(options => {
5362
options.ConnectionString = $"InstrumentationKey={configuration.ApplicationInsightsInstrumentationKey}";
5463
});
5564
})
5665
.ConfigureServices((context, services) => {
66+
services.AddAzureAppConfiguration();
67+
_ = services.AddFeatureManagement();
5768
services.Configure<JsonSerializerOptions>(options => {
5869
options = EntityConverter.GetJsonSerializerOptions();
5970
});

‎src/ApiService/ApiService/ServiceConfiguration.cs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ public interface IServiceConfig {
1616

1717
public string? ApplicationInsightsAppId { get; }
1818
public string? ApplicationInsightsInstrumentationKey { get; }
19+
public string? AppConfigurationEndpoint { get; }
1920
public string? AzureSignalRConnectionString { get; }
2021
public string? AzureSignalRServiceTransportType { get; }
2122

@@ -82,6 +83,8 @@ public ServiceConfiguration() {
8283
public string? ApplicationInsightsAppId => GetEnv("APPINSIGHTS_APPID");
8384
public string? ApplicationInsightsInstrumentationKey => GetEnv("APPINSIGHTS_INSTRUMENTATIONKEY");
8485

86+
public string? AppConfigurationEndpoint => GetEnv("APPCONFIGURATION_ENDPOINT");
87+
8588
public string? AzureSignalRConnectionString => GetEnv("AzureSignalRConnectionString");
8689
public string? AzureSignalRServiceTransportType => GetEnv("AzureSignalRServiceTransportType");
8790

‎src/ApiService/ApiService/onefuzzlib/OnefuzzContext.cs‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
11
using Microsoft.OneFuzz.Service.OneFuzzLib.Orm;
22

3+
34
namespace Microsoft.OneFuzz.Service;
45

6+
using Microsoft.Extensions.Configuration.AzureAppConfiguration;
57
using Microsoft.Extensions.DependencyInjection;
8+
using Microsoft.FeatureManagement;
69

710
public interface IOnefuzzContext {
811
IAutoScaleOperations AutoScaleOperations { get; }
@@ -46,6 +49,9 @@ public interface IOnefuzzContext {
4649
ITeams Teams { get; }
4750
IGithubIssues GithubIssues { get; }
4851
IAdo Ado { get; }
52+
53+
IFeatureManagerSnapshot FeatureManagerSnapshot { get; }
54+
IConfigurationRefresher ConfigurationRefresher { get; }
4955
}
5056

5157
public class OnefuzzContext : IOnefuzzContext {
@@ -95,4 +101,8 @@ public OnefuzzContext(IServiceProvider serviceProvider) {
95101
public ITeams Teams => _serviceProvider.GetRequiredService<ITeams>();
96102
public IGithubIssues GithubIssues => _serviceProvider.GetRequiredService<IGithubIssues>();
97103
public IAdo Ado => _serviceProvider.GetRequiredService<IAdo>();
104+
105+
public IFeatureManagerSnapshot FeatureManagerSnapshot => _serviceProvider.GetRequiredService<IFeatureManagerSnapshot>();
106+
107+
public IConfigurationRefresher ConfigurationRefresher => _serviceProvider.GetRequiredService<IConfigurationRefresherProvider>().Refreshers.First();
98108
}

‎src/ApiService/ApiService/onefuzzlib/notifications/NotificationsBase.cs‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ protected class Renderer {
3434
private readonly Uri _targetUrl;
3535
private readonly Uri _inputUrl;
3636
private readonly Uri _reportUrl;
37+
private readonly bool _scribanOnly;
3738

3839
public static async Async.Task<Renderer> ConstructRenderer(
3940
IOnefuzzContext context,
@@ -66,6 +67,9 @@ public static async Async.Task<Renderer> ConstructRenderer(
6667
inputUrl = new Uri(context.Containers.AuthDownloadUrl(report.InputBlob.Container, report.InputBlob.Name));
6768
}
6869

70+
await context.ConfigurationRefresher.TryRefreshAsync().IgnoreResult();
71+
var scribanOnly = await context.FeatureManagerSnapshot.IsEnabledAsync(FeatureFlagConstants.EnableScribanOnly);
72+
6973
return new Renderer(
7074
container,
7175
filename,
@@ -74,7 +78,8 @@ public static async Async.Task<Renderer> ConstructRenderer(
7478
checkedJob,
7579
targetUrl,
7680
inputUrl!, // TODO: incorrect
77-
reportUrl);
81+
reportUrl,
82+
scribanOnly);
7883
}
7984
public Renderer(
8085
Container container,
@@ -84,7 +89,8 @@ public Renderer(
8489
Job job,
8590
Uri targetUrl,
8691
Uri inputUrl,
87-
Uri reportUrl) {
92+
Uri reportUrl,
93+
bool scribanOnly) {
8894
_report = report;
8995
_container = container;
9096
_filename = filename;
@@ -93,13 +99,17 @@ public Renderer(
9399
_reportUrl = reportUrl;
94100
_targetUrl = targetUrl;
95101
_inputUrl = inputUrl;
102+
_scribanOnly = scribanOnly;
96103
}
97104

98105
// TODO: This function is fallible but the python
99106
// implementation doesn't have that so I'm trying to match it.
100107
// We should probably propagate any errors up
101108
public async Async.Task<string> Render(string templateString, Uri instanceUrl) {
102-
templateString = JinjaTemplateAdapter.IsJinjaTemplate(templateString) ? JinjaTemplateAdapter.AdaptForScriban(templateString) : templateString;
109+
if (!_scribanOnly && JinjaTemplateAdapter.IsJinjaTemplate(templateString)) {
110+
templateString = JinjaTemplateAdapter.AdaptForScriban(templateString);
111+
}
112+
103113
var template = Template.Parse(templateString);
104114
if (template != null) {
105115
return await template.RenderAsync(new {

‎src/ApiService/ApiService/packages.lock.json‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,16 @@
157157
"resolved": "0.12.2",
158158
"contentHash": "JgMAGj8ekeAzKkagubXqf1UqgfHq89GyA1UQYWbkAe441uRr2Rh2rktkx5Z0LPwmD/aOqu9cxjekD2GZjP8rbw=="
159159
},
160+
"Microsoft.Azure.Functions.Extensions": {
161+
"type": "Direct",
162+
"requested": "[1.1.0, )",
163+
"resolved": "1.1.0",
164+
"contentHash": "zYKtQQoS1fdzufxFApuMFiFtoi9QAGH6McXxntpylwLKgKjmCMWdgUd1dcekzTKNR9DPSDPRLiulvukqXnpWrQ==",
165+
"dependencies": {
166+
"Microsoft.Azure.WebJobs": "3.0.18",
167+
"Microsoft.Extensions.DependencyInjection": "2.1.0"
168+
}
169+
},
160170
"Microsoft.Azure.Functions.Worker": {
161171
"type": "Direct",
162172
"requested": "[1.10.0, )",
@@ -262,6 +272,31 @@
262272
"System.Net.Http": "4.3.0"
263273
}
264274
},
275+
"Microsoft.Extensions.Configuration.AzureAppConfiguration": {
276+
"type": "Direct",
277+
"requested": "[5.1.0, )",
278+
"resolved": "5.1.0",
279+
"contentHash": "FoAfgvT/rjL/+c7BP7q0LrJIdc4Hu6SH56BTIUbwCwVjHoUw4dpgGtLQULi5GmMjdbdAxyLQSnbwpOEWuBy+RA==",
280+
"dependencies": {
281+
"Azure.Data.AppConfiguration": "1.2.0",
282+
"Azure.Messaging.EventGrid": "4.7.0",
283+
"Azure.Security.KeyVault.Secrets": "4.0.1",
284+
"Microsoft.Extensions.Configuration": "3.1.18",
285+
"Microsoft.Extensions.DependencyInjection.Abstractions": "3.1.18",
286+
"Microsoft.Extensions.Logging": "3.1.18",
287+
"System.Text.Json": "4.6.0"
288+
}
289+
},
290+
"Microsoft.FeatureManagement": {
291+
"type": "Direct",
292+
"requested": "[2.5.1, )",
293+
"resolved": "2.5.1",
294+
"contentHash": "ERbRjk0etZs4d5Pv17unfogO4iBwV2c/HoBt4jqIJmfbKbmTLV+GbjBPYzidIg2RgYIFi8yA+EoEapSAIOp19g==",
295+
"dependencies": {
296+
"Microsoft.Extensions.Configuration.Binder": "2.1.10",
297+
"Microsoft.Extensions.Logging": "2.1.1"
298+
}
299+
},
265300
"Microsoft.Graph": {
266301
"type": "Direct",
267302
"requested": "[4.37.0, )",
@@ -355,6 +390,16 @@
355390
"resolved": "2.0.0",
356391
"contentHash": "rXkSI9t4vP2EaPhuchsWiD3elcLNth3UOZAlGohGmuckpkiOr57oMHuzM5WDzz7MJd+ZewE27/WfrZhhhFDHzA=="
357392
},
393+
"Azure.Data.AppConfiguration": {
394+
"type": "Transitive",
395+
"resolved": "1.2.0",
396+
"contentHash": "KA1dAM9TuDsq0CRFd+3cJTYUAzA2z9N8t9/xKdDbP9URuReq/NDFcKYr7GW2W9xzVGDtCHlD5j5am/+zLLBdSg==",
397+
"dependencies": {
398+
"Azure.Core": "1.20.0",
399+
"Microsoft.Bcl.AsyncInterfaces": "1.0.0",
400+
"System.Text.Json": "4.6.0"
401+
}
402+
},
358403
"Azure.Storage.Common": {
359404
"type": "Transitive",
360405
"resolved": "12.12.0",
@@ -567,6 +612,33 @@
567612
"Microsoft.CodeAnalysis.CSharp": "3.11.0"
568613
}
569614
},
615+
"Microsoft.Azure.WebJobs": {
616+
"type": "Transitive",
617+
"resolved": "3.0.18",
618+
"contentHash": "aYJ76yjPkIpsafqFp1Xz1sA06RvhUwqJnk4AqX4I0teuRjPyig9Sv7LTzxUMAppKXc4JyR/Asos2At/LMiblqg==",
619+
"dependencies": {
620+
"Microsoft.Azure.WebJobs.Core": "3.0.18",
621+
"Microsoft.Extensions.Configuration": "2.1.0",
622+
"Microsoft.Extensions.Configuration.Abstractions": "2.1.0",
623+
"Microsoft.Extensions.Configuration.EnvironmentVariables": "2.1.0",
624+
"Microsoft.Extensions.Configuration.Json": "2.1.0",
625+
"Microsoft.Extensions.Hosting": "2.1.0",
626+
"Microsoft.Extensions.Logging": "2.1.0",
627+
"Microsoft.Extensions.Logging.Abstractions": "2.1.0",
628+
"Microsoft.Extensions.Logging.Configuration": "2.1.0",
629+
"Newtonsoft.Json": "11.0.2",
630+
"System.Threading.Tasks.Dataflow": "4.8.0"
631+
}
632+
},
633+
"Microsoft.Azure.WebJobs.Core": {
634+
"type": "Transitive",
635+
"resolved": "3.0.18",
636+
"contentHash": "ajYI8pPzPn4qq7FL8C2tz9WmFEG5PorUlkw8W9CF5M+5egnFJaF7yH48WYC+zBoQIzv2vHmFq0zhQpnv+O8v5Q==",
637+
"dependencies": {
638+
"System.ComponentModel.Annotations": "4.4.0",
639+
"System.Diagnostics.TraceSource": "4.3.0"
640+
}
641+
},
570642
"Microsoft.Bcl.AsyncInterfaces": {
571643
"type": "Transitive",
572644
"resolved": "6.0.0",
@@ -1381,6 +1453,22 @@
13811453
"System.Runtime": "4.3.0"
13821454
}
13831455
},
1456+
"System.Diagnostics.TraceSource": {
1457+
"type": "Transitive",
1458+
"resolved": "4.3.0",
1459+
"contentHash": "VnYp1NxGx8Ww731y2LJ1vpfb/DKVNKEZ8Jsh5SgQTZREL/YpWRArgh9pI8CDLmgHspZmLL697CaLvH85qQpRiw==",
1460+
"dependencies": {
1461+
"Microsoft.NETCore.Platforms": "1.1.0",
1462+
"System.Collections": "4.3.0",
1463+
"System.Diagnostics.Debug": "4.3.0",
1464+
"System.Globalization": "4.3.0",
1465+
"System.Resources.ResourceManager": "4.3.0",
1466+
"System.Runtime": "4.3.0",
1467+
"System.Runtime.Extensions": "4.3.0",
1468+
"System.Threading": "4.3.0",
1469+
"runtime.native.System": "4.3.0"
1470+
}
1471+
},
13841472
"System.Diagnostics.Tracing": {
13851473
"type": "Transitive",
13861474
"resolved": "4.3.0",
@@ -2077,6 +2165,11 @@
20772165
"System.Runtime": "4.3.0"
20782166
}
20792167
},
2168+
"System.Threading.Tasks.Dataflow": {
2169+
"type": "Transitive",
2170+
"resolved": "4.8.0",
2171+
"contentHash": "PSIdcgbyNv7FZvZ1I9Mqy6XZOwstYYMdZiXuHvIyc0gDyPjEhrrP9OvTGDHp+LAHp1RNSLjPYssyqox9+Kt9Ug=="
2172+
},
20802173
"System.Threading.Tasks.Extensions": {
20812174
"type": "Transitive",
20822175
"resolved": "4.5.4",

0 commit comments

Comments
 (0)