What's new in .NET 8 Preview 4 [WIP]

[leecow]

What's new in .NET 8 Preview 4

This issue is for teams to highlight work for the community that will release in .NET 8 Preview 4

To add content, use a new conversation entry. The entry should include the team name and feature title as the first line shown in the template below.

Required

## Team Name: Feature title

[link to the tracking issue or epic item for the work]

Tell the story of the feature and anything the community should pay particular attention 
to be successful in using the feature.

Optional

Below are three additional items to consider. These will help the .NET 8 blog team and the community throughout the release.

• Link to documentation.
  • Where can we send people to learn more about the feature?
• Link to where you'd like people to provide feedback for this feature.
  • Where can people best comment on the direction of this feature?
• Whether you would like assistance from the .NET 8 writers to craft a story for this feature.
  • Please @JonDouglas or @JeremyLikness in your comment if so.

Index of .NET 8 releases

Preview 1: #8133
Preview 2: #8134
Preview 3: #8135
Preview 4: #8234
Preview 5: #8436
Preview 6: #8437
Preview 7: #8438
RC 1: #8439
RC 2: #8440

[baronfel]

MSBuild: New, modern terminal build output

• MSBuild TerminalLogger keeps users informed about their build and delights new users msbuild#8370

We often get feedback from users that the default MSBuild output (internally known as the console logger) is difficult to parse. It's pretty static, often is a wall of text, and it emits errors as they are triggered during the build instead of logically showing them as part of the project being built. We think these are all great feedback, and are happy to introduce our first iteration on a newer, more modern take on MSBuild output logging. We're calling it Terminal Logger, and it has a few main goals:

• logically group errors with the project they belong with
• present projects/builds in a way that users think of the build (especially multi-targeted projects)
• better differentiate the TargetFrameworks a project builds for
• continue to provide at-a-glance information about the outputs of a project
• provide information about what the build is doing right now over the course of a build.

Here's what it looks like:

The new output can be enabled using /tl, optionally with one of the following options:

• auto - the default, which checks if the terminal is capable of using the new features and isn't using a redirected standard output before enabling the new logger,
• on - overrides the environment detection mentioned above and forces the new logger to be used
• off - overrides the environment detection mentioned above and forces the previous console logger to be used

Once enabled, the new logger shows you the restore phase, followed by the build phase. During each phase, the currently-building projects are at the bottom of the terminal, and each building project tells you both the MSBuild Target currently being built, as well as the amount of time spent on that target. We hope this information makes builds less mysterious to users, as well as giving them a place to start searches when they want to learn more about the build! As projects are fully built, a single 'build completed' section is written for each build that captures

• the name of the built project
• the Target Framework (if multitargeted!)
• the status of that build
• the primary output of that build (hyperlinked for quick access)
• and finally any diagnostics generated by the build for that project

There weren't any diagnostics for that example - let's look at another build of the same project where a typo has been introduced:

Here you can clearly see the project and typo error outlined.

We think that this layout fits the modern feel of .NET and makes use of the capabilities of modern terminals to tell users more about their builds. We hope you'll try it out and provide us feedback on how it works for you, as well as other information you'd like to see here. We hope to use this logger as the foundation for a new batch of UX improvements for MSBuild - including aspects like progress reporting and structured errors in the future! As you use it, please give us feedback via this survey, or via the Discussions section of the MSBuild repository. We look forward to hearing from you all!

This work was inspired and begun by Eduardo Villalpando, our intern for the Winter season, who dug into the problem and really helped blaze a trail for the rest of us to follow. It wouldn't have been possible without his help and enthusiasm for the problem!

[baronfel]

SDK: Simplified output path updates

• Add proposal for simplified output paths (version 2) designs#281
• Use 'artifacts' instead of .artifacts for default artifacts path sdk#31955

In preview 3 we announced the new simplified output path layout for .NET SDK projects and asked you for your feedback and experiences using the new layout. Thank you for doing so! Your feedback spawned many discussions and based on what we heard from everyone trialing the changes we've made the following updates to the feature:

• The default path for the new layout is changing from .artifacts to artifacts
• We're removing the ability to use the feature from Project files instead of Directory.Build.props
• We're making it easier to get started with the feature by including the required properties as an option on the buildprops template for dotnet new

I want to go into the thought process that led us to these changes. You all overwhelmingly supported removing the . from the folder name, mostly for visiblity on unix-systems, where . usually signifies a 'hidden' file or folder. So we knew we wanted to make that change. However, there were two main reasons we didn't want to use artifacts as the root path initially -

• .gitignore support
• .NET SDK file globbing hurdles

We didn't want people to suddenly have to deal with changes to their .gitignore files just to try out the feature, but after some research we discovered that some future-facing, enterprising contributor (thanks @sayedihashimi!) already made sure that artifacts is in all of the common templates for .gitignore files. This means that we didn't have to worry about users checking in binaries unexpectedly.

We also didn't want to accidentally include the artifacts outputs in the default glob patterns that the .NET SDK uses to find the source files to build in a project. If we changed the root path from .artifacts to artifacts and let users use the new features from the Project-file level, then we would also have to go change all of the default includes that make SDK project files so succinct. This seemed very error prone, and frankly a pit of failure in the user experience. As a result, we've tightened the requirements to use the feature - you now must opt into the feature via Directory.Build.props file. This has the side effect of making the feature more stable. Before this change, the inferred root folder location would change when a Directory.Build.props file was created. Now, because a Directory.Build.props must exist, the location of the Artifacts path should remain stable.

To try out the new version of the feature, we've made it easier to generate the correct Directory.Build.props file: just run dotnet new buildprops --use-artifacts and we'll generate all you need. The generated Directory.Build.props file looks like this:

<Project>
  <!-- See https://aka.ms/dotnet/msbuild/customize for more details on customizing your build -->
  <PropertyGroup>
    <ArtifactsPath>$(MSBuildThisFileDirectory)artifacts</ArtifactsPath>
  </PropertyGroup>
</Project>

Please go try out these changes and continue to let us know what you think at our survey for the feature.

[cshung]

GC: RefreshMemoryLimit

In .NET 8 Preview 4, we have the capability to adjust the memory limit on the fly.

What is the problem?

In a cloud service scenario, demand comes and goes. To be cost-effective, it is important that our services can scale up and scale down on resource consumption as the demand fluctuates.

What is the solution?

When we detect a decrease in demand, we can scale down the resource consumption by reducing the memory limit of the service. Previously, this will fail because the GC is unaware of the change and may allocate more memory than the new limit. With this change, you can call _RefreshMemoryLimit API to update the GC with the new memory limit.

Limitations

• Right now the API is private, meaning you will need to call the API through private reflection. This is done to avoid future breaking changes if we have to.
• On 32-bit platforms (e.g. Windows x86, Linux ARM), we were unable to establish a new heap hard limit if there isn't already one.
• The API might return a non-zero status code indicating the refresh failed. This can happen if the scale-down is too aggressive and leave no room for the GC to maneuver. In this case, consider calling GC.Collect(2, GCCollectionMode.Aggressive) to shrink current memory usage and try again.
• If we scale up the memory limit beyond the size that the GC believes the process can handle during startup, the _RefreshMemoryLimit call will succeed, but it won't be able to use memory more than that.

How to use the API?

MethodInfo refreshMemoryLimitMethod = typeof(GC).GetMethod("_RefreshMemoryLimit", BindingFlags.NonPublic | BindingFlags.Static);
refreshMemoryLimitMethod.Invoke(null, Array<object>.Empty);

You can also have the GC to refresh some of the GC configuration settings related to the memory limit as follow:

AppContext.SetData("GCHeapHardLimit", (ulong)100 * 1024 * 1024);
MethodInfo refreshMemoryLimitMethod = typeof(GC).GetMethod("_RefreshMemoryLimit", BindingFlags.NonPublic | BindingFlags.Static);
refreshMemoryLimitMethod.Invoke(null, Array<object>.Empty);

This will set the heap hard limit to 100M.

[YuliiaKovalova]

Template Engine: secure experience with packages from Nuget.org

In .NET 8 we're integrating several of NuGet.org's security-related features into the Template Engine, especially in the dotnet new experience.

Improvements

• Prevent downloading packages from http:\\ feeds, but allow overrides with the --force flag

The NuGet team has a plan of record for gradually moving over to a secure-by-default stance. You can read more about their plan, and the timelines involved, in the HTTPS Everywhere blog post. In support of that goal, we're going to start erroring by default when a non-HTTPS source is used. This can be overriden with --force for the .NET 8 time frame, but the current plan is to remove this flag for the .NET 9 time frame, in line with the HTTPS Everywhere timeline.

• Notify a customer if a template package has any vulnerabilities on install/update/outdated checks, and require --force to install vulnerable versions
  

• Add data to the search and uninstall commands that shows if a template is installed from a package that has prefix reserved in NuGet.org
  

• Add information about the template package owner. The ownership is verified by nuget portal and can be considered a trustworthy characteristic.
  

[stephentoub]

Libraries: UTF8 improvements

With .NET 8 Preview 4, we've introduced the new IUtf8SpanFormattable interface, which like its ISpanFormattable cousin, can be implemented on a type to enable writing out a string-like representation of that type to a destination span. Whereas ISpanFormattable targets UTF16 and Span<char>, IUtf8SpanFormattable targets UTF8 and Span<byte>. It's also been implemented on all of the primitive types (plus others), with the exact same shared logic (thanks to static abstract interfaces) whether targeting string, Span<char>, or Span<byte>, which means it has full support for all formats (including the "B" binary specifier that's also new in .NET 8 Preview 4) and all cultures. This means you can now format directly to UTF8 from Byte, Complex, Char, DateOnly, DateTime, DateTimeOffset, Decimal, Double, Guid, Half, IPAddress, IPNetwork, Int16, Int32, Int64, Int128, IntPtr, NFloat, SByte, Single, Rune, TimeOnly, TimeSpan, UInt16, UInt32, UInt64, UInt128, UIntPtr, and Version.

In addition, the new Utf8.TryWrite methods now provide a UTF8-based counterpart to the existing MemoryExtensions.TryWrite UTF16-based methods. These methods rely on the interpolated string handler support introduced in .NET 6 and C# 10, such that you can use interpolated string syntax to format a complex expression directly into a span of UTF8 bytes, e.g.

static bool FormatHexVersion(short major, short minor, short build, short revision, Span<byte> utf8Bytes, out int bytesWritten) =>
    Utf8.TryWrite(utf8Bytes, CultureInfo.InvariantCulture, $"{major:X4}.{minor:X4}.{build:X4}.{revision:X4}", out bytesWritten);

The implementation recognizes IUtf8SpanFormattable on the format values and uses their implementations to write their UTF8 representations directly to the destination span.

The implementation also utilizes the new Encoding.TryGetBytes method, which along with its Encoding.TryGetChars counterpart, supports encoding/decoding into a destination span as long as the span is long enough to hold the resulting state, and returning false rather than throwing an exception if it's not.

We expect more UTF8 improvements, including but not limited to improvements to the performance of this functionality, to show up in subsequent .NET 8 previews.

[dtivel]

NuGet: signed package verification on Linux

Starting with .NET 8 Preview 4 SDK, NuGet will verify signed packages on Linux by default. Verification remains enabled on Windows and disabled on macOS.

For most Linux users, verification should just work transparently. However, users with an existing root certificate bundle located at /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem may see trust failures accompanied by NU3042.

Users can opt out of verification by setting the environment variable DOTNET_NUGET_SIGNATURE_VERIFICATION to false. Please give your feedback to help the NuGet team improve the experience on Linux!

For more information, see #7688.

[tarekgh]

Introducing Time abstraction

The introduction of the TimeProvider abstract class adds time abstraction, which enables time mocking in test scenarios. This functionality is also supported by other features that rely on time progression, such as Task.Delay and Task.Async. This means that even Task operations can be easily mocked using the time abstraction. The abstraction supports essential time operations such as retrieving local and UTC time, obtaining a timestamp for performance measurement, and creating timers.

    public abstract class TimeProvider
    {
        public static TimeProvider System { get; }
        protected TimeProvider() 
        public virtual DateTimeOffset GetUtcNow()
        public DateTimeOffset GetLocalNow()
        public virtual TimeZoneInfo LocalTimeZone { get; }
        public virtual long TimestampFrequency { get; }
        public virtual long GetTimestamp()
        public TimeSpan GetElapsedTime(long startingTimestamp)
        public TimeSpan GetElapsedTime(long startingTimestamp, long endingTimestamp)
        public virtual ITimer CreateTimer(TimerCallback callback, object? state,TimeSpan dueTime, TimeSpan period)
    }

    public interface ITimer : IDisposable, IAsyncDisposable
    {
        bool Change(TimeSpan dueTime, TimeSpan period);
    }

    public partial class CancellationTokenSource : IDisposable
    {
        public CancellationTokenSource(TimeSpan delay, TimeProvider timeProvider)
    }

    public sealed partial class PeriodicTimer : IDisposable
    {
        public PeriodicTimer(TimeSpan period, TimeProvider timeProvider) 
    }

    public partial class Task : IAsyncResult, IDisposable
    {
        public static Task Delay(System.TimeSpan delay, System.TimeProvider timeProvider)
        public static Task Delay(System.TimeSpan delay, System.TimeProvider timeProvider, System.Threading.CancellationToken cancellationToken)
        
        public Task WaitAsync(TimeSpan timeout, TimeProvider timeProvider)
        public Task WaitAsync(TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken) 
    }

    public partial class Task<TResult> : Task
    {
        public new Task<TResult> WaitAsync(TimeSpan timeout, TimeProvider timeProvider)
        public new Task<TResult> WaitAsync(TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken) 
     }

Furthermore, we have made the abstraction available in .NET 8.0 and created a netstandard 2.0 library called Microsoft.Bcl.TimeProvider. This enables the use of the abstraction on supported versions of the .NET Framework and earlier versions of .NET.

namespace System.Threading.Tasks
{
    public static class TimeProviderTaskExtensions
    {
        public static Task Delay(this TimeProvider timeProvider, TimeSpan delay, CancellationToken cancellationToken = default) 
        public static Task<TResult> WaitAsync<TResult>(this Task<TResult> task, TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken = default)
        public static Tasks.Task WaitAsync(this Task task, TimeSpan timeout, TimeProvider timeProvider, CancellationToken cancellationToken = default)
        public static CancellationTokenSource CreateCancellationTokenSource(this TimeProvider timeProvider, TimeSpan delay)
    }
}

Usage Examples

	//  Get System time
    DateTimeOffset utcNow= TimeProvider.System.GetUtcNow();
    DateTimeOffset localNow = TimeProvider.System.GetLocalNow();

    // Create a time provider that work with a time zone different than the local time zone 
    private class ZonedTimeProvider : TimeProvider
    {
        private TimeZoneInfo _zoneInfo;
        public ZonedTimeProvider(TimeZoneInfo zoneInfo) : base()
        {
            _zoneInfo = zoneInfo ?? TimeZoneInfo.Local;
        }
        public override TimeZoneInfo LocalTimeZone { get => _zoneInfo; }
        public static TimeProvider FromLocalTimeZone(TimeZoneInfo zoneInfo) => new ZonedTimeProvider(zoneInfo);
    }

    // Create a time using a time provider 
    ITimer timer = timeProvider.CreateTimer(callBack, state, delay, Timeout.InfiniteTimeSpan);
    
    // Measure a period using the system time provider 
    long providerTimestamp1 = TimeProvider.System.GetTimestamp();
    long providerTimestamp2 = TimeProvider.System.GetTimestamp();
	var period = GetElapsedTime(providerTimestamp1, providerTimestamp2);

dotnet/runtime#36617