-
Notifications
You must be signed in to change notification settings - Fork 850
Add LoadFromAsync and SaveToAsync helper methods to DataContent #7159
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
stephentoub
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/MediaTypeMap.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/MediaTypeMap.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/MediaTypeMap.cs
Outdated
Show resolved
Hide resolved
|
All review comments addressed in 840f61c. Changes include: MediaTypeMap moved to LegacySupport with full mappings, methods are now stable (not experimental), DefaultMediaType moved to DataContent as private const, using GetBuffer() for MemoryStream, proper null handling for Name, SaveToAsync handles directory paths and always infers extensions, and NET9_0_OR_GREATER conditional for ReadOnlyMemory overload. |
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds helper methods to DataContent for loading from and saving to file system paths and streams, with automatic media type (MIME type) inference based on file extensions.
Key Changes
- Adds static
LoadFromAsyncmethods (path and stream overloads) that infer media types and names from file extensions - Adds instance
SaveToAsyncmethod that handles directory paths and extension inference - Implements
MediaTypeMappolyfill for .NET 9 and earlier to enable file extension-to-MIME type mapping - Marks the new APIs as stable (non-experimental) in the JSON baseline
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs |
Implements the new LoadFromAsync and SaveToAsync methods with media type inference logic |
src/LegacySupport/MediaTypeMap/MediaTypeMap.cs |
Adds polyfill implementation of media type mapping for pre-.NET 10 frameworks |
src/LegacySupport/MediaTypeMap/README.md |
Documents the MediaTypeMap polyfill purpose and origin |
eng/MSBuild/LegacySupport.props |
Configures MSBuild to inject MediaTypeMap polyfill for legacy target frameworks |
src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.csproj |
Enables MediaTypeMap injection for the abstractions project |
src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.json |
Adds the new methods to the stable API baseline |
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Contents/DataContentTests.cs |
Adds comprehensive unit tests covering all new functionality and edge cases |
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Show resolved
Hide resolved
…o corruption Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
| { | ||
| private static readonly Dictionary<string, string> s_extensionToMediaType = new(StringComparer.OrdinalIgnoreCase) | ||
| { | ||
| [".3g2"] = "video/3gpp2", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot, please double check that these mappings are in sync with the current state of https://raw.githubusercontent.com/dotnet/runtime/refs/heads/main/src/libraries/System.Net.Mail/src/System/Net/Mime/MediaTypeMap.cs . It's possible that changed since this PR was created. This should have exactly the same set of entries for both directions of mapping.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 3bf025c. Found one missing entry: .cgm/image/cgm - the runtime has 404 extension mappings and 295 media type mappings, but our polyfill had 403 and 294. Now synced with the current runtime source.
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
Head branch was pushed to by a user without write access
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Show resolved
Hide resolved
| /// </param> | ||
| /// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests.</param> | ||
| /// <returns>The actual path where the data was saved, which may include an inferred file name and/or extension.</returns> | ||
| /// <exception cref="ArgumentNullException"><paramref name="path"/> is <see langword="null"/>.</exception> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be worthwhile to document that an IOException will be thrown in the file already exists.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot, please update the XML docs accordingly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 71efd51. Added <exception cref="IOException">A file already exists at the specified path.</exception> to the XML documentation.
src/Libraries/Microsoft.Extensions.AI.Abstractions/Contents/DataContent.cs
Show resolved
Hide resolved
| // If path is a directory, infer the file name from the Name property if available, | ||
| // or use a random name along with an extension inferred from the media type. | ||
| // If the path is empty, treat it as the current directory. | ||
| if (path.Length == 0 || Directory.Exists(path)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there might also be a case of the path representing a directory by having a trailing slash, but that directory doesn't yet exist. We should also handle that here (if you accept that we should create the directory).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment about lack of precedent. Do we do that anywhere else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like most of our lower level APIs do not create the directory, so we can leave it. Let's make sure to test the case where someone calls this API specifying a path to a directory that does not exist, with a parent directory that does exist. Make sure that we don't incidentally trim a trailing slash and create a file with the directory name.
Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>
| public async Task SaveToAsync_WithEmptyPath_UsesCurrentDirectory() | ||
| { | ||
| // Empty string path is valid - it uses the current directory with inferred filename. | ||
| // We use a unique name to avoid conflicts with concurrent tests. | ||
| string tempDir = Path.Combine(Path.GetTempPath(), $"test_empty_path_{Guid.NewGuid()}"); | ||
| Directory.CreateDirectory(tempDir); | ||
| string uniqueName = $"test_{Guid.NewGuid()}.json"; | ||
|
|
||
| try | ||
| { | ||
| // Save with empty path from within the temp directory context | ||
| // Since we can't change current directory, we test the directory path behavior instead | ||
| DataContent content = new(new byte[] { 1, 2, 3 }, "application/json") { Name = uniqueName }; | ||
| string savedPath = await content.SaveToAsync(tempDir); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't appear to do what the method name and comment says. It claims to be testing with an empty path / empty-string path, but instead it passes a directory.
I think this test meant to pass in String.Empty to SaveToAsync. It should do that, and then also test passing just a file-name. Both should write to current dir. The test can use the returned path for cleanup.
| } | ||
|
|
||
| [Fact] | ||
| public async Task LoadFromAsync_AbsolutePath_LoadsCorrectly() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test looks redundant to me.
| } | ||
|
|
||
| [Fact] | ||
| public async Task LoadFromAsync_RelativePath_LoadsCorrectly() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test doesn't do what it claims. It's using an absolute temp path.
| } | ||
|
|
||
| [Fact] | ||
| public async Task LoadFromAsync_RelativePathWithSubdirectory_LoadsCorrectly() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This also doesn't do what it claims
System.Net.MimenamespaceLoadFromAsyncmethods toDataContentclass:SaveToAsyncmethod toDataContentclass:Path.GetFileName(Name)FileMode.CreateNewto prevent accidentally overwriting existing files[Experimental]attribute and add methods to JSON API baseline as stableGetBuffer()instead ofToArray()for MemoryStream efficiencyWriteAllBytesAsync(ReadOnlyMemory<byte>)polyfill to remove#ifdeffromSaveToAsyncTask<T>toValueTask<T>for all new public async methodsFile.ReadAllBytesAsyncandFile.WriteAllBytesAsyncin LegacySupport directorynameparameter fromLoadFromAsync(Stream)- Name can be assigned after the callInjectFilePolyfillsOnLegacyto test projectMarkItDownMcpReader.csto useDataContent.LoadFromAsyncwith optional media type parameterMicrosoft.Extensions.AI.Abstractionsproject reference to MarkItDown project forLoadFromAsyncaccessImageGeneratorExtensions.cs- Replace_extensionToMimeTypedictionary withMediaTypeMap.GetMediaTypeOpenAIClientExtensions.cs- ReplaceImageUriToMediaTypewithMediaTypeMap.GetMediaTypeOpenAIImageGenerator.cs- Replace_mimeTypeToExtensiondictionary withMediaTypeMap.GetExtensionIngestionDocumentReader.cs- Remove custom lookup, only useMediaTypeMap.GetMediaTypeOriginal prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.
Microsoft Reviewers: Open in CodeFlow