Skip to content

Improve CreateDirectory on Windows #66754

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
8ce1a96
Reduce Kernel32 calls
deeprobin Mar 16, 2022
de9e3a9
Eliminate unnessecary length check
deeprobin Mar 16, 2022
a0b5388
Restrict EnsureExtendedPrefix call to paths longer than `MaxShortDire…
deeprobin Mar 16, 2022
e43967a
Use `EnsureExtendedPrefixIfNeeded` instead of conditional `EnsureExte…
deeprobin Mar 16, 2022
d6b06b3
Slim implementation of FillAttributeInfo to reduce range checks
deeprobin Mar 16, 2022
31961a4
Put `stackDir.Count - 1` into a local to improve codegen
deeprobin Mar 16, 2022
12d7729
Use down-counting loop instead of List-RemoveAt construct
deeprobin Mar 16, 2022
6faa0e1
Spannify strings
deeprobin Mar 16, 2022
bf0e715
Reduce range checks in interop calls
deeprobin Mar 16, 2022
7fc97aa
Replace modern language with legacy language to fix netfx builds
deeprobin Mar 20, 2022
920bf49
Apply suggestions
deeprobin Mar 20, 2022
8cfca86
Revert "Reduce Kernel32 calls" because of breaking changes
deeprobin Mar 20, 2022
b9e4754
Remove "yoda conditions"
deeprobin Mar 20, 2022
8fb6ecb
Add newlines
deeprobin Mar 22, 2022
5bc0806
Inline `EndsInDirectorySeparator`
deeprobin Mar 22, 2022
c926642
Inline `GetFileAttributesExPrefixed`
deeprobin Mar 22, 2022
378a6f7
Merge branch 'main' into issue-61954-rev
deeprobin May 2, 2022
61af4d3
Apply suggestions
deeprobin May 2, 2022
056c9b9
Merge branch 'main' into issue-61954-rev
jeffhandley Jun 30, 2022
cb6a732
Merge branch 'main' into issue-61954-rev
jeffhandley Jun 30, 2022
ff116e4
Merge branch 'main' into issue-61954-rev
deeprobin Jul 4, 2022
bc22603
Revert behavior change
deeprobin Jul 4, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using Microsoft.Win32.SafeHandles;
using System.IO;
using System.Runtime.InteropServices;

Expand All @@ -20,8 +19,9 @@ private static partial bool CreateDirectoryPrivate(

internal static bool CreateDirectory(string path, ref SECURITY_ATTRIBUTES lpSecurityAttributes)
{
// We always want to add for CreateDirectory to get around the legacy 248 character limitation
path = PathInternal.EnsureExtendedPrefix(path);
// If length is greater than `MaxShortDirectoryPath` we add a extended prefix to get around the legacy character limitation
path = PathInternal.EnsureExtendedPrefixIfNeeded(path);

return CreateDirectoryPrivate(path, ref lpSecurityAttributes);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ internal static partial class Kernel32
/// </summary>
[LibraryImport(Libraries.Kernel32, EntryPoint = "GetFileAttributesExW", SetLastError = true, StringMarshalling = StringMarshalling.Utf16)]
[return: MarshalAs(UnmanagedType.Bool)]
private static partial bool GetFileAttributesExPrivate(
internal static partial bool GetFileAttributesExPrivate(
string? name,
GET_FILEEX_INFO_LEVELS fileInfoLevel,
ref WIN32_FILE_ATTRIBUTE_DATA lpFileInformation);
Expand Down
92 changes: 57 additions & 35 deletions src/libraries/Common/src/System/IO/FileSystem.Attributes.Windows.cs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,17 @@ private static bool DirectoryExists(string? path, out int lastError)
((data.dwFileAttributes & Interop.Kernel32.FileAttributes.FILE_ATTRIBUTE_DIRECTORY) != 0);
}

private static bool DirectoryExistsSlim(string? path, out int lastError)
{
Interop.Kernel32.WIN32_FILE_ATTRIBUTE_DATA data = default;
lastError = FillAttributeInfoSlim(path, ref data, returnErrorOnNotFound: true);

return
(lastError == 0) &&
(data.dwFileAttributes != -1) &&
((data.dwFileAttributes & Interop.Kernel32.FileAttributes.FILE_ATTRIBUTE_DIRECTORY) != 0);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding these "Slim" functions seems to be an attempt to improve throughput, but the perf results shared don't really show a throughput improvement. Seems like these changes should be reverted?


public static bool FileExists(string fullPath)
{
Interop.Kernel32.WIN32_FILE_ATTRIBUTE_DATA data = default;
Expand All @@ -49,53 +60,64 @@ public static bool FileExists(string fullPath)
/// <param name="returnErrorOnNotFound">Return the error code for not found errors?</param>
internal static int FillAttributeInfo(string? path, ref Interop.Kernel32.WIN32_FILE_ATTRIBUTE_DATA data, bool returnErrorOnNotFound)
{
int errorCode = Interop.Errors.ERROR_SUCCESS;

// Neither GetFileAttributes or FindFirstFile like trailing separators
path = PathInternal.TrimEndingDirectorySeparator(path);

using (DisableMediaInsertionPrompt.Create())
{
if (!Interop.Kernel32.GetFileAttributesEx(path, Interop.Kernel32.GET_FILEEX_INFO_LEVELS.GetFileExInfoStandard, ref data))
return FillAttributeInfoSlim(path, ref data, returnErrorOnNotFound);
}
}

/// <summary>
/// Same as <see cref="FillAttributeInfo"/> without separator-trimming and media-insertion blocking
/// </summary>
/// <param name="path">The file path from which the file attribute information will be filled.</param>
/// <param name="data">A struct that will contain the attribute information.</param>
/// <param name="returnErrorOnNotFound">Return the error code for not found errors?</param>
internal static int FillAttributeInfoSlim(string? path, ref Interop.Kernel32.WIN32_FILE_ATTRIBUTE_DATA data, bool returnErrorOnNotFound)
{
int errorCode = Interop.Errors.ERROR_SUCCESS;
string? prefixedString = PathInternal.EnsureExtendedPrefixIfNeeded(path);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FindFirstFilePrefixed path here is the rare case, right? (<1%) Where the file is locked.

Given that, I don't think you're really saving anything by ensuring you only call EnsureExtendedPrefixIfNeeded once. I would reverse this and leave GetFileAttributesExPrivate private. That avoids creating a way for future code to accidentally call GetFileAttributesEx without prefixing. And it leaves this code a bit simpler.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was marked resolved but I don't see the feedback addressed. Was it commented on somewhere else?


if (!Interop.Kernel32.GetFileAttributesExPrivate(prefixedString, Interop.Kernel32.GET_FILEEX_INFO_LEVELS.GetFileExInfoStandard, ref data))
{
errorCode = Marshal.GetLastWin32Error();

Interop.Kernel32.WIN32_FIND_DATA findData = default;
if (!IsPathUnreachableError(errorCode))
{
errorCode = Marshal.GetLastWin32Error();
// Assert so we can track down other cases (if any) to add to our test suite
Debug.Assert(errorCode == Interop.Errors.ERROR_ACCESS_DENIED || errorCode == Interop.Errors.ERROR_SHARING_VIOLATION || errorCode == Interop.Errors.ERROR_SEM_TIMEOUT,
$"Unexpected error code getting attributes {errorCode} from path {path}");

// Files that are marked for deletion will not let you GetFileAttributes,
// ERROR_ACCESS_DENIED is given back without filling out the data struct.
// FindFirstFile, however, will. Historically we always gave back attributes
// for marked-for-deletion files.
//
// Another case where enumeration works is with special system files such as
// pagefile.sys that give back ERROR_SHARING_VIOLATION on GetAttributes.
//
// Ideally we'd only try again for known cases due to the potential performance
// hit. The last attempt to do so baked for nearly a year before we found the
// pagefile.sys case. As such we're probably stuck filtering out specific
// cases that we know we don't want to retry on.

if (!IsPathUnreachableError(errorCode))
using SafeFindHandle handle = Interop.Kernel32.FindFirstFile(prefixedString!, ref findData);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this change?

if (handle.IsInvalid)
{
// Assert so we can track down other cases (if any) to add to our test suite
Debug.Assert(errorCode == Interop.Errors.ERROR_ACCESS_DENIED || errorCode == Interop.Errors.ERROR_SHARING_VIOLATION || errorCode == Interop.Errors.ERROR_SEM_TIMEOUT,
$"Unexpected error code getting attributes {errorCode} from path {path}");

// Files that are marked for deletion will not let you GetFileAttributes,
// ERROR_ACCESS_DENIED is given back without filling out the data struct.
// FindFirstFile, however, will. Historically we always gave back attributes
// for marked-for-deletion files.
//
// Another case where enumeration works is with special system files such as
// pagefile.sys that give back ERROR_SHARING_VIOLATION on GetAttributes.
//
// Ideally we'd only try again for known cases due to the potential performance
// hit. The last attempt to do so baked for nearly a year before we found the
// pagefile.sys case. As such we're probably stuck filtering out specific
// cases that we know we don't want to retry on.

Interop.Kernel32.WIN32_FIND_DATA findData = default;
using (SafeFindHandle handle = Interop.Kernel32.FindFirstFile(path!, ref findData))
{
if (handle.IsInvalid)
{
errorCode = Marshal.GetLastWin32Error();
}
else
{
errorCode = Interop.Errors.ERROR_SUCCESS;
data.PopulateFrom(ref findData);
}
}
errorCode = Marshal.GetLastWin32Error();
}
else
{
errorCode = Interop.Errors.ERROR_SUCCESS;
data.PopulateFrom(ref findData);
}
}
}


if (errorCode != Interop.Errors.ERROR_SUCCESS && !returnErrorOnNotFound)
{
switch (errorCode)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,36 +37,44 @@ public static unsafe void CreateDirectory(string fullPath, byte[]? securityDescr
int length = fullPath.Length;

// We need to trim the trailing slash or the code will try to create 2 directories of the same name.
if (length >= 2 && PathInternal.EndsInDirectorySeparator(fullPath.AsSpan()))
if (length >= 2 && PathInternal.IsDirectorySeparator(fullPath.AsSpan()[^1]))
{
length--;
}

int lengthRoot = PathInternal.GetRootLength(fullPath.AsSpan());

if (length > lengthRoot)
using (DisableMediaInsertionPrompt.Create())
{
// Special case root (fullpath = X:\\)
int i = length - 1;
while (i >= lengthRoot && !somepathexists)
if (length > lengthRoot)
{
string dir = fullPath.Substring(0, i + 1);

if (!DirectoryExists(dir)) // Create only the ones missing
{
stackDir.Add(dir);
}
else
// Special case root (fullpath = X:\\)
int i = length - 1;
while (i >= lengthRoot && !somepathexists)
{
somepathexists = true;
}
ReadOnlySpan<char> dir = fullPath.AsSpan()[..(i + 1)];

// Neither GetFileAttributes or FindFirstFile like trailing separators
dir = PathInternal.TrimEndingDirectorySeparator(dir);

string dirString = new(dir);

if (!DirectoryExistsSlim(dirString, out _)) // Create only the ones missing
{
stackDir.Add(dirString);
}
else
{
somepathexists = true;
}

while (i > lengthRoot && !PathInternal.IsDirectorySeparator(fullPath[i]))
{
i--;
}

while (i > lengthRoot && !PathInternal.IsDirectorySeparator(fullPath[i]))
{
i--;
}

i--;
}
}

Expand All @@ -83,10 +91,9 @@ public static unsafe void CreateDirectory(string fullPath, byte[]? securityDescr
lpSecurityDescriptor = (IntPtr)pSecurityDescriptor
};

while (stackDir.Count > 0)
for (int i = count - 1; i >= 0; i--)
{
string name = stackDir[stackDir.Count - 1];
stackDir.RemoveAt(stackDir.Count - 1);
string name = stackDir[i];

r = Interop.Kernel32.CreateDirectory(name, ref secAttrs);
if (!r && (firstError == 0))
Expand Down
3 changes: 1 addition & 2 deletions src/libraries/Common/src/System/IO/PathInternal.Windows.cs
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,10 @@ internal static bool IsValidDriveChar(char value)
return (uint)((value | 0x20) - 'a') <= (uint)('z' - 'a');
}

internal static bool EndsWithPeriodOrSpace(string? path)
internal static bool EndsWithPeriodOrSpace(string path)
{
if (string.IsNullOrEmpty(path))
return false;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are the changes in this file necessary?

char c = path[path.Length - 1];
return c == ' ' || c == '.';
}
Expand Down
10 changes: 2 additions & 8 deletions src/libraries/Common/src/System/IO/PathInternal.cs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ internal static partial class PathInternal
internal static bool StartsWithDirectorySeparator(ReadOnlySpan<char> path) => path.Length > 0 && IsDirectorySeparator(path[0]);

internal static string EnsureTrailingSeparator(string path)
=> EndsInDirectorySeparator(path.AsSpan()) ? path : path + DirectorySeparatorCharAsString;
=> path.Length > 0 && IsDirectorySeparator(path[path.Length - 1]) ? path : path + DirectorySeparatorCharAsString;

internal static bool IsRoot(ReadOnlySpan<char> path)
=> path.Length == GetRootLength(path);
Expand Down Expand Up @@ -231,16 +231,10 @@ internal static bool EndsInDirectorySeparator(string? path) =>
/// Trims one trailing directory separator beyond the root of the path.
/// </summary>
internal static ReadOnlySpan<char> TrimEndingDirectorySeparator(ReadOnlySpan<char> path) =>
EndsInDirectorySeparator(path) && !IsRoot(path) ?
path.Length > 0 && IsDirectorySeparator(path[path.Length - 1]) && !IsRoot(path) ?
path.Slice(0, path.Length - 1) :
path;

/// <summary>
/// Returns true if the path ends in a directory separator.
/// </summary>
internal static bool EndsInDirectorySeparator(ReadOnlySpan<char> path) =>
path.Length > 0 && IsDirectorySeparator(path[path.Length - 1]);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are the changes in this file necessary?

internal static string GetLinkTargetFullPath(string path, string pathToTarget)
=> IsPartiallyQualified(pathToTarget.AsSpan()) ?
Path.Join(Path.GetDirectoryName(path.AsSpan()), pathToTarget.AsSpan()) : pathToTarget;
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Private.CoreLib/src/System/IO/Path.cs
Original file line number Diff line number Diff line change
Expand Up @@ -986,7 +986,7 @@ private static string GetRelativePath(string relativeTo, string path, StringComp
/// <summary>
/// Returns true if the path ends in a directory separator.
/// </summary>
public static bool EndsInDirectorySeparator(ReadOnlySpan<char> path) => PathInternal.EndsInDirectorySeparator(path);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are the changes in this file necessary?

public static bool EndsInDirectorySeparator(ReadOnlySpan<char> path) => path.Length > 0 && PathInternal.IsDirectorySeparator(path[path.Length - 1]);

/// <summary>
/// Returns true if the path ends in a directory separator.
Expand Down