Skip to content

NRE in GetFileList #11528

Description

@rainersigwald

Got a report of this error (coming from the NuGet executable inside a static-graph restore):

MSBuild version = "17.14.0-preview-25105-02+915bcd94d"

Object reference not set to an instance of an object.
   at System.Object.GetType()
   at Microsoft.Build.Internal.EngineFileUtilities.GetFileList(String directoryEscaped, String filespecEscaped, Boolean returnEscaped, Boolean forceEvaluateWildCards, IEnumerable`1 excludeSpecsEscaped, FileMatcher fileMatcher, Object loggingMechanism, IElementLocation includeLocation, IElementLocation excludeLocation, IElementLocation importLocation, BuildEventContext buildEventContext, String buildEventFileInfoFullPath, Boolean disableExcludeDriveEnumerationWarning)
   at Microsoft.Build.Evaluation.Expander`2.ItemExpander.IntrinsicItemFunctions`1.<GetItemPairEnumerable>d__3.MoveNext()
   at Microsoft.Build.Evaluation.Expander`2.ItemExpander.IntrinsicItemFunctions`1.<GetPathsOfAllDirectoriesAbove>d__8.MoveNext()
   at Microsoft.Build.Evaluation.Expander`2.ItemExpander.<Transform>d__0`1.MoveNext()
   at Microsoft.Build.Evaluation.Expander`2.ItemExpander.<Transform>d__0`1.MoveNext()
   at Microsoft.Build.Evaluation.Expander`2.ItemExpander.ExpandExpressionCapture[S](Expander`2 expander, ItemExpressionCapture expressionCapture, IItemProvider`1 evaluatedItems, IElementLocation elementLocation, ExpanderOptions options, Boolean includeNullEntries, Boolean& isTransformExpression, List`1& itemsFromCapture)
   at Microsoft.Build.Evaluation.Expander`2.ItemExpander.ExpandExpressionCaptureIntoItems[S,T](ItemExpressionCapture expressionCapture, Expander`2 expander, IItemProvider`1 items, IItemFactory`2 itemFactory, ExpanderOptions options, Boolean includeNullEntries, Boolean& isTransformExpression, IElementLocation elementLocation)
   at Microsoft.Build.Evaluation.LazyItemEvaluator`4.IncludeOperation.SelectItems(Builder listBuilder, ImmutableHashSet`1 globsToIgnore)
   at Microsoft.Build.Evaluation.LazyItemEvaluator`4.LazyItemOperation.ApplyImpl(Builder listBuilder, ImmutableHashSet`1 globsToIgnore)
   at Microsoft.Build.Evaluation.LazyItemEvaluator`4.LazyItemOperation.Apply(Builder listBuilder, ImmutableHashSet`1 globsToIgnore)
   at Microsoft.Build.Evaluation.LazyItemEvaluator`4.LazyItemList.ComputeItems(LazyItemList lazyItemList, ImmutableHashSet`1 globsToIgnore)
   at Microsoft.Build.Evaluation.LazyItemEvaluator`4.LazyItemList.GetItemData(ImmutableHashSet`1 globsToIgnore)
   at System.Linq.Enumerable.<SelectManyIterator>d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.<GetEnumerator>d__1.MoveNext()
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate()
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate(IEvaluatorData`4 data, Project project, ProjectRootElement root, ProjectLoadSettings loadSettings, Int32 maxNodeCount, PropertyDictionary`1 environmentProperties, ILoggingService loggingService, IItemFactory`2 itemFactory, IToolsetProvider toolsetProvider, IDirectoryCacheFactory directoryCacheFactory, ProjectRootElementCacheBase projectRootElementCache, BuildEventContext buildEventContext, ISdkResolverService sdkResolverService, Int32 submissionId, EvaluationContext evaluationContext, Boolean interactive)
   at Microsoft.Build.Execution.ProjectInstance.Initialize(ProjectRootElement xml, IDictionary`2 globalProperties, String explicitToolsVersion, String explicitSubToolsetVersion, Int32 visualStudioVersionFromSolution, BuildParameters buildParameters, ILoggingService loggingService, BuildEventContext buildEventContext, ISdkResolverService sdkResolverService, Int32 submissionId, Nullable`1 projectLoadSettings, EvaluationContext evaluationContext, IDirectoryCacheFactory directoryCacheFactory)
   at Microsoft.Build.Execution.ProjectInstance..ctor(String projectFile, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, Nullable`1 projectLoadSettings, EvaluationContext evaluationContext, IDirectoryCacheFactory directoryCacheFactory, Boolean interactive)
   at Microsoft.Build.Execution.ProjectInstance.FromFile(String file, ProjectOptions options)
   at Microsoft.Build.Graph.GraphBuilder.ParseProject(ConfigurationMetadata configurationMetadata)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.LazyInitValue()
   at Microsoft.Build.Graph.ParallelWorkSet`2.ExecuteWorkItem()

let's at least turn nullable analysis on for this method:

private static string[] GetFileList(
string directoryEscaped,
string filespecEscaped,
bool returnEscaped,
bool forceEvaluateWildCards,
IEnumerable<string> excludeSpecsEscaped,
FileMatcher fileMatcher,
object loggingMechanism = null,
IElementLocation includeLocation = null,
IElementLocation excludeLocation = null,
IElementLocation importLocation = null,
BuildEventContext buildEventContext = null,
string buildEventFileInfoFullPath = null,
bool disableExcludeDriveEnumerationWarning = false)
{
ErrorUtilities.VerifyThrowInternalLength(filespecEscaped, nameof(filespecEscaped));
string[] fileList = [];
// Used to properly detect and log drive enumerating wildcards when applicable.
string excludeFileSpec = string.Empty;
var filespecHasNoWildCards = !FilespecHasWildcards(filespecEscaped);
var filespecMatchesLazyWildcard = FilespecMatchesLazyWildcard(filespecEscaped, forceEvaluateWildCards);
var excludeSpecsAreEmpty = excludeSpecsEscaped?.Any() != true;
// Return original value if:
// FileSpec matches lazyloading regex or
// file has no wildcard and excludeSpecs are empty
if (filespecMatchesLazyWildcard || (filespecHasNoWildCards && excludeSpecsAreEmpty))
{
// Just return the original string.
fileList = [returnEscaped ? filespecEscaped : EscapingUtilities.UnescapeAll(filespecEscaped)];
}
else
{
if (Traits.Instance.LogExpandedWildcards)
{
ErrorUtilities.DebugTraceMessage("Expanding wildcard for file spec {0}", filespecEscaped);
}
// Unescape before handing it to the filesystem.
var directoryUnescaped = EscapingUtilities.UnescapeAll(directoryEscaped);
var filespecUnescaped = EscapingUtilities.UnescapeAll(filespecEscaped);
var excludeSpecsUnescaped = excludeSpecsEscaped?.Where(IsValidExclude).Select(i => EscapingUtilities.UnescapeAll(i)).ToList();
// Extract file spec information
FileMatcher.Default.GetFileSpecInfo(filespecUnescaped, out string directoryPart, out string wildcardPart, out string filenamePart, out bool needsRecursion, out bool isLegalFileSpec);
// Check if the file spec contains a drive-enumerating wildcard
bool logDriveEnumeratingWildcard = FileMatcher.IsDriveEnumeratingWildcardPattern(directoryPart, wildcardPart);
// Process exclude specs (if provided) and check if any of them contain a drive-enumerating wildcard
if (excludeSpecsUnescaped != null)
{
foreach (string excludeSpec in excludeSpecsUnescaped)
{
FileMatcher.Default.GetFileSpecInfo(excludeSpec, out directoryPart, out wildcardPart, out filenamePart, out needsRecursion, out isLegalFileSpec);
bool logDriveEnumeratingWildcardFromExludeSpec = FileMatcher.IsDriveEnumeratingWildcardPattern(directoryPart, wildcardPart);
if (logDriveEnumeratingWildcardFromExludeSpec)
{
excludeFileSpec = excludeSpec;
}
logDriveEnumeratingWildcard |= logDriveEnumeratingWildcardFromExludeSpec;
}
}
// Determines whether Exclude filespec or passed in file spec should be
// used in drive enumeration warning or exception.
bool excludeFileSpecIsEmpty = string.IsNullOrWhiteSpace(excludeFileSpec);
string fileSpec = excludeFileSpecIsEmpty ? filespecUnescaped : excludeFileSpec;
if (logDriveEnumeratingWildcard)
{
switch (loggingMechanism)
{
// Logging mechanism received from ItemGroupIntrinsicTask.
case TargetLoggingContext targetLoggingContext:
LogDriveEnumerationWarningWithTargetLoggingContext(
targetLoggingContext,
includeLocation,
excludeLocation,
excludeFileSpecIsEmpty,
disableExcludeDriveEnumerationWarning,
fileSpec);
break;
// Logging mechanism received from Evaluator.
case ILoggingService loggingService:
LogDriveEnumerationWarningWithLoggingService(
loggingService,
includeLocation,
buildEventContext,
buildEventFileInfoFullPath,
filespecUnescaped);
break;
// Logging mechanism received from Evaluator and LazyItemEvaluator.IncludeOperation.
case EvaluationLoggingContext evaluationLoggingContext:
LogDriveEnumerationWarningWithEvaluationLoggingContext(
evaluationLoggingContext,
importLocation,
includeLocation,
excludeLocation,
excludeFileSpecIsEmpty,
filespecUnescaped,
fileSpec);
break;
default:
throw new InternalErrorException(ResourceUtilities.FormatResourceStringIgnoreCodeAndKeyword(
"UnknownLoggingType",
loggingMechanism.GetType(),
nameof(GetFileList)));
}
}
if (logDriveEnumeratingWildcard && Traits.Instance.ThrowOnDriveEnumeratingWildcard)
{
switch (loggingMechanism)
{
// Logging mechanism received from ItemGroupIntrinsicTask.
case TargetLoggingContext targetLoggingContext:
ThrowDriveEnumerationExceptionWithTargetLoggingContext(
includeLocation,
excludeLocation,
excludeFileSpecIsEmpty,
filespecUnescaped,
fileSpec);
break;
// Logging mechanism received from Evaluator.
case ILoggingService loggingService:
ThrowDriveEnumerationExceptionWithLoggingService(includeLocation, filespecUnescaped);
break;
// Logging mechanism received from Evaluator and LazyItemEvaluator.IncludeOperation.
case EvaluationLoggingContext evaluationLoggingContext:
ThrowDriveEnumerationExceptionWithEvaluationLoggingContext(
importLocation,
includeLocation,
excludeLocation,
filespecUnescaped,
fileSpec,
excludeFileSpecIsEmpty);
break;
default:
throw new InternalErrorException(ResourceUtilities.FormatResourceStringIgnoreCodeAndKeyword(
"UnknownLoggingType",
loggingMechanism.GetType(),
nameof(GetFileList)));
}
}
else
{
// Get the list of actual files which match the filespec. Put
// the list into a string array. If the filespec started out
// as a relative path, we will get back a bunch of relative paths.
// If the filespec started out as an absolute path, we will get
// back a bunch of absolute paths
(fileList, _, _, string globFailure) = fileMatcher.GetFiles(directoryUnescaped, filespecUnescaped, excludeSpecsUnescaped);
// log globing failure with the present logging mechanism
if (globFailure != null)
{
switch (loggingMechanism)
{
case TargetLoggingContext targetLoggingContext:
targetLoggingContext.LogCommentFromText(MessageImportance.Low, globFailure);
break;
case ILoggingService loggingService:
loggingService.LogCommentFromText(buildEventContext, MessageImportance.Low, globFailure);
break;
case EvaluationLoggingContext evaluationLoggingContext:
evaluationLoggingContext.LogCommentFromText(MessageImportance.Low, globFailure);
break;
default:
throw new InternalErrorException(ResourceUtilities.FormatResourceStringIgnoreCodeAndKeyword(
"UnknownLoggingType",
loggingMechanism.GetType(),
nameof(GetFileList)));
}
}
ErrorUtilities.VerifyThrow(fileList != null, "We must have a list of files here, even if it's empty.");
// Before actually returning the file list, we sort them alphabetically. This
// provides a certain amount of extra determinism and reproducability. That is,
// we're sure that the build will behave in exactly the same way every time,
// and on every machine.
Array.Sort(fileList, StringComparer.OrdinalIgnoreCase);
if (returnEscaped)
{
// We must now go back and make sure all special characters are escaped because we always
// store data in the engine in escaped form so it doesn't interfere with our parsing.
// Note that this means that characters that were not escaped in the original filespec
// may now be escaped, but that's not easy to avoid.
for (int i = 0; i < fileList.Length; i++)
{
fileList[i] = EscapingUtilities.Escape(fileList[i]);
}
}
}
}
return fileList;
}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Priority:1Work that is critical for the release, but we could probably ship withouttriaged

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions