Skip to content

Extension point for package manager. #2667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions VisualFSharp.sln
Original file line number Diff line number Diff line change
Expand Up @@ -280,8 +280,9 @@ Global
{6196B0F8-CAEA-4CF1-AF82-1B520F77FE44}.Release|Any CPU.Build.0 = Release|Any CPU
{6196B0F8-CAEA-4CF1-AF82-1B520F77FE44}.Release|x86.ActiveCfg = Release|Any CPU
{6196B0F8-CAEA-4CF1-AF82-1B520F77FE44}.Release|x86.Build.0 = Release|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Debug|Any CPU.Build.0 = Debug|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Debug|Any CPU.ActiveCfg = Release|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Debug|Any CPU.Build.0 = Release|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Debug|Any CPU.Deploy.0 = Release|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Debug|x86.ActiveCfg = Debug|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Debug|x86.Build.0 = Debug|Any CPU
{FBD4B354-DC6E-4032-8EC7-C81D8DFB1AF7}.Proto|Any CPU.ActiveCfg = Proto|Any CPU
Expand Down
133 changes: 102 additions & 31 deletions src/fsharp/CompileOps.fs

Large diffs are not rendered by default.

6 changes: 4 additions & 2 deletions src/fsharp/CompileOps.fsi
Original file line number Diff line number Diff line change
Expand Up @@ -263,9 +263,11 @@ type TcConfigBuilder =
mutable light: bool option
mutable conditionalCompilationDefines: string list
/// Sources added into the build with #load
mutable loadedSources: (range * string) list
mutable loadedSources: (range * string * string) list

mutable referencedDLLs: AssemblyReference list
mutable packageManagerLines : Map<string,(string*range) list>

mutable projectReferences : IProjectReference list
mutable knownUnresolvedReferences : UnresolvedAssemblyReference list
optimizeForMemory: bool
Expand Down Expand Up @@ -666,7 +668,7 @@ val RequireDLL : CompilationThreadToken * TcImports * TcEnv * thisAssemblyName:

/// Processing # commands
val ProcessMetaCommandsFromInput :
(('T -> range * string -> 'T) * ('T -> range * string -> 'T) * ('T -> range * string -> unit))
(('T -> range * string -> 'T) * ('T -> range * string -> 'T) * ('T -> DependencyManagerIntegration.IDependencyManagerProvider * range * string -> 'T) * ('T -> range * string -> unit))
-> TcConfigBuilder * Ast.ParsedInput * string * 'T
-> 'T

Expand Down
228 changes: 228 additions & 0 deletions src/fsharp/DependencyManager.Integration.fs
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
// Copyright (c) Microsoft Corporation. All Rights Reserved. Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information.

/// Helper members to integrate DependencyManagers into F# codebase
module internal Microsoft.FSharp.Compiler.DependencyManagerIntegration

open System
open System.Reflection
open System.IO
open Microsoft.FSharp.Compiler.ErrorLogger

#if FX_RESHAPED_REFLECTION
open Microsoft.FSharp.Core.ReflectionAdapters
#endif

// NOTE: this contains mostly members whose intents are :
// * to keep ReferenceLoading.PaketHandler usable outside of F# (so it can be used in scriptcs & others)
// * to minimize footprint of integration in fsi/CompileOps

/// hardcoded to net461 as we don't have fsi on netcore
let targetFramework = "net461"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this, and does it need to be hard-coded?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is remnant from hardcoded parameter we need to give in case of paket.

Notice that IDependencyManagerProvider.ResolveDependencies takes a targetFramework parameter, which has been this value so far.

If we don't give a single framework, it makes it difficult for paket to identify which is intended framework.

I think this should come from tooling, similarly that we have an issue of intellisense always working with latest .net framework rather than having an user option to specify it.


/// reflection helpers
module ReflectionHelper =
let assemblyHasAttributeNamed (theAssembly: Assembly) attributeName =
try
theAssembly.GetCustomAttributes false
|> Seq.tryFind (fun a -> a.GetType().Name = attributeName)
|> function | Some _ -> true | _ -> false
with | _ -> false

let getAttributeNamed (theType: Type) attributeName =
try
theType.GetCustomAttributes false
|> Seq.tryFind (fun a -> a.GetType().Name = attributeName)
with | _ -> None

let getInstanceProperty<'treturn> (theType: Type) indexParameterTypes propertyName =
try
let property = theType.GetProperty(propertyName, typeof<'treturn>)
if isNull property then
None
elif not (property.GetGetMethod().IsStatic)
&& property.GetIndexParameters() = indexParameterTypes
then
Some property
else
None
with | _ -> None

let getInstanceMethod<'treturn> (theType: Type) (parameterTypes: Type array) methodName =
try
let theMethod = theType.GetMethod(methodName, parameterTypes)
if isNull theMethod then
None
else
Some theMethod
with | _ -> None

let implements<'timplemented> (theType: Type) =
typeof<'timplemented>.IsAssignableFrom(theType)

/// Contract for dependency anager provider. This is a loose contract for now, just to define the shape,
/// it is resolved through reflection (ReflectionDependencyManagerProvider)
type internal IDependencyManagerProvider =
inherit System.IDisposable
abstract Name : string
abstract ToolName: string
abstract Key: string
abstract ResolveDependencies : targetFramework: string * scriptDir: string * scriptName: string * packageManagerTextLines: string seq -> string option * string list

/// Reference
[<RequireQualifiedAccess>]
type ReferenceType =
| RegisteredDependencyManager of IDependencyManagerProvider
| Library of string
| UnknownType

/// Dependency Manager Provider using dotnet reflection
type ReflectionDependencyManagerProvider(theType: Type, nameProperty: PropertyInfo, toolNameProperty: PropertyInfo, keyProperty: PropertyInfo, resolveDeps: MethodInfo) =
let instance = Activator.CreateInstance(theType) :?> IDisposable
let nameProperty = nameProperty.GetValue >> string
let toolNameProperty = toolNameProperty.GetValue >> string
let keyProperty = keyProperty.GetValue >> string
static member InstanceMaker (theType: System.Type) =
if not (ReflectionHelper.implements<IDisposable> theType) then None
else
match ReflectionHelper.getAttributeNamed theType "FSharpDependencyManagerAttribute" with
| None -> None
| Some _ ->
match ReflectionHelper.getInstanceProperty<string> theType Array.empty "Name",
ReflectionHelper.getInstanceProperty<string> theType Array.empty "ToolName",
ReflectionHelper.getInstanceProperty<string> theType Array.empty "Key",
ReflectionHelper.getInstanceMethod<string * string list> theType [|typeof<string>;typeof<string>;typeof<string>;typeof<string seq>;|] "ResolveDependencies"
with
| Some nameProperty, Some toolNameProperty, Some keyProperty, Some resolveDependenciesMethod ->
Some (fun () -> new ReflectionDependencyManagerProvider(theType, nameProperty, toolNameProperty, keyProperty, resolveDependenciesMethod) :> IDependencyManagerProvider)
| _ -> None

interface IDependencyManagerProvider with
member __.Name = instance |> nameProperty
member __.ToolName = instance |> toolNameProperty
member __.Key = instance |> keyProperty
member __.ResolveDependencies(targetFramework, scriptDir, scriptName, packageManagerTextLines) =
let arguments = [|box targetFramework; box scriptDir; box scriptName; box packageManagerTextLines|]
resolveDeps.Invoke(instance, arguments) :?> _

interface IDisposable with
member __.Dispose () = instance.Dispose()


let assemblySearchLocations (additionalIncludePaths:string list) =
additionalIncludePaths @
[
yield Path.GetDirectoryName typeof<IDependencyManagerProvider>.Assembly.Location
#if FX_NO_APP_DOMAINS
yield AppContext.BaseDirectory
#else
yield AppDomain.CurrentDomain.BaseDirectory
#endif
] |> List.distinct

let enumerateDependencyManagerAssembliesFromCurrentAssemblyLocation (additionalIncludePaths:string list) =
/// Where to search for providers
/// Algorithm TBD
/// 1. Directory containing FSharp.Compiler.dll
/// 2. AppContext (AppDomain on desktop) Base directory
/// 3. directories supplied using --lib
(assemblySearchLocations additionalIncludePaths)
|> Seq.collect (fun path -> Directory.EnumerateFiles(path,"*DependencyManager*.dll"))
|> Seq.choose (fun path -> try Assembly.LoadFrom path |> Some with | _ -> None)
|> Seq.filter (fun a -> ReflectionHelper.assemblyHasAttributeNamed a "FSharpDependencyManagerAttribute")

/// TBD
type ProjectDependencyManager() =
interface IDependencyManagerProvider with
member __.Name = "Project loader"
member __.ToolName = ""
member __.Key = "project"
member __.ResolveDependencies(_targetFramework:string, _scriptDir: string, _scriptName: string, _packageManagerTextLines: string seq) =
None,[]

interface System.IDisposable with
member __.Dispose() = ()

/// Contract for DependencyManager for #r assemblies
type RefDependencyManager() =
interface IDependencyManagerProvider with
member __.Name = "Library loader"
member __.ToolName = ""
member __.Key = "ref"
member __.ResolveDependencies(_targetFramework:string, _scriptDir: string, _scriptName: string, _packageManagerTextLines: string seq) =
None,[]

interface System.IDisposable with
member __.Dispose() = ()

/// Get the list of registered DependencyManagers
let registeredDependencyManagers (additionalIncludePaths: string list) =
let dependencyManagers =
lazy (
let defaultProviders =
[new ProjectDependencyManager() :> IDependencyManagerProvider
new RefDependencyManager() :> IDependencyManagerProvider ]

let loadedProviders =
enumerateDependencyManagerAssembliesFromCurrentAssemblyLocation(additionalIncludePaths)
|> Seq.collect (fun a -> a.GetTypes())
|> Seq.choose ReflectionDependencyManagerProvider.InstanceMaker
|> Seq.map (fun maker -> maker ())

defaultProviders
|> Seq.append loadedProviders
|> Seq.map (fun pm -> pm.Key, pm)
|> Map.ofSeq
)
dependencyManagers.Force()

/// Issue PackageManner error
let createPackageManagerUnknownError packageManagerKey m (additionalIncludePaths: string list) =
let registeredKeys = String.Join(", ", registeredDependencyManagers(additionalIncludePaths) |> Seq.map (fun kv -> kv.Value.Key))
let searchPaths = assemblySearchLocations additionalIncludePaths
Error(FSComp.SR.packageManagerUnknown(packageManagerKey, String.Join(", ", searchPaths), registeredKeys),m)

/// Issue Look for a packagemanager given a #r path. (Path may contain a package manager moniker 'nuget', 'paket' followed by ':'
/// or be a fully qualified Windows path 'C:...', a relative path or a UNC qualified path)
let tryFindDependencyManagerInPath m (path:string) (additionalIncludePaths: string list): ReferenceType =
try
match path.IndexOf(":") with
| -1 | 1 ->
ReferenceType.Library path
| _ ->
let managers = registeredDependencyManagers(additionalIncludePaths)
match managers |> Seq.tryFind (fun kv -> path.StartsWith(kv.Value.Key + ":" )) with
| None ->
errorR(createPackageManagerUnknownError (path.Split(':').[0]) m additionalIncludePaths)
ReferenceType.UnknownType
| Some kv -> (ReferenceType.RegisteredDependencyManager kv.Value)
with
| e ->
errorR(Error(FSComp.SR.packageManagerError(e.Message),m))
ReferenceType.UnknownType

let removeDependencyManagerKey (packageManagerKey:string) (path:string) = path.Substring(packageManagerKey.Length + 1).Trim()

let tryFindDependencyManagerByKey m (key:string) (additionalIncludePaths: string list): IDependencyManagerProvider option =
try
registeredDependencyManagers(additionalIncludePaths) |> Map.tryFind key
with
| e ->
errorR(Error(FSComp.SR.packageManagerError(e.Message),m))
None

let resolve (packageManager:IDependencyManagerProvider) implicitIncludeDir fileName m packageManagerTextLines =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we have /// comments on all new top-level let , type and module please? thanks

try
let loadScript,additionalIncludeFolders =
packageManager.ResolveDependencies(
targetFramework,
implicitIncludeDir,
fileName,
packageManagerTextLines)

Some(loadScript,additionalIncludeFolders)
with e ->
if e.InnerException <> null then
errorR(Error(FSComp.SR.packageManagerError(e.InnerException.Message),m))
else
errorR(Error(FSComp.SR.packageManagerError(e.Message),m))
None
29 changes: 29 additions & 0 deletions src/fsharp/DependencyManager.Integration.fsi
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// Copyright (c) Microsoft Corporation. All Rights Reserved. Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information.

/// Helper members to integrate DependencyManagers into F# codebase
module internal Microsoft.FSharp.Compiler.DependencyManagerIntegration

open Microsoft.FSharp.Compiler.Range

/// Contract for dependency anager provider. This is a loose contract for now, just to define the shape,
/// it is resolved through reflection (ReflectionDependencyManagerProvider)
type IDependencyManagerProvider =
inherit System.IDisposable
abstract Name : string
abstract ToolName: string
abstract Key: string
abstract ResolveDependencies : string * string * string * string seq -> string option * string list

/// Reference
[<RequireQualifiedAccess>]
type ReferenceType =
| RegisteredDependencyManager of IDependencyManagerProvider
| Library of string
| UnknownType

val registeredDependencyManagers : string list -> Map<string,IDependencyManagerProvider>
val tryFindDependencyManagerInPath : range -> string -> string list -> ReferenceType
val tryFindDependencyManagerByKey : range -> string -> string list -> IDependencyManagerProvider option
val removeDependencyManagerKey : string -> string -> string
val createPackageManagerUnknownError : string -> range -> string list -> exn
val resolve : IDependencyManagerProvider -> string -> string -> range -> string seq -> (string option * string list) option
2 changes: 2 additions & 0 deletions src/fsharp/FSComp.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1331,6 +1331,8 @@ tcGlobalsSystemTypeNotFound,"The system type '%s' was required but no referenced
3213,typrelMemberHasMultiplePossibleDispatchSlots,"The member '%s' matches multiple overloads of the same method.\nPlease restrict it to one of the following:%s."
3214,methodIsNotStatic,"Method or object constructor '%s' is not static"
3215,parsUnexpectedSymbolEqualsInsteadOfIn,"Unexpected symbol '=' in expression. Did you intend to use 'for x in y .. z do' instead?"
3216,packageManagerUnknown,"Package manager key '%s' was not registered in %s. Currently registered: %s"
3217,packageManagerError,"%s"
keywordDescriptionAbstract,"Indicates a method that either has no implementation in the type in which it is declared or that is virtual and has a default implementation."
keyworkDescriptionAnd,"Used in mutually recursive bindings, in property declarations, and with multiple constraints on generic parameters."
keywordDescriptionAs,"Used to give the current class object an object name. Also used to give a name to a whole pattern within a pattern match."
Expand Down
6 changes: 6 additions & 0 deletions src/fsharp/FSharp.Compiler-proto/FSharp.Compiler-proto.fsproj
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,12 @@
<Compile Include="..\IlxGen.fs">
<Link>IlxGen.fs</Link>
</Compile>
<Compile Include="..\DependencyManager.Integration.fsi">
<Link>Driver\DependencyManager.Integration.fsi</Link>
</Compile>
<Compile Include="..\DependencyManager.Integration.fs">
<Link>Driver\DependencyManager.Integration.fs</Link>
</Compile>
<Compile Include="..\CompileOps.fsi">
<Link>CompileOps.fsi</Link>
</Compile>
Expand Down
8 changes: 7 additions & 1 deletion src/fsharp/FSharp.Compiler/FSharp.Compiler.fsproj
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,12 @@
<Compile Include="..\IlxGen.fs">
<Link>CodeGen\IlxGen.fs</Link>
</Compile>
<Compile Include="..\DependencyManager.Integration.fsi">
<Link>Driver\DependencyManager.Integration.fsi</Link>
</Compile>
<Compile Include="..\DependencyManager.Integration.fs">
<Link>Driver\DependencyManager.Integration.fs</Link>
</Compile>
<Compile Include="..\CompileOps.fsi">
<Link>Driver\CompileOps.fsi</Link>
</Compile>
Expand Down Expand Up @@ -593,4 +599,4 @@
<Name>FSharp.Core</Name>
</ProjectReference>
</ItemGroup>
</Project>
</Project>
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) Microsoft Corporation. All Rights Reserved. Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information. -->
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
Expand Down Expand Up @@ -475,6 +475,12 @@
<Compile Include="..\IlxGen.fs">
<Link>CodeGen/IlxGen.fs</Link>
</Compile>
<Compile Include="..\DependencyManager.Integration.fsi">
<Link>Driver\DependencyManager.Integration.fsi</Link>
</Compile>
<Compile Include="..\DependencyManager.Integration.fs">
<Link>Driver\DependencyManager.Integration.fs</Link>
</Compile>
<Compile Include="..\CompileOps.fsi">
<Link>Driver\CompileOps.fsi</Link>
</Compile>
Expand Down
Loading