Add design for dotnetup installation tracking#52834
Add design for dotnetup installation tracking#52834dsplaisted wants to merge 3 commits intodotnet:release/dnupfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a design document that describes how dotnetup will track installations, handle updates, and support uninstalls. The design is based on a discussion from the runtime installation design PR (#52409) and proposes using a shared manifest to track install specifications and installed components.
Changes:
- Added a new design document describing the installation tracking mechanism for dotnetup
- Defined the structure for install specs, installations, and subcomponents in the shared manifest
- Outlined the implementation approach for install, update, delete, and garbage collection operations
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
|
||
| ## Implementation | ||
|
|
||
| ### Installing a component |
There was a problem hiding this comment.
We may want to note that all operations against the manifest shall require a lock, or we should think about the designing the manifest in a way that enables concurrency more effectively (e.g. last writer wins.)
There are many ways to design that lock:
- Simplest : Doing any installation at all requires holding the same lock
- Directory Specific Lock - the specific hive/directory determines the lock - multiple readers/writers to the same manifest but they are working in different directories.
- Install version specific lock - enable concurrency for numerous installs across process (or instead across 1 install)
The check for which installs exist should never try to operate without holding the lock as well, so another interesting conversation is whether we want to hold read/write specific locks. (e.g. read only lock checks existing installs, but we can't get the read only lock if the write lock is taken.)
There was a problem hiding this comment.
I think we would start simply, by having an exclusive lock. I don't know if there's much user need for multiple installs happening at once, and that would make it a lot more complicated.
We may want a reader/writer lock though so that multiple windows can list what's installed at once. But maybe it's OK if we don't, the commands should execute quickly so it would be OK if they were serialized.
There was a problem hiding this comment.
I agree, let's start simple. I think we should reflect this conversation in the doc and note how we could expand upon it later.
| - Component | ||
| - Version (this is the exact version that is installed) | ||
| - Dotnet root | ||
| - Subcomponents |
There was a problem hiding this comment.
Architecture is also important here
There was a problem hiding this comment.
Good point. I haven't figured out how we'll handle architecture. I think probably each dotnet root is architecture-specific, so maybe the architecture is stored or associated with the dotnet root.
There was a problem hiding this comment.
this should actually go into the install spec not installation
There was a problem hiding this comment.
We should Architecture to the install spec here
| - Component | ||
| - Version (this is the exact version that is installed) | ||
| - Dotnet root | ||
| - Subcomponents |
There was a problem hiding this comment.
I believe we may also want to record the hive in which it was installed so we can properly support --directory installs. That could point to the global json path instead to reduce the amount of properties and indicate the hive is controlled by the global.json (either default hive if no sdks path specified or that path in the global.json)
| - For install specs that came from a global.json file, update the versions in them if the global.json file has changed. Delete those specs if the global.json file has been deleted (or no longer specifies a version). | ||
| - For each install spec, find the latest installation record that matches. Mark that installation record to keep for this garbage collection. | ||
| - Delete all installation records from the manifest which weren't marked. | ||
| - Iterate through all components installed in the dotnet root. Remove any which are no longer listed under an installation record in the manifest. |
There was a problem hiding this comment.
If dotnet_root is set to a path including installs also managed by the user, this may cause us to delete those installs. I think we likely shouldn't do this step.
There was a problem hiding this comment.
Several related issues which I haven't really figured out or included in this design:
- How do we handle architecture?
- Should we include data for multiple dotnet roots in the same manifest or have a different manifest for each root? Having everything in one manifest would let us keep track of all installs centrally. But maybe it would be better to have the manifest in each dotnet root so that the paths can be relative and things don't break if you move the dotnet root folder around.
- Should we support installing into folders with an installation that was not tracked by dotnetup? We probably wouldn't be able to handle upgrades and installations well. Some options:
- Don't support installing into folders with existing non-dnup installs at all
- Install, but only in an "untracked" mode which lays down the files but doesn't add them to a manifest and doesn't support update and uninstall.
- Create a manifest and add information to it about all the existing files. Those files should never be deleted as part of garbage collection (unless the user chooses to delete all the pre-existing installs).
There was a problem hiding this comment.
We should have an individual manifest and not clean up dotnet roots by default that we don't own (not dotnetup's root.) Eventually we can have the option to also track the existing installs in the directory. There will always be a point in time problem where the user adds new installs but that's ok.
| - `packs/Microsoft.AspNetCore.App.Ref/10.0.2` - 3 levels | ||
| - `sdk-manifests/10.0.100/microsoft.net.sdk.android/36.1.2` - 4 levels | ||
|
|
||
| ## Implementation |
There was a problem hiding this comment.
A manifest version should also likely be included for compat / breaking changes.
|
|
||
| ## Dotnetup shared manifest contents | ||
|
|
||
| ### Install specs |
There was a problem hiding this comment.
We should likely provide a json schema for the manifest file as well.
| - `packs/Microsoft.AspNetCore.App.Ref/10.0.2` - 3 levels | ||
| - `sdk-manifests/10.0.100/microsoft.net.sdk.android/36.1.2` - 4 levels | ||
|
|
||
| ## Implementation |
There was a problem hiding this comment.
Question: Do we fail if the manifest doesn't exist? I think we don't. We can try to recreate the manifest from a hive but it will never be the same - not including custom directories nor the intent of a user (e.g. explicit runtime install request vs sdk install runtime)
I also wonder if we should keep backup copies of the manifests, say in the temp folder, in case something goes wrong so we can restore the manifest. Then we get to the interesting question of 'history' like workload history but we don't need to do that for now 😁 This is probably better
|
we should likely incorporate this comment in the spec #52792 (comment) |
| ### Dotnet roots | ||
|
|
||
| - Path to dotnet root | ||
| - Architecture of installs in the root |
There was a problem hiding this comment.
I think adding architecture to the dotnet root is interesting and could reduce duplicated information. But one question I have is what data structure in the json we will use to represent the root. Will it have a directory and then the architecture? Will it have a management cadence (as defined by our current type)? Or will it have some other enum for representing whether we own the directory or not? I don't think management cadence works because global.json managed installs vs locked update based installs can both go in the same main dotnetup hive.
This also adds some level of complexity in terms of comparing / loading into memory. Should we have a section in the manifest that just has dotnet roots, and then a section for installations, that hold an enum to that dotnet_root? How do we represent that - does the dotnet root get a guid to prevent duplicating the path information?
We alternatively can keep the directory with each install instead of keeping a list of dotnet_roots - I think it's important to consider the tradeoffs of that in this doc. This would also have the full detail, because we'd simply scan each install and see which directory it lived in.
nagilson
left a comment
nagilson
left a comment
There was a problem hiding this comment.
Please see my remaining feedback from now/last time - this has led to some great discussion!
2 participants
Based on discussion we had on the runtime installation design, here's a proposal for how dotnetup should handle installation tracking, updates, and uninstalls.