Skip to content

Commit

Permalink
Tailor state possibility checking based on relationship with conflict
Browse files Browse the repository at this point in the history 
Improve unwinding algorithm to handle different relationships with conflict
- If a requirment that directly caused a conflict has alternative possibilities,
  check whether any of those possibilities will fix the conflict
- If a requirement's parent has alternative possibilities, check whether any of
  those possibilities would lead to a different requirement being created

Also improves the possibility filtering we do after unwinding
- Previously we filtered our unsatisfactory possibility sets when unwinding to a
  state that directly caused the conflict
- Now we will also filter out unsatisfactory possibility sets when unwinding to
  a state that is the parent of a state that caused the conflict

Since Unwinding has become an increasingly complicated process, this commit also
adds details of the above to the ARCHITECTURE.md file.
  • Loading branch information
@greysteil
greysteil committed Jul 24, 2017
1 parent a2be120 commit 94d4f48
Show file tree
Hide file tree
Showing 2 changed files with 157 additions and 64 deletions.
43 changes: 43 additions & 0 deletions ARCHITECTURE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,49 @@ This stack-based approach is used because backtracking (also known as *unwinding
15. Terminate with the topmost state's dependency graph when there are no more requirements left
16. For each vertex with a payload of allowable versions for this resolution (i.e., a `PossibilitySet`), pick a single specific version.

### Optimal unwinding

For our backtracking algorithm to be efficient as well as correct, we need to
unwind efficiently after a conflict is encountered. Unwind too far and we'll
miss valid resolutions - once we unwind passed a DependencyState we can never
get there again. Unwind too little and resolution will be extremely slow - we'll
repeatedly hit the same conflict, processing many unnecessary iterations before
getting to a branch that avoids it.

To unwind the optimal amount, we consider each of the conflicts that have
determined our current state, and the requirements that led to them. For each
conflict we:

1. Discard any requirements that aren't "binding" - that is, any that are
unnecessary to cause the current conflict. We do this by looping over through
the requirements in revers order to how they were added, removing each one if it
is no necessary
2. Loop through the array of binding requirements, looking for the highest index
state that has possibilities that may still lead to a resolution. For each
requirement:
- check if the DependencyState responsible for the requirement has alternative
possibilities that would satisfy all the other requirements. If so, the index
of that state becomes our new candidate index (if it is higher than the
current one)
- if no alternative possibilities existed for that state responsible for the
requirement, check the state's parent. Look for alternative possibilities that
parent state could take that would mean the requirement would never have been
created. If any exist, that parent state's index becomes our candidate index
(if it is higher than the current one)
- if no alternative possibilities for the parent of the state responsible for
the requirement, look at that state's parent and so forth, until we reach a
state with no alternative possibilities and no parents

We then take the highest index found for any of the past conflicts, unwind to
it, and prune out any possibilities on it that we now know would lead to the
same conflict we just encountered. If no index to unwind to is found for any
of the conflicts that determine our state, we throw a VersionConflict error, as
resolution must not be possible.

Finally, once an unwind has taken place, we use the additional information from
the conflict we unwound from to filter the possibilities for the sate we have
unwound to in `filter_possibilities_after_unwind`.

## Specification Provider

The `SpecificationProvider` module forms the basis for the key integration point for a client library with Molinillo.
Expand Down
178 changes: 114 additions & 64 deletions lib/molinillo/resolution.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -220,84 +220,114 @@ def initial_state
# Unwinds the states stack because a conflict has been encountered
# @return [void]
def unwind_for_conflict
debug(depth) { "Unwinding for conflict: #{requirement} to #{state_index_for_unwind / 2}" }
details_for_unwind = build_details_for_unwind
debug(depth) { "Unwinding for conflict: #{requirement} to #{details_for_unwind[:index] / 2}" }
conflicts.tap do |c|
conflict_being_unwound = c[name]
sliced_states = states.slice!((state_index_for_unwind + 1)..-1)
sliced_states = states.slice!((details_for_unwind[:index] + 1)..-1)
raise VersionConflict.new(c) unless state
activated.rewind_to(sliced_states.first || :initial_state) if sliced_states
state.conflicts = c
filter_possibilities_after_unwind(conflict_being_unwound)
filter_possibilities_after_unwind(details_for_unwind)
index = states.size - 1
@parents_of.each { |_, a| a.reject! { |i| i >= index } }
end
end

# @return [Integer] The index to which the resolution should unwind in the
# case of conflict.
def state_index_for_unwind
index = -1
# @return [Hash] Details of the nearest index to which we could unwind
def build_details_for_unwind
# Process the current conflict first, as it's like to produce the highest
# index, allowing us to short-circuit subsequent checks
current_conflict = conflicts[name]
unwind_details = unwind_details_for_conflict(current_conflict)
return unwind_details if unwind_details[:index] == states.size - 2

# Process previous conflicts
(conflicts.values - [current_conflict]).each do |conflict|
unwind_details = unwind_details_for_conflict(conflict, unwind_details)
end

unwind_details
end

# @param [Conflict] conflict to be unwound from
# @param [Hash] details of the currently proposed unwind details
# @return [Hash] Details of the nearest index to which we could unwind to
# resolve the given conflict conflict
# rubocop:disable Metrics/CyclomaticComplexity
def unwind_details_for_conflict(conflict, existing_details = { :index => -1 })
maximal_index = states.size - 2
conflicts.each do |dependency_name, conflict|
# To avoid this conflict, we need to rewind to one of:
# 1) the state where the requirement was activated (and therefore its
# current PossibilitySet chosen), or its parent, OR
# 2) a state that added a binding requirement in the conflict, or its
# parent.
# There's no point re-winding to states that added non-binding
# requirements to the conflict - doing so would only result in us
# encountering the same conflict again, possibly after processing many
# wasted steps.
requirements_to_attempt_to_relax = binding_requirements_for_conflict(conflict).
unshift(initial_requirement_for_name(dependency_name)).
tap(&:uniq!)

requirements_to_attempt_to_relax.each do |r|
until r.nil?
return index if index == maximal_index
current_state = find_state_for(r)
if conflict_fixing_possibilities?(current_state, conflict)
current_index = states.index(current_state)
index = current_index if current_index > index
break
end
r = parent_of(r)
unwind_details = existing_details.dup

binding_requirements = binding_requirements_for_conflict(conflict)
binding_requirements.reverse_each do |r|
# If this requirement has alternative possibilities, check if any would
# satisfy the other requirements that created this conflict
requirement_state = find_state_for(r)
candidate_index = states.index(requirement_state)
next if candidate_index && candidate_index < unwind_details[:index]
if conflict_fixing_possibilities?(requirement_state, binding_requirements)
unwind_details[:index] = candidate_index
unwind_details[:relationship] = :primary
unwind_details[:requirement] = r
unwind_details[:conflicting_requirements] = binding_requirements
return unwind_details if unwind_details[:index] == maximal_index

next # No need to look at this requirement's parent, as it couldn't have a higher index
end

# Next, look at the parent of this requirement, and check if the requirement
# could have been avoided if an alternative PossibilitySet had been chosen
parent_r = parent_of(r)
requirement_state = find_state_for(parent_r)
candidate_index = states.index(requirement_state)
next unless candidate_index && candidate_index >= unwind_details[:index]
if requirement_state &&
requirement_state.possibilities.any? { |set| set.dependencies.none? { |dep| dep == r } }
if candidate_index > unwind_details[:index] || unwind_details[:relationship] == :primary
unwind_details[:index] = candidate_index
unwind_details[:relationship] = :parent
unwind_details[:requirement] = r
unwind_details[:conflicting_requirements] = binding_requirements
end

next # No need to look at this requirement's grandparent, as it couldn't have a higher index
end
end

index
end
# Finally, look at the grandparent and up of this requirement, looking
# for any possibilities at all - we're now too far removed from the conflict
# to know how any such possibility will affect it, so just have to try it
grandparent_r = parent_of(parent_r)
until grandparent_r.nil?
requirement_state = find_state_for(grandparent_r)
candidate_index = states.index(requirement_state)
break unless candidate_index && candidate_index >= unwind_details[:index]
if requirement_state && !requirement_state.possibilities.empty?
unwind_details[:index] = candidate_index
unwind_details[:relationship] = :grandparent
unwind_details[:requirement] = r
unwind_details[:conflicting_requirements] = binding_requirements
break
end
grandparent_r = parent_of(grandparent_r)
end
end

# @param [String] name of a dependency
# @return [Object] the requirement that led to a version of a possibility
# with the given name being activated.
def initial_requirement_for_name(name)
return nil unless activated.vertex_named(name)
return nil unless activated.vertex_named(name).payload
states.find { |s| s.name == name }.requirement
unwind_details
end

# @param [DependencyState] state
# @param [Conflict] conflict
# @param [Array] array of requirements
# @return [Boolean] whether or not the given state has any possibilities
# that could fix or avoid the given conflict
def conflict_fixing_possibilities?(state, conflict)
return false unless state && !state.possibilities.empty?

# If the state introduces a requirement that caused the conflict, we
# need to check the possibilities fix the conflict. Otherwise we can
# return true (optimistically)
return true unless name_for(conflict.requirement) == state.name

all_requirements = conflict.requirements.values.flatten(1).uniq
# that could satisfy the given requirements
def conflict_fixing_possibilities?(state, binding_requirements)
return false unless state

state.possibilities.any? do |possibility_set|
possibility_set.possibilities.any? do |poss|
activated.tag(:swap)
name = name_for(poss)
activated.set_payload(name, poss) if activated.vertex_named(name)
satisfied = all_requirements.all? do |r|
satisfied = binding_requirements.all? do |r|
requirement_satisfied_by?(r, activated, poss)
end
activated.rewind_to(:swap)
Expand All @@ -308,16 +338,23 @@ def conflict_fixing_possibilities?(state, conflict)

# Filter's a state's possibilities to remove any that would not fix the
# conflict we've just rewound from
# @param [Conflict] conflict just unwound from
# @param [Hash] details of the conflict just unwound from
# @return [void]
def filter_possibilities_after_unwind(conflict)
def filter_possibilities_after_unwind(unwind_details)
return unless state && !state.possibilities.empty?

# If the state we've rewound to didn't introduce a requirement that
# caused the conflict, just no-op
return unless name_for(conflict.requirement) == state.name
case unwind_details[:relationship]
when :primary then filter_possibilities_for_primary_unwind(unwind_details)
when :parent then filter_possibilities_for_parent_unwind(unwind_details)
end
end

all_requirements = conflict.requirements.values.flatten(1).uniq
# Filter's a state's possibilities to remove any that would not satisfy
# the requirements in the conflict we've just rewound from
# @param [Hash] details of the conflict just unwound from
# @return [void]
def filter_possibilities_for_primary_unwind(unwind_details)
all_requirements = unwind_details[:conflicting_requirements]

state.possibilities.reject! do |possibility_set|
possibility_set.possibilities.none? do |poss|
Expand All @@ -333,6 +370,16 @@ def filter_possibilities_after_unwind(conflict)
end
end

# Filter's a state's possibilities to remove any that would create a
# specific requirement in the conflict we've just rewound from
# @param [Hash] details of the conflict just unwound from
# @return [void]
def filter_possibilities_for_parent_unwind(unwind_details)
state.possibilities.reject! do |possibility_set|
possibility_set.dependencies.include?(unwind_details[:requirement])
end
end

# @param [Conflict] conflict
# @return [Array] minimal array of requirements that would cause the passed
# conflict to occur.
Expand All @@ -343,11 +390,14 @@ def binding_requirements_for_conflict(conflict)
possible_binding_requirements = conflict.requirements.values.flatten(1).uniq

# Loop through the possible binding requirements, removing each one
# that doesn't bind. Use an `each` rather than a `reject!` as we wish
# to refine the array *on each iteration*.
possible_binding_requirements.reject! do |req|
# that doesn't bind. Use a `reverse_each` as we want the earliest set of
# binding requirements, and don't use `reject!` as we wish to refine the
# array *on each iteration*.
possible_binding_requirements.reverse_each do |req|
next false if req == conflict.requirement
!binding_requirement_in_set?(req, possible_binding_requirements, possibilities)
unless binding_requirement_in_set?(req, possible_binding_requirements, possibilities)
possible_binding_requirements -= [req]
end
end

possible_binding_requirements
Expand Down

0 comments on commit 94d4f48

Please sign in to comment.