Skip to content

Commit

Permalink
479 do not backtranslate fancy sorts in get-model (#563)
Browse files Browse the repository at this point in the history
Values of any sort other than `SortBool` and `SortInt` cannot currently
be back-translated from the SMT representation (all sorts unknown to the
solver are opaque). In particular, this is true for `Map`-sorted values
(which are used frequently in EVM).

* Do not attempt to back-translate values of sorts other than `SortBool`
and `SortInt`
* but still include the related SMT variables in the `get-values`
requestl;
* and indicate untranslated variables by adding `VAR = VAR` to the
returned substitution.

As shown in the added integration test, there are cases where the
`Map`-sorted value is relevant, but cannot currently be returned. We
might consider modeling `Map`s specifically as arrays in the SMT solver
in the future.

Fixes runtimeverification/kontrol#479
  • Loading branch information
jberthold authored Mar 27, 2024
1 parent 1653f52 commit 248f67b
Show file tree
Hide file tree
Showing 12 changed files with 1,597 additions and 573 deletions.
36 changes: 25 additions & 11 deletions library/Booster/SMT/Interface.hs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import Data.Map qualified as Map
import Data.Set (Set)
import Data.Set qualified as Set
import Data.Text as Text (Text, pack, unlines, unwords)
import Prettyprinter (Pretty, pretty, vsep)
import Prettyprinter (Pretty, hsep, pretty, vsep)

import Booster.Definition.Base
import Booster.Pattern.Base
Expand Down Expand Up @@ -149,13 +149,6 @@ getModelFor ctxt ps subst
when (isLeft translated) $
smtTranslateError (fromLeft' translated)
let (smtAsserts, transState) = fromRight' translated
freeVarsMap =
Map.filterWithKey
(const . (`Set.member` Set.map Var freeVars))
transState.mappings
getVar (Var v) = v
getVar other = smtTranslateError . pack $ "Solver returned non-var in translation state: " <> show other
freeVarsToSExprs = Map.mapKeys getVar $ Map.map Atom freeVarsMap

runCmd_ SMT.Push -- assuming the prelude has been run already,

Expand Down Expand Up @@ -192,10 +185,29 @@ getModelFor ctxt ps subst
runCmd_ SMT.Pop
throwSMT' $ "Unexpected SMT response to CheckSat: " <> show satResponse
Sat -> do
let freeVarsMap =
Map.map Atom . Map.mapKeys getVar $
Map.filterWithKey
(const . (`Set.member` Set.map Var freeVars))
transState.mappings
getVar (Var v) = v
getVar other =
smtTranslateError . pack $
"Solver returned non-var in translation state: " <> show other
sortsToTranslate = Set.fromList [SortInt, SortBool]

(freeVarsToSExprs, untranslatableVars) =
Map.partitionWithKey
(const . ((`Set.member` sortsToTranslate) . (.variableSort)))
freeVarsMap
unless (Map.null untranslatableVars) $
let vars = Pretty.renderText . hsep . map pretty $ Map.keys untranslatableVars
in logSMT ("Untranslatable variables in model: " <> vars)

response <-
if Map.null freeVarsToSExprs
if Map.null freeVarsMap
then pure $ Values []
else runCmd $ GetValue (Map.elems freeVarsToSExprs)
else runCmd $ GetValue (Map.elems freeVarsMap)
runCmd_ SMT.Pop
case response of
Error msg ->
Expand All @@ -205,8 +217,10 @@ getModelFor ctxt ps subst
Map.partition isLeft
. Map.map (valueToTerm transState)
$ Map.compose (Map.fromList pairs) freeVarsToSExprs
untranslated =
Map.mapWithKey (const . Var) untranslatableVars
in if null errors
then pure $ Right $ Map.map fromRight' values
then pure $ Right $ Map.map fromRight' values <> untranslated
else
throwSMT . Text.unlines $
( "SMT errors while converting results: "
Expand Down
12 changes: 12 additions & 0 deletions test/rpc-integration/resources/int-and-bool.k
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
module TEST
imports INT
imports BOOL
imports MAP

// Int map definition from evm-semantics
syntax Int ::= #lookup ( Map, Int) [symbol(#lookup), function, total, smtlib(lookup)]
//-------------------------------------------------------------------------------------
rule [#lookup.some]:
#lookup( (KEY |-> VAL:Int) _M, KEY) => VAL
rule [#lookup.none]:
#lookup( M, KEY) => 0 requires notBool KEY in_keys(M)
rule [#lookup.notInt]:
#lookup( (KEY |-> VAL) _M, KEY) => VAL requires notBool isInt(VAL)
//-------------------------------------------------------------------------------------

endmodule
1,086 changes: 525 additions & 561 deletions test/rpc-integration/resources/int-and-bool.kore

Large diffs are not rendered by default.

30 changes: 29 additions & 1 deletion test/rpc-integration/test-get-model/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Testing `get-model`

For the time being, this is testing the pass-through of `get-model` requests to the `kore-rpc`. At the time of writing, all tests are [replicated from `haskell-backend`](https://github.com/runtimeverification/haskell-backend/tree/master/test/rpc-server/get-model) test suite without modifications.
Tests `get-model` requests. Many of these tests were [replicated from the `haskell-backend`](https://github.com/runtimeverification/haskell-backend/tree/master/test/rpc-server/get-model) test suite without modifications.

* an input without a predicate:
- Input kore term: `\and(notBool(X:SortBool), X:SortBool)` , interpreted as a _term_, not as a predicate. Therefore, the input does not have any predicate, and the server returns an indeterminate result.
Expand All @@ -20,3 +20,31 @@ For the time being, this is testing the pass-through of `get-model` requests to
* a predicate that causes the SMT solver to return `Unknown`
- Input kore term: `X == X ^ 256`
- Result: `unknown`, no substitution

Tests involving variables that cannot be back-translated

* `with-map`: satisfiable predicate involving an irrelevant `Map` lookup
- Input predicates:
- `Map.lookup(?STORAGE, KEY) < Map.lookup(?STORAGE, KEY) + AMOUNT`
- `KEY <= 42`
- `AMOUNT <= 1`
- Result: `sat`, substitution `AMOUNT = 1, KEY = 0, ?STORAGE = ?STORAGE`
(the latter irrelevant but present)
* `with-map-unsat`: unsatisfiable predicate involving an irrelevant `Map` lookup
- Input predicates:
- `Map.lookup(?STORAGE, KEY) < Map.lookup(?STORAGE, KEY) + AMOUNT`
- `KEY <= 42`
- `AMOUNT < 1`
- Result: `unsat`, no substitution
* `with-map-relevant`: predicate involving a relevant `Map` lookup
- Input predicates:
- `AMOUNT < Map.lookup(?STORAGE, KEY)`
- `KEY <= 42`
- `AMOUNT <= 1`
- Result: `sat`, substitution `AMOUNT = 0, KEY = 0, ?STORAGE = ?STORAGE`.
(NB: `?STORAGE` value is relevant but not provided)
* `with-map-unsat2`: unsatisfiable predicate with a relevant `Map` lookup
- Input predicates:
- `Map.lookup(?STORAGE, KEY) < Map.lookup(?STORAGE, KEY)`
- `KEY <= 42`
- Result: `unsat`, no substitution
114 changes: 114 additions & 0 deletions test/rpc-integration/test-get-model/response-with-map-relevant.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"substitution": {
"format": "KORE",
"version": 1,
"term": {
"tag": "And",
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"patterns": [
{
"tag": "Equals",
"argSort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"first": {
"tag": "EVar",
"name": "AMOUNT",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
}
},
"second": {
"tag": "DV",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"value": "0"
}
},
{
"tag": "Equals",
"argSort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"first": {
"tag": "EVar",
"name": "KEY",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
}
},
"second": {
"tag": "DV",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"value": "0"
}
},
{
"tag": "Equals",
"argSort": {
"tag": "SortApp",
"name": "SortMap",
"args": []
},
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"first": {
"tag": "EVar",
"name": "Var'Ques'STORAGE",
"sort": {
"tag": "SortApp",
"name": "SortMap",
"args": []
}
},
"second": {
"tag": "EVar",
"name": "Var'Ques'STORAGE",
"sort": {
"tag": "SortApp",
"name": "SortMap",
"args": []
}
}
}
]
}
},
"satisfiable": "Sat"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"satisfiable": "Unsat"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"satisfiable": "Unsat"
}
}
114 changes: 114 additions & 0 deletions test/rpc-integration/test-get-model/response-with-map.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"substitution": {
"format": "KORE",
"version": 1,
"term": {
"tag": "And",
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"patterns": [
{
"tag": "Equals",
"argSort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"first": {
"tag": "EVar",
"name": "AMOUNT",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
}
},
"second": {
"tag": "DV",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"value": "1"
}
},
{
"tag": "Equals",
"argSort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"first": {
"tag": "EVar",
"name": "KEY",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
}
},
"second": {
"tag": "DV",
"sort": {
"tag": "SortApp",
"name": "SortInt",
"args": []
},
"value": "0"
}
},
{
"tag": "Equals",
"argSort": {
"tag": "SortApp",
"name": "SortMap",
"args": []
},
"sort": {
"tag": "SortApp",
"name": "SortGeneratedTopCell",
"args": []
},
"first": {
"tag": "EVar",
"name": "Var'Ques'STORAGE",
"sort": {
"tag": "SortApp",
"name": "SortMap",
"args": []
}
},
"second": {
"tag": "EVar",
"name": "Var'Ques'STORAGE",
"sort": {
"tag": "SortApp",
"name": "SortMap",
"args": []
}
}
}
]
}
},
"satisfiable": "Sat"
}
}
Loading

0 comments on commit 248f67b

Please sign in to comment.