Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataflow: Add support for pretty-printed alert provenance in tests #16210

Merged
merged 8 commits into from
Jun 7, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,44 @@ private import semmle.code.csharp.dispatch.OverridableCallable
private import semmle.code.csharp.frameworks.System
private import codeql.mad.ModelValidation as SharedModelVal

/**
* Holds if the given extension tuple `madId` should pretty-print as `model`.
*
* This predicate should only be used in tests.
*/
predicate interpretModelForTest(QlBuiltins::ExtensionId madId, string model) {
exists(
string namespace, string type, boolean subtypes, string name, string signature, string ext,
string output, string kind, string provenance
|
sourceModel(namespace, type, subtypes, name, signature, ext, output, kind, provenance, madId) and
model =
"Source: " + namespace + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just use ; as separator (this is what we typically do in other places we print models).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This string is only for inclusion in test output, and I found that it was more readable with the space included. Originally, before we had MaD rows in external yml files, I went with the no-space separation for the QL embedded csv rows in the name of compactness, but that's not relevant here.

+ ext + "; " + output + "; " + kind + "; " + provenance
)
or
exists(
string namespace, string type, boolean subtypes, string name, string signature, string ext,
string input, string kind, string provenance
|
sinkModel(namespace, type, subtypes, name, signature, ext, input, kind, provenance, madId) and
model =
"Sink: " + namespace + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; " +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment

ext + "; " + input + "; " + kind + "; " + provenance
)
or
exists(
string namespace, string type, boolean subtypes, string name, string signature, string ext,
string input, string output, string kind, string provenance
|
summaryModel(namespace, type, subtypes, name, signature, ext, input, output, kind, provenance,
madId) and
model =
"Summary: " + namespace + "; " + type + "; " + subtypes + "; " + name + "; " + signature +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment

"; " + ext + "; " + input + "; " + output + "; " + kind + "; " + provenance
)
}

private predicate relevantNamespace(string namespace) {
sourceModel(namespace, _, _, _, _, _, _, _, _, _) or
sinkModel(namespace, _, _, _, _, _, _, _, _, _) or
Expand Down
Original file line number Diff line number Diff line change
@@ -1,28 +1,32 @@
models
| 1 | Summary: System.Net; IPHostEntry; false; get_HostName; (); ; Argument[this]; ReturnValue; taint; manual |
| 2 | Summary: System.Web; HttpCookie; false; get_Value; (); ; Argument[this]; ReturnValue; taint; manual |
| 3 | Summary: System.Collections.Specialized; NameValueCollection; false; get_Item; (System.String); ; Argument[this]; ReturnValue; taint; df-generated |
edges
| ConditionalBypass.cs:12:16:12:22 | access to local variable isAdmin : String | ConditionalBypass.cs:16:13:16:30 | ... == ... | provenance | |
| ConditionalBypass.cs:12:26:12:48 | access to property QueryString : NameValueCollection | ConditionalBypass.cs:12:16:12:22 | access to local variable isAdmin : String | provenance | |
| ConditionalBypass.cs:12:26:12:48 | access to property QueryString : NameValueCollection | ConditionalBypass.cs:12:26:12:59 | access to indexer : String | provenance | MaD:11390 |
| ConditionalBypass.cs:12:26:12:48 | access to property QueryString : NameValueCollection | ConditionalBypass.cs:12:26:12:59 | access to indexer : String | provenance | MaD:3 |
| ConditionalBypass.cs:12:26:12:59 | access to indexer : String | ConditionalBypass.cs:12:16:12:22 | access to local variable isAdmin : String | provenance | |
| ConditionalBypass.cs:19:20:19:30 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:22:13:22:23 | access to local variable adminCookie : HttpCookie | provenance | |
| ConditionalBypass.cs:19:20:19:30 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:27:13:27:23 | access to local variable adminCookie : HttpCookie | provenance | |
| ConditionalBypass.cs:19:34:19:52 | access to property Cookies : HttpCookieCollection | ConditionalBypass.cs:19:20:19:30 | access to local variable adminCookie : HttpCookie | provenance | |
| ConditionalBypass.cs:22:13:22:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:22:13:22:29 | access to property Value : String | provenance | MaD:2161 |
| ConditionalBypass.cs:22:13:22:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:22:13:22:29 | access to property Value : String | provenance | MaD:2 |
| ConditionalBypass.cs:22:13:22:29 | access to property Value : String | ConditionalBypass.cs:22:13:22:45 | call to method Equals | provenance | |
| ConditionalBypass.cs:27:13:27:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:27:13:27:29 | access to property Value : String | provenance | MaD:2161 |
| ConditionalBypass.cs:27:13:27:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:27:13:27:29 | access to property Value : String | provenance | MaD:2 |
| ConditionalBypass.cs:27:13:27:29 | access to property Value : String | ConditionalBypass.cs:27:13:27:40 | ... == ... | provenance | |
| ConditionalBypass.cs:42:21:42:28 | access to local variable hostInfo : IPHostEntry | ConditionalBypass.cs:44:13:44:20 | access to local variable hostInfo : IPHostEntry | provenance | |
| ConditionalBypass.cs:42:21:42:28 | access to local variable hostInfo : IPHostEntry | ConditionalBypass.cs:49:13:49:20 | access to local variable hostInfo : IPHostEntry | provenance | |
| ConditionalBypass.cs:42:32:42:66 | call to method GetHostByAddress : IPHostEntry | ConditionalBypass.cs:42:21:42:28 | access to local variable hostInfo : IPHostEntry | provenance | |
| ConditionalBypass.cs:44:13:44:20 | access to local variable hostInfo : IPHostEntry | ConditionalBypass.cs:44:13:44:29 | access to property HostName : String | provenance | MaD:1827 |
| ConditionalBypass.cs:44:13:44:20 | access to local variable hostInfo : IPHostEntry | ConditionalBypass.cs:44:13:44:29 | access to property HostName : String | provenance | MaD:1 |
| ConditionalBypass.cs:44:13:44:29 | access to property HostName : String | ConditionalBypass.cs:44:13:44:46 | ... == ... | provenance | |
| ConditionalBypass.cs:49:13:49:20 | access to local variable hostInfo : IPHostEntry | ConditionalBypass.cs:49:13:49:29 | access to property HostName | provenance | MaD:1827 |
| ConditionalBypass.cs:49:13:49:20 | access to local variable hostInfo : IPHostEntry | ConditionalBypass.cs:49:13:49:29 | access to property HostName | provenance | MaD:1 |
| ConditionalBypass.cs:70:20:70:30 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:72:13:72:23 | access to local variable adminCookie : HttpCookie | provenance | |
| ConditionalBypass.cs:70:34:70:52 | access to property Cookies : HttpCookieCollection | ConditionalBypass.cs:70:20:70:30 | access to local variable adminCookie : HttpCookie | provenance | |
| ConditionalBypass.cs:72:13:72:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:72:13:72:29 | access to property Value : String | provenance | MaD:2161 |
| ConditionalBypass.cs:72:13:72:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:72:13:72:29 | access to property Value : String | provenance | MaD:2 |
| ConditionalBypass.cs:72:13:72:29 | access to property Value : String | ConditionalBypass.cs:72:13:72:40 | ... == ... | provenance | |
| ConditionalBypass.cs:83:20:83:30 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:84:13:84:23 | access to local variable adminCookie : HttpCookie | provenance | |
| ConditionalBypass.cs:83:34:83:52 | access to property Cookies : HttpCookieCollection | ConditionalBypass.cs:83:20:83:30 | access to local variable adminCookie : HttpCookie | provenance | |
| ConditionalBypass.cs:84:13:84:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:84:13:84:29 | access to property Value : String | provenance | MaD:2161 |
| ConditionalBypass.cs:84:13:84:23 | access to local variable adminCookie : HttpCookie | ConditionalBypass.cs:84:13:84:29 | access to property Value : String | provenance | MaD:2 |
| ConditionalBypass.cs:84:13:84:29 | access to property Value : String | ConditionalBypass.cs:84:13:84:40 | ... == ... | provenance | |
nodes
| ConditionalBypass.cs:12:16:12:22 | access to local variable isAdmin : String | semmle.label | access to local variable isAdmin : String |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/**
* @kind path-problem
*/

import csharp
import semmle.code.csharp.security.dataflow.ConditionalBypassQuery
import codeql.dataflow.test.ProvenancePathGraph
import semmle.code.csharp.dataflow.internal.ExternalFlow
import ShowProvenance<interpretModelForTest/2, ConditionalBypass::PathNode, ConditionalBypass::PathGraph>

from ConditionalBypass::PathNode source, ConditionalBypass::PathNode sink
where ConditionalBypass::flowPath(source, sink)
select sink.getNode(), source, sink, "This condition guards a sensitive $@, but a $@ controls it.",
sink.getNode().(Sink).getSensitiveMethodCall(), "action", source.getNode(), "user-provided value"

This file was deleted.

38 changes: 38 additions & 0 deletions go/ql/lib/semmle/go/dataflow/ExternalFlow.qll
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,44 @@ private import internal.FlowSummaryImpl::Private
private import internal.FlowSummaryImpl::Private::External
private import codeql.mad.ModelValidation as SharedModelVal

/**
* Holds if the given extension tuple `madId` should pretty-print as `model`.
*
* This predicate should only be used in tests.
*/
predicate interpretModelForTest(QlBuiltins::ExtensionId madId, string model) {
exists(
string package, string type, boolean subtypes, string name, string signature, string ext,
string output, string kind, string provenance
|
sourceModel(package, type, subtypes, name, signature, ext, output, kind, provenance, madId) and
model =
"Source: " + package + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; " +
ext + "; " + output + "; " + kind + "; " + provenance
)
or
exists(
string package, string type, boolean subtypes, string name, string signature, string ext,
string input, string kind, string provenance
|
sinkModel(package, type, subtypes, name, signature, ext, input, kind, provenance, madId) and
model =
"Sink: " + package + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; " +
ext + "; " + input + "; " + kind + "; " + provenance
)
or
exists(
string package, string type, boolean subtypes, string name, string signature, string ext,
string input, string output, string kind, string provenance
|
summaryModel(package, type, subtypes, name, signature, ext, input, output, kind, provenance,
madId) and
model =
"Summary: " + package + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; " +
ext + "; " + input + "; " + output + "; " + kind + "; " + provenance
)
}

private predicate relevantPackage(string package) {
sourceModel(package, _, _, _, _, _, _, _, _, _) or
sinkModel(package, _, _, _, _, _, _, _, _, _) or
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
models
| 1 | Summary: net/url; URL; true; Query; ; ; Argument[receiver]; ReturnValue; taint; manual |
| 2 | Summary: path; ; false; Clean; ; ; Argument[0]; ReturnValue; taint; manual |
edges
| TaintedPath.go:14:18:14:22 | selection of URL | TaintedPath.go:14:18:14:30 | call to Query | provenance | MaD:735 |
| TaintedPath.go:14:18:14:22 | selection of URL | TaintedPath.go:14:18:14:30 | call to Query | provenance | MaD:1 |
| TaintedPath.go:14:18:14:30 | call to Query | TaintedPath.go:17:29:17:40 | tainted_path | provenance | |
| TaintedPath.go:14:18:14:30 | call to Query | TaintedPath.go:21:57:21:68 | tainted_path | provenance | |
| TaintedPath.go:14:18:14:30 | call to Query | TaintedPath.go:68:39:68:56 | ...+... | provenance | |
| TaintedPath.go:21:57:21:68 | tainted_path | TaintedPath.go:21:28:21:69 | call to Join | provenance | FunctionModel |
| TaintedPath.go:68:39:68:56 | ...+... | TaintedPath.go:68:28:68:57 | call to Clean | provenance | MaD:761 |
| TaintedPath.go:68:39:68:56 | ...+... | TaintedPath.go:68:28:68:57 | call to Clean | provenance | MaD:2 |
nodes
| TaintedPath.go:14:18:14:22 | selection of URL | semmle.label | selection of URL |
| TaintedPath.go:14:18:14:30 | call to Query | semmle.label | call to Query |
Expand Down
14 changes: 14 additions & 0 deletions go/ql/test/query-tests/Security/CWE-022/TaintedPath.ql
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/**
* @kind path-problem
*/

import go
import semmle.go.security.TaintedPath
import codeql.dataflow.test.ProvenancePathGraph
import semmle.go.dataflow.ExternalFlow
import ShowProvenance<interpretModelForTest/2, TaintedPath::Flow::PathNode, TaintedPath::Flow::PathGraph>

from TaintedPath::Flow::PathNode source, TaintedPath::Flow::PathNode sink
where TaintedPath::Flow::flowPath(source, sink)
select sink.getNode(), source, sink, "This path depends on a $@.", source.getNode(),
"user-provided value"

This file was deleted.

38 changes: 38 additions & 0 deletions java/ql/lib/semmle/code/java/dataflow/ExternalFlow.qll
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,44 @@ predicate summaryModel(
)
}

/**
* Holds if the given extension tuple `madId` should pretty-print as `model`.
*
* This predicate should only be used in tests.
*/
predicate interpretModelForTest(QlBuiltins::ExtensionId madId, string model) {
exists(
string package, string type, boolean subtypes, string name, string signature, string ext,
string output, string kind, string provenance
|
sourceModel(package, type, subtypes, name, signature, ext, output, kind, provenance, madId) and
model =
"Source: " + package + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; " +
ext + "; " + output + "; " + kind + "; " + provenance
)
or
exists(
string package, string type, boolean subtypes, string name, string signature, string ext,
string input, string kind, string provenance
|
sinkModel(package, type, subtypes, name, signature, ext, input, kind, provenance, madId) and
model =
"Sink: " + package + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; " +
ext + "; " + input + "; " + kind + "; " + provenance
)
or
exists(
string package, string type, boolean subtypes, string name, string signature, string ext,
string input, string output, string kind, string provenance
|
summaryModel(package, type, subtypes, name, signature, ext, input, output, kind, provenance,
madId) and
model =
"Summary: " + package + "; " + type + "; " + subtypes + "; " + name + "; " + signature + "; " +
ext + "; " + input + "; " + output + "; " + kind + "; " + provenance
)
}

/** Holds if a neutral model exists for the given parameters. */
predicate neutralModel = Extensions::neutralModel/6;

Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
models
| 1 | Sink: java.net; URL; false; openConnection; ; ; Argument[this]; request-forgery; manual |
| 2 | Summary: java.net; URL; false; URL; (String); ; Argument[0]; Argument[this]; taint; manual |
| 3 | Summary: java.net; URL; false; URL; (URL,String); ; Argument[1]; Argument[this]; taint; ai-manual |
edges
| HttpsUrlsTest.java:23:23:23:31 | "http://" : String | HttpsUrlsTest.java:24:21:24:56 | ... + ... : String | provenance | |
| HttpsUrlsTest.java:24:13:24:57 | new URL(...) : URL | HttpsUrlsTest.java:28:50:28:50 | u | provenance | Sink:MaD:42944 |
| HttpsUrlsTest.java:24:13:24:57 | new URL(...) : URL | HttpsUrlsTest.java:28:50:28:50 | u | provenance | Sink:MaD:1 |
| HttpsUrlsTest.java:24:21:24:56 | ... + ... : String | HttpsUrlsTest.java:24:13:24:57 | new URL(...) : URL | provenance | Config |
| HttpsUrlsTest.java:24:21:24:56 | ... + ... : String | HttpsUrlsTest.java:24:13:24:57 | new URL(...) : URL | provenance | MaD:42977 |
| HttpsUrlsTest.java:24:21:24:56 | ... + ... : String | HttpsUrlsTest.java:24:13:24:57 | new URL(...) : URL | provenance | MaD:2 |
| HttpsUrlsTest.java:36:23:36:28 | "http" : String | HttpsUrlsTest.java:37:21:37:28 | protocol : String | provenance | |
| HttpsUrlsTest.java:37:13:37:62 | new URL(...) : URL | HttpsUrlsTest.java:41:50:41:50 | u | provenance | Sink:MaD:42944 |
| HttpsUrlsTest.java:37:13:37:62 | new URL(...) : URL | HttpsUrlsTest.java:41:50:41:50 | u | provenance | Sink:MaD:1 |
| HttpsUrlsTest.java:37:21:37:28 | protocol : String | HttpsUrlsTest.java:37:13:37:62 | new URL(...) : URL | provenance | Config |
| HttpsUrlsTest.java:49:23:49:31 | "http://" : String | HttpsUrlsTest.java:51:64:51:98 | ... + ... : String | provenance | |
| HttpsUrlsTest.java:51:13:51:99 | new URL(...) : URL | HttpsUrlsTest.java:55:50:55:50 | u | provenance | Sink:MaD:42944 |
| HttpsUrlsTest.java:51:13:51:99 | new URL(...) : URL | HttpsUrlsTest.java:55:50:55:50 | u | provenance | Sink:MaD:1 |
| HttpsUrlsTest.java:51:64:51:98 | ... + ... : String | HttpsUrlsTest.java:51:13:51:99 | new URL(...) : URL | provenance | Config |
| HttpsUrlsTest.java:51:64:51:98 | ... + ... : String | HttpsUrlsTest.java:51:13:51:99 | new URL(...) : URL | provenance | MaD:42985 |
| HttpsUrlsTest.java:51:64:51:98 | ... + ... : String | HttpsUrlsTest.java:51:13:51:99 | new URL(...) : URL | provenance | MaD:3 |
| HttpsUrlsTest.java:87:23:87:28 | "http" : String | HttpsUrlsTest.java:88:21:88:28 | protocol : String | provenance | |
| HttpsUrlsTest.java:88:13:88:52 | new URL(...) : URL | HttpsUrlsTest.java:92:50:92:50 | u | provenance | Sink:MaD:42944 |
| HttpsUrlsTest.java:88:13:88:52 | new URL(...) : URL | HttpsUrlsTest.java:92:50:92:50 | u | provenance | Sink:MaD:1 |
| HttpsUrlsTest.java:88:21:88:28 | protocol : String | HttpsUrlsTest.java:88:13:88:52 | new URL(...) : URL | provenance | Config |
nodes
| HttpsUrlsTest.java:23:23:23:31 | "http://" : String | semmle.label | "http://" : String |
Expand Down
14 changes: 14 additions & 0 deletions java/ql/test/query-tests/security/CWE-311/CWE-319/HttpsUrls.ql
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/**
* @kind path-problem
*/

import java
import semmle.code.java.security.HttpsUrlsQuery
import codeql.dataflow.test.ProvenancePathGraph
import semmle.code.java.dataflow.ExternalFlow
import ShowProvenance<interpretModelForTest/2, HttpStringToUrlOpenMethodFlow::PathNode, HttpStringToUrlOpenMethodFlow::PathGraph>

from HttpStringToUrlOpenMethodFlow::PathNode source, HttpStringToUrlOpenMethodFlow::PathNode sink
where HttpStringToUrlOpenMethodFlow::flowPath(source, sink)
select sink.getNode(), source, sink, "URL may have been constructed with HTTP protocol, using $@.",
source.getNode(), "this HTTP URL"

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -369,6 +369,28 @@ private predicate typeVariableModel(string name, string path) {
Extensions::typeVariableModel(name, path)
}

/**
* Holds if the given extension tuple `madId` should pretty-print as `model`.
*
* This predicate should only be used in tests.
*/
predicate interpretModelForTest(QlBuiltins::ExtensionId madId, string model) {
exists(string type, string path, string kind |
Extensions::sourceModel(type, path, kind, madId) and
model = "Source: " + type + "; " + path + "; " + kind
)
or
exists(string type, string path, string kind |
Extensions::sinkModel(type, path, kind, madId) and
model = "Sink: " + type + "; " + path + "; " + kind
)
or
exists(string type, string path, string input, string output, string kind |
Extensions::summaryModel(type, path, input, output, kind, madId) and
model = "Summary: " + type + "; " + path + "; " + input + "; " + output + "; " + kind
)
}

/**
* Holds if rows involving `type` might be relevant for the analysis of this database.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -369,6 +369,28 @@ private predicate typeVariableModel(string name, string path) {
Extensions::typeVariableModel(name, path)
}

/**
* Holds if the given extension tuple `madId` should pretty-print as `model`.
*
* This predicate should only be used in tests.
*/
predicate interpretModelForTest(QlBuiltins::ExtensionId madId, string model) {
exists(string type, string path, string kind |
Extensions::sourceModel(type, path, kind, madId) and
model = "Source: " + type + "; " + path + "; " + kind
)
or
exists(string type, string path, string kind |
Extensions::sinkModel(type, path, kind, madId) and
model = "Sink: " + type + "; " + path + "; " + kind
)
or
exists(string type, string path, string input, string output, string kind |
Extensions::summaryModel(type, path, input, output, kind, madId) and
model = "Summary: " + type + "; " + path + "; " + input + "; " + output + "; " + kind
)
}

/**
* Holds if rows involving `type` might be relevant for the analysis of this database.
*/
Expand Down
Loading
Loading