Skip to content

Python: Promote XXE and XML-bomb queries #8634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yoff merged 54 commits into from
May 9, 2022
Merged

Python: Promote XXE and XML-bomb queries #8634

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
65907c9
Python: Copy Xxe/XmlBomb queries from JS
RasmusWL Mar 24, 2022
e45f9d6
Python: Adjust Xxe/XmlBomb for Python
RasmusWL Mar 24, 2022
91795b8
Python: Add simple test of Xxe/XmlBomb
RasmusWL Mar 24, 2022
a1d88e3
Python: Adjust XXE PoC for newer lxml versions
RasmusWL Mar 24, 2022
57b9780
Python: XXE: Add example of exfiltrating data through dtd-retrival
RasmusWL Mar 24, 2022
769f569
Python: Add taint for `StringIO` and `BytesIO`
RasmusWL Mar 29, 2022
c365337
Python: Delete `XmlEntityInjection.ql`
RasmusWL Mar 29, 2022
b00766b
Python: Adjust XXE qhelp
RasmusWL Mar 29, 2022
56b9c89
Python: Adjust `XmlBomb.qhelp` from JS
RasmusWL Mar 29, 2022
9caf4be
Python: Add PortSwigger link to `Xxe.qhelp`
RasmusWL Mar 29, 2022
e005a5c
Python: Promote `XMLParsing` concept
RasmusWL Mar 29, 2022
e45288e
Python: => `XMLParsingVulnerabilityKind`
RasmusWL Mar 29, 2022
35ccba2
Python: Promote `XMLParsing` concept test
RasmusWL Mar 29, 2022
1ea4bcc
Python: Make `XMLParsing` a `Decoding` subclass
RasmusWL Mar 29, 2022
c4473c5
Python: Rename lxml XPath tests
RasmusWL Mar 31, 2022
3040adf
Python: Handle `XMLParser().close()` for XPath
RasmusWL Mar 31, 2022
80b5cde
Python: Promote lxml parsing modeling
RasmusWL Mar 31, 2022
7f5f767
Python: Promote `xmltodict` modeling
RasmusWL Mar 31, 2022
64aa503
Python: Promote `xml.etree` modeling
RasmusWL Mar 31, 2022
a315aa8
Python: Add some links in QLDocs
RasmusWL Mar 31, 2022
6774085
Python: Add note about parseid/XMLID
RasmusWL Mar 31, 2022
12cbdcd
Python: Model `lxml.etree.XMLID`
RasmusWL Mar 31, 2022
386ff53
Python: Model `lxml.iterparse`
RasmusWL Mar 31, 2022
543454e
Python: Model file access from XML parsing
RasmusWL Mar 31, 2022
db43d04
Python: Add test showing misalignment of xml.etree modeling
RasmusWL Mar 31, 2022
70b3eec
Python: Merge `xml.etree.ElementTree` models
RasmusWL Mar 31, 2022
05bb0ef
Python: Align `xml.etree.ElementTree` modeling
RasmusWL Mar 31, 2022
e112697
Python: Promote `xml.sax` and `xml.dom.*` modeling
RasmusWL Mar 31, 2022
1d7cec6
Python: `xml.sax.parse` is not a method call
RasmusWL Mar 31, 2022
b4c0065
Python: Extend FileSystemAccess for `xml.sax` and `xml.dom.*` parsing
RasmusWL Mar 31, 2022
673220b
Python: Minor cleanup of `XmlParsingTest`
RasmusWL Mar 31, 2022
5083023
Python: Move XML parsing PoC
RasmusWL Mar 31, 2022
b8d3c5e
Python: Remove last bits of experimental XML modeling
RasmusWL Mar 31, 2022
4abab22
Python: Promote XXE and XML-bomb queries
RasmusWL Mar 31, 2022
d2b03bb
Python: Fix `SimpleXmlRpcServer.ql`
RasmusWL Mar 31, 2022
ab59d5c
Python: Rename to `XmlParsing`
RasmusWL Apr 5, 2022
1f285b8
Python: Rename to `XmlParsingVulnerabilityKind`
RasmusWL Apr 5, 2022
a7dab53
Python: Add change-note
RasmusWL Apr 5, 2022
b7f56dd
Python: Rewrite concepts to use `extends ... instanceof ...`
RasmusWL Apr 5, 2022
23637fd
Merge branch 'main' into promote-xxe
RasmusWL Apr 6, 2022
c784f15
Python: Rename more XML classes to follow convention
RasmusWL Apr 6, 2022
f2f0873
Python: Use new `API::CallNode` for XML constant check
RasmusWL Apr 6, 2022
7728b6c
Python: Change XmlBomb vulnerability kind
RasmusWL Apr 7, 2022
405480c
Python: Rename sink definitions for XXE/XML bomb
RasmusWL Apr 7, 2022
8191be9
Python: Move last XXE/XML bomb out of experimental
RasmusWL Apr 7, 2022
517444b
Python: Fix `SimpleXmlRpcServer.expected`
RasmusWL Apr 7, 2022
bb6969a
Merge branch 'main' into promote-xxe
RasmusWL Apr 20, 2022
5f01fc2
Merge branch 'main' into promote-xxe
RasmusWL May 2, 2022
714465b
Python: Refactor `SaxParserSetFeatureCall`
RasmusWL May 2, 2022
f5854f3
Python: Apply suggestions from code review
RasmusWL May 9, 2022
f22bd03
Python: Slight refactor of `LxmlParsing`
RasmusWL May 9, 2022
3634922
Python: Fix casing of `XMLDomParsing`
RasmusWL May 9, 2022
de05b10
Python: Fix singleton set
RasmusWL May 9, 2022
4a67891
Python: Apply suggestions from code review
RasmusWL May 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Python: Model lxml.etree.XMLID
  • Loading branch information
@RasmusWL
RasmusWL committed Mar 31, 2022
commit 12cbdcde284e4e8fbce7a02ae0f65cedeee7e4eb
8 changes: 5 additions & 3 deletions python/ql/lib/semmle/python/frameworks/Lxml.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,13 +221,15 @@ private module Lxml {
* - `lxml.etree.fromstring`
* - `lxml.etree.fromstringlist`
* - `lxml.etree.XML`
* - `lxml.etree.XMLID`
* - `lxml.etree.parse`
* - `lxml.etree.parseid`
*
* See
* - https://lxml.de/apidoc/lxml.etree.html?highlight=parseids#lxml.etree.fromstring
* - https://lxml.de/apidoc/lxml.etree.html?highlight=parseids#lxml.etree.fromstringlist
* - https://lxml.de/apidoc/lxml.etree.html?highlight=parseids#lxml.etree.XML
* - https://lxml.de/apidoc/lxml.etree.html?highlight=parseids#lxml.etree.XMLID
* - https://lxml.de/apidoc/lxml.etree.html?highlight=parseids#lxml.etree.parse
* - https://lxml.de/apidoc/lxml.etree.html?highlight=parseids#lxml.etree.parseid
*/
Expand All @@ -236,14 +238,14 @@ private module Lxml {
this =
API::moduleImport("lxml")
.getMember("etree")
.getMember(["fromstring", "fromstringlist", "XML", "parse", "parseid"])
.getMember(["fromstring", "fromstringlist", "XML", "XMLID", "parse", "parseid"])
.getACall()
}

override DataFlow::Node getAnInput() {
result in [
this.getArg(0),
// fromstring / XML
// fromstring / XML / XMLID
this.getArgByName("text"),
// fromstringlist
this.getArgByName("strings"),
Expand All @@ -264,7 +266,7 @@ private module Lxml {
override predicate mayExecuteInput() { none() }

override DataFlow::Node getOutput() {
// Note: for `parseid` the result of the call is a tuple with `(root, dict)`, so
// Note: for `parseid`/XMLID the result of the call is a tuple with `(root, dict)`, so
// maybe we should not just say that the entire tuple is the decoding output... my
// gut feeling is that THIS instance doesn't matter too much, but that it would be
// nice to be able to do this in general. (this is a problem for both `lxml.etree`
Expand Down
3 changes: 3 additions & 0 deletions python/ql/test/library-tests/frameworks/lxml/parsing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
lxml.etree.XML(x) # $ decodeFormat=XML decodeInput=x xmlVuln='XXE' decodeOutput=lxml.etree.XML(..)
lxml.etree.XML(text=x) # $ decodeFormat=XML decodeInput=x xmlVuln='XXE' decodeOutput=lxml.etree.XML(..)

lxml.etree.XMLID(x) # $ decodeFormat=XML decodeInput=x xmlVuln='XXE' decodeOutput=lxml.etree.XMLID(..)
lxml.etree.XMLID(text=x) # $ decodeFormat=XML decodeInput=x xmlVuln='XXE' decodeOutput=lxml.etree.XMLID(..)

lxml.etree.parse(StringIO(x)) # $ decodeFormat=XML decodeInput=StringIO(..) xmlVuln='XXE' decodeOutput=lxml.etree.parse(..)
lxml.etree.parse(source=StringIO(x)) # $ decodeFormat=XML decodeInput=StringIO(..) xmlVuln='XXE' decodeOutput=lxml.etree.parse(..)

Expand Down