[SR-2301] NSXMLParser not fully implemented

[swift-ci]

Previous ID	SR-2301
Radar	rdar://problem/76905860
Original Reporter	TimKreger (JIRA User)
Type	Bug

Attachment: Download

Environment

Ubuntu 14.04
swift-3.0-PREVIEW-2

Additional Detail from JIRA

Votes	5
Component/s	Foundation
Labels	Bug, Linux
Assignee	KingOfBrian (JIRA)
Priority	Medium

md5: 803674b2c85303d495af09f40e0e6338

Issue Description:

I'm attempting to implement an xml parser using (swift-3.0-PREVIEW-2) on Ubuntu. When compiling it requires all of the NSXMLParserDelegate functions to be implemented (they are optional on OSX) which I have implemented. However at run time none of the the delegate functions are called even though .parse() completes with no error. Ie this function never gets called during the parsing.

 public func parserDidStartDocument(_ parser: Foundation.NSXMLParser) { print("Did start") } 

so I have been using swift-3.0-PREVIEW-2 on ubuntu.

I have included an example parser.
Note that the important delegate functions

public func parserDidStartDocument(_ parser: XMLParser)
public func parserDidEndDocument(_ parser: XMLParser)
public func parser(_ parser: XMLParser, didStartElement elementName: String, namespaceURI: String?, qualifiedName qName: String?, attributes attributeDict: [String : String])
public func parser(_ parser: XMLParser, didEndElement elementName: String, namespaceURI: String?, qualifiedName qName: String?)

are never called.

[norio-nomura]

I see same issue on drmohundro/SWXMLHash#97 using swift-3.0-PREVIEW-6 on ubuntu.

[swift-ci]

Comment by Brian (JIRA)

I put a PR up with some more tests and a fix.

#700

[swift-ci]

Comment by Maksim Rogov (JIRA)

I believe this has been merged in and can be closed?

[egorzhdan]

lupinglade (JIRA User) No, it's still not working properly on linux

[swift-ci]

Comment by Maksim Rogov (JIRA)

@egorzhdan Really? We've tested it here and it seemed to be fine besides some inconsistency issues with the delegate method signatures.

[dhoepfl]

lupinglade (JIRA User)

I noticed that the bug (partially) still exists using the current master.

parserDidStartDocument, didStartElement, and didEndElement are called but parserDidEndDocument still is not called.

I noticed that the tests in TestXMLParser.swift know about the endDocument event but do not expect it in the tests (see here). Actually, all XMLParser tests would fail if XMLParser worked as it should.

[dhoepfl]

I opened a pull request to add the endDocument call.

[dhoepfl]

When investigating SR-13546, I noticed that the entity declaration callbacks of XMLParserDelegate are never called, so this issue is still unresolved:

func parser(_ parser: XMLParser,
            foundExternalEntityDeclarationWithName name: String,
            publicID: String?,
            systemID: String?)
func parser(_ parser: XMLParser,
            foundInternalEntityDeclarationWithName name: String,
            value: String?)
func parser(_ parser: XMLParser,
            foundUnparsedEntityDeclarationWithName name: String,
            publicID: String?,
            systemID: String?,
            notationName: String?)

Test case:

import Foundation
import FoundationXML

class MyDelegate : NSObject, XMLParserDelegate {
   func parser(_ parser: XMLParser, foundExternalEntityDeclarationWithName name: String, publicID: String?, systemID: String?) {
      print("foundExternalEntityDeclarationWithName: \(name)")
   }

   func parser(_ parser: XMLParser, foundInternalEntityDeclarationWithName name: String, value: String?) {
      print("foundInternalEntityDeclarationWithName: \(name)")
   }

   func parser(_ parser: XMLParser, foundUnparsedEntityDeclarationWithName name: String, publicID: String?, systemID: String?, notationName: String?) {
      print("foundUnparsedEntityDeclarationWithName: \(name)")
   }
}

let xml = """
          <?xml version="1.0" encoding="ISO-8859-1"?>
          <!DOCTYPE root [

          <!NOTATION notation PUBLIC "empty file">

          <!ENTITY unparsedEntity SYSTEM "file:///dev/null" NDATA notation>
          <!ENTITY externalEntity SYSTEM "file:///dev/null" >
          <!ENTITY internalEntity "Internal" >

          <!ELEMENT root (foo*) >

          <!ELEMENT foo ANY >

          <!ATTLIST foo bar ENTITY #REQUIRED>

          ]>
          <root>
          <foo>internal entity: &internalEntity;</foo>
          <foo>external entity: &externalEntity;</foo>
          <foo bar="unparsedEntity" />
          </root>
          """

let parser = XMLParser(data: xml.data(using: .utf8)!)
parser.shouldProcessNamespaces = true
parser.shouldReportNamespacePrefixes = true
parser.shouldResolveExternalEntities = true
parser.externalEntityResolvingPolicy = .always
let delegate = MyDelegate()
parser.delegate = delegate
let res = parser.parse()

Expected output:

foundUnparsedEntityDeclarationWithName: unparsedEntity
foundExternalEntityDeclarationWithName: externalEntity
foundInternalEntityDeclarationWithName: internalEntity

(Working on it)

[dhoepfl]

Mostly (if not all) fixed in pull request #2920.

Please note that the NSXMLParser implementation in macOS (and probably iOS) also fails to support entities.

I did not get NSXMLParser to ever call the func parser(XMLParser, foundIgnorableWhitespace: String) delegate callback.

[weissi]

Don't think this is fixed on current main

root@0ae6b2f0fd1a:/Users/johannes/devel/swift-corelibs-foundation# swift
Welcome to Swift version 5.5-dev (LLVM fb5a91b4e892ed9, Swift f17142883421ec0).
Type :help for assistance.
  1> import Foundation 
  2. import FoundationXML 
  3.  
  4. class MyDelegate : NSObject, XMLParserDelegate { 
  5.    func parser(_ parser: XMLParser, foundExternalEntityDeclarationWithName name: String, publicID: String?, systemID: String?) { 
  6.       print("foundExternalEntityDeclarationWithName: \(name)") 
  7.    } 
  8.  
  9.    func parser(_ parser: XMLParser, foundInternalEntityDeclarationWithName name: String, value: String?) { 
 10.       print("foundInternalEntityDeclarationWithName: \(name)") 
 11.    } 
 12.  
 13.    func parser(_ parser: XMLParser, foundUnparsedEntityDeclarationWithName name: String, publicID: String?, systemID: String?, notationName: String?) { 
 14.       print("foundUnparsedEntityDeclarationWithName: \(name)") 
 15.    } 
 16. } 
 17.  
 18. let xml = """ 
 19.           <?xml version="1.0" encoding="ISO-8859-1"?> 
 20.           <!DOCTYPE root [ 
 21.  
 22.           <!NOTATION notation PUBLIC "empty file"> 
 23.  
 24.           <!ENTITY unparsedEntity SYSTEM "file:///dev/null" NDATA notation> 
 25.           <!ENTITY externalEntity SYSTEM "file:///dev/null" > 
 26.           <!ENTITY internalEntity "Internal" > 
 27.  
 28.           <!ELEMENT root (foo*) > 
 29.  
 30.           <!ELEMENT foo ANY > 
 31.  
 32.           <!ATTLIST foo bar ENTITY #REQUIRED> 
 33.  
 34.           ]> 
 35.           <root> 
 36.           <foo>internal entity: &internalEntity;</foo> 
 37.           <foo>external entity: &externalEntity;</foo> 
 38.           <foo bar="unparsedEntity" /> 
 39.           </root> 
 40.           """ 
 41.  
 42. let parser = XMLParser(data: xml.data(using: .utf8)!) 
 43. parser.shouldProcessNamespaces = true 
 44. parser.shouldReportNamespacePrefixes = true 
 45. parser.shouldResolveExternalEntities = true 
 46. parser.externalEntityResolvingPolicy = .always 
 47. let delegate = MyDelegate() 
 48. parser.delegate = delegate 
 49. let res = parser.parse()
xml: String = {
  _guts = {
    _object = {
      _countAndFlagsBits = <extracting data from value failed>

      _object = <extracting data from value failed>

    }
  }
}
parser: XMLParser = <extracting data from value failed>

delegate: MyDelegate = <extracting data from value failed>

res: Bool = true
Execution interrupted. Enter code to recover and continue.
Enter LLDB commands to investigate (type :help for assistance.)

[weissi]

@swift-ci create

[weissi]

   Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:
0.  Program arguments: /usr/bin/swift-frontend -frontend -interpret t.swift -disable-objc-interop -color-diagnostics -module-name t
1.  Swift version 5.5-dev (LLVM fb5a91b4e892ed9, Swift f17142883421ec0)
2.  While running user code "t.swift"
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
/usr/bin/swift-frontend[0x5c19cb3]
/usr/bin/swift-frontend[0x5c17a0e]
/usr/bin/swift-frontend[0x5c1a03c]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f0051643980]
/lib/x86_64-linux-gnu/libc.so.6(+0x18e4e1)[0x7f00500354e1]
/usr/lib/swift/linux/libFoundationXML.so($s13FoundationXML24_NSXMLParserNotationDecl_4name8publicId06systemH0ys13OpaquePointerV_SPys5UInt8VGA2JtF+0x87)[0x7f004c871d07]
/usr/lib/x86_64-linux-gnu/libxml2.so.2(xmlParseNotationDecl+0x206)[0x7f0045fda826]
/usr/lib/x86_64-linux-gnu/libxml2.so.2(xmlParseMarkupDecl+0x101)[0x7f0045fdeda1]
/usr/lib/x86_64-linux-gnu/libxml2.so.2(+0x4ee95)[0x7f0045fdee95]
/usr/lib/x86_64-linux-gnu/libxml2.so.2(+0x56c29)[0x7f0045fe6c29]
/usr/lib/x86_64-linux-gnu/libxml2.so.2(xmlParseChunk+0x14e)[0x7f0045fe75de]
/usr/lib/swift/linux/libFoundationXML.so(_CFXMLInterfaceParseChunk+0x8)[0x7f004c877c08]
/usr/lib/swift/linux/libFoundationXML.so(+0x3d1b7)[0x7f004c8771b7]
/usr/lib/swift/linux/libFoundationXML.so($s13FoundationXML9XMLParserC9parseData_011lastChunkOfE0Sb0A00E0V_SbtF+0x64f)[0x7f004c873e9f]
/usr/lib/swift/linux/libFoundationXML.so($s13FoundationXML9XMLParserC5parseSbyF+0x241)[0x7f004c8744c1]
[0x7f0051a771e1]
/usr/bin/swift-frontend[0x6c986a]
/usr/bin/swift-frontend[0x5a122e]
/usr/bin/swift-frontend[0x56c3fe]
/usr/bin/swift-frontend[0x56ba17]
/usr/bin/swift-frontend[0x55f7fd]
/usr/bin/swift-frontend[0x4b0616]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f004fec8bf7]
/usr/bin/swift-frontend[0x4b024a]
Segmentation fault

[swift-ci]

Comment by Stefan Springer (JIRA)

Note that the processing of entities is still in part problematic, see my little example Github repository https://github.com/stefanspringer1/SwiftXMLParserExamples and the following issues that I posted: https://bugs.swift.org/browse/SR-14581, https://bugs.swift.org/browse/SR-14582, https://bugs.swift.org/browse/SR-14583, https://bugs.swift.org/browse/SR-14584, https://bugs.swift.org/browse/SR-14585, https://bugs.swift.org/browse/SR-14586.