[stdlib] Failing string ordering on linux test

[rtreffer]

This testcase illustrates a problem on linux: the strings "a" and "a\0" satisfy

"a" <= "a\0"
"a" >= "a\0"
"a" != "a\0"

The testcase should succeed on OS X (@Lvt send me the OS X output of the compares - it's basically "a" < "a\0", which is reasonable).

Here is a sample output of the failure on linux (swift trunk with this pull request, ubuntu 15.10):

Command Output (stdout):
--
[ RUN      ] StringOrderRelation.StringOrderRelation/ASCII/NullByte
out>>> check failed at /home/rtreffer/work/swift/test/1_stdlib/StringOrderRelation.swift, line 19
out>>> expected: true
out>>> check failed at /home/rtreffer/work/swift/test/1_stdlib/StringOrderRelation.swift, line 22
out>>> expected: false
[     FAIL ] StringOrderRelation.StringOrderRelation/ASCII/NullByte
StringOrderRelation: Some tests failed, aborting
UXPASS: []
FAIL: ["StringOrderRelation/ASCII/NullByte"]
SKIP: []

SR-530 reports another inconsistent ordering on linux but is related to case sensitive vs. insensitive ordering. This might be another nice addition for an ordering testcase.

[moiseev]

Can you please XFAIL this test for Linux, and raise a bug at bugs.swift.org?

[rtreffer]

Added .xfail. I'd expect it to fail on FreeBSD as well as it might be related to #1119

And it's SR-630

[moiseev]

Thank you!

[jrose-apple]

I'm starting to think we want .xfail(.NonApple(reason: ...)).

[rtreffer]

@jrose-apple Here is the `#if`` highscore throughout swift:

$ grep -ir '^#if' |sed 's/^[^:]*://'|sort|uniq -c|sort -rn|head -n12
    338 #if _runtime(_ObjC)
    128 #if SWIFT_OBJC_INTEROP
    119 #ifndef NDEBUG
     42 #if BEFORE
     38 #if arch(i386) || arch(arm)
     25 #if arch(i386) || arch(x86_64)
     19 #if os(OSX)
     14 #if os(iOS)
     13 #if FOO
     13 #ifdef __cplusplus
     13 #if AFTER
     12 #if os(OSX) || os(iOS) || os(watchOS) || os(tvOS)

Looks like the biggest chunk is "do we have an ObjC runtime or do we need a swift fallback?". It might thus be useful to classify failures due to non-objc systems :-)

[modocache]

Yes, please! Many of the failures I've seen when porting Swift to a new platform have been due to holes in xfails: #1090, #1032, #1119.

[glessard]

@jrose-apple Here's a .noObjC case for TestPredicate: #1130