SR-1005: [Foundation] Use NSFileHandle instead of fopen

[Bouke]

A partial implementation of SR-1005, replacing POSIX file handling by NSFileHandle.

Instead of passing file descriptors, references to NSFileHandle are passed around. fclose has been replaced by closeFile(), however not strictly necessary as NSFileHandle automatically closes the pointer on deallocation.

Rationale for reading the whole file when enumerating the lines:

• Iterating over a File instance would read using a buffer,
  however most usages would concatenate the chunks
  and work on the complete file (no memory savings).
• Files that are read are mostly small files, should not
  take lots of memory.

Remarks:

• Tested on both OSX and Linux. The SwiftFoundation has some differing API, #if os(OSX) has been used for in the interim.
• Performance of the unit tests did not change.
• The PR has been based on the latest development snapshot.

[mxcl]

LGTM but conflicts.

[Bouke]

Indeed it does, it was written against the latest development toolchain snapshot. The Foundation API is being imported now with new rules, so I'm unable to compile until I have an updated toolchain. I've yet to learn to build my own toolchain, maybe I'll just wait for a new toolchain snapshot.

[Bouke]

I've updated the PR and tested on OS X and Linux against Swift's master branch.

[mxcl]

I use this script to build my own toolchain: https://gist.github.com/mxcl/05ced0268bbd5db6ab0c

[mxcl]

@ddunbar how do you feel about starting to depend on Foundation?

[ddunbar]

Let's do it! :)

[mxcl]

@swift-ci Please test and merge

[Bouke]

Any ideas why the CI reports a failure? From the log I can see that;

• On OSX: Some Swift test is failing (typeref_decoding_objc.swift), is this related to this PR?
• On Linux: cannot import Foundation, should SwiftPM require that somewhere in the build scripts?

[kostiakoval]

@Bouke the abspath is changed to a property in other PR, #304
Maybe it has something to do with this error. Could it be a CI error?

[briancroom]

Hey @Bouke, you are correct that the build scripts will need some changes to facilitate a dependency on Foundation here. Here are some notes and source references that should help get you started on this:

• The PRODUCTS array in build-script-impl needs to be set up to contain Foundation before SwiftPM (https://github.com/apple/swift/blob/master/utils/build-script-impl#L993)
• Currently, build-script-impl and SwiftPM's bootstrap require the Foundation and XCTest build paths to be passed in as a pair. It may make sense to adjust them to handle just a Foundation build path if only that one is provided (https://github.com/apple/swift/blob/master/utils/build-script-impl#L1309 and https://github.com/apple/swift/blob/master/utils/build-script#L997)
• Similar to what being done for XCTest, build-script could be made to infer building Foundation if SwiftPM is requested (https://github.com/apple/swift-package-manager/blob/master/Utilities/bootstrap#L630)

[briancroom]

As a side note (and this may not be the right place for this discussion..), @parkera mentioned recently that a longer-term goal may be to make SwiftPM able to build "all the things on all the platforms". Does a Foundation dependency potentially interfere with that?

@mxcl @ddunbar

[parkera]

I'm hoping that Foundation is available everywhere that Swift is, so using it should not prevent portability of swiftpm.

[parkera]

In fact, Foundation has an important role to play as an insulator of those same platform differences.

[briancroom]

Sure, that makes sense. So to be clear, you're invisioning that swift-corelibs-foundation will keep its own build system indefinitely, while swift-corelibs-xctest could eventually migrate to be built by SwiftPM, since SwiftPM has no build-time dependency on XCTest. Is that right?

[Bouke]

@briancroom thanks for the pointers in the build script. I'm however stuck at building SwiftPM's stage1, which calls out to swift-build-tool.

swiftpm/Sources/POSIX/mkdir.swift:19:12: error: no such module 'Foundation'
import Foundation
       ^

swift-build-tool doesn't expose any -Xlinker arguments like other tools do. I can set LD_LIBRARY_PATH, but that doesn't have any effect. I symlinked Foundation.swiftmodule, Foundation.swiftdoc and libFoundation.so into build/Ninja-ReleaseAssert/swiftpm-linux-x86_64/.bootstrap/modules, but now it cannot find CoreFoundation:

swiftpm/Sources/OptionsParser/OptionsParser.swift:12:12: 
    error: missing required module 'CoreFoundation'
import Foundation
       ^

Update
I've also symlinked usr/lib/include/CoreFoundation. I'm not sure if this is the correct way?

[ddunbar]

You need to make sure Foundation is accessible through the search paths used during the stage1 build.

The stage2 build does this by setting up a dummy toolchain inside the build directory, you might be able to just move that logic early and then use that.

[parkera]

We have a kind of circular dependency today in the build system for all of the OSes: Foundation builds using xcodebuild, which uses Foundation. Once we get the entire system bootstrapped I don't see a real issue with having these depend on each other. Foundation can continue to use its custom python scripts in the meantime though.

[Bouke]

Rebased the PR. The tests should run fine now Foundation is available.

[ddunbar]

Conflicts again. :(

[ddunbar]

@swift-ci please test and merge