All concerning the Swift Package deal Supervisor and the Swift toolchain

If you do not know an excessive amount of concerning the Swift Package deal Supervisor, however you might be in search of the fundamentals please learn my tutorial about SPM that explains just about every thing. The intention of this text is to go deep into the SPM structure, additionally earlier than you begin studying this I might suggest to additionally learn my article about frameworks and instruments. 📖

Prepared? Go! I imply Swift! 😂

Swift Package deal Supervisor

Have you ever ever questioned about how does SPM parse it is manifest file with a view to set up your packages? Nicely, the Package deal.swift manifest is a wierd beast. Let me present you an fast instance of an everyday package deal description file:

import PackageDescription

let package deal = Package deal(
    identify: "HelloSwift",
    dependencies: [
        
    ],
    targets: [
        .target(
            name: "HelloSwift",
            dependencies: []),
        .testTarget(
            identify: "HelloSwiftTests",
            dependencies: ["HelloSwift"]),
    ]
)

The primary line accommodates the model data, subsequent we’ve to import the PackageDescription module which accommodates all of the required components to correctly describe a Swift package deal. Should you run for instance swift package deal replace all of your dependencies on this manifest file shall be resolved & you should utilize them inside your personal code recordsdata. ✅

However how on earth are they doing this magic? 💫

That query was bugging me for some time, so I did a little analysis. First I used to be attempting to duplicate this conduct with out wanting on the unique implementation of the Swift Package deal Supervisor at GitHub. I knew I should not parse the Swift file, as a result of that’d be a horrible factor to do – Swift recordsdata are messy – so let’s attempt to import it by some means… 🙃

Dynamic library loading strategy

I looked for the “dynamic swift library” key phrases and located an fascinating discussion board matter on swift.org. Yeah, I am making some progress I assumed. WRONG! I used to be means farther from the precise resolution than I although, but it surely was enjoyable, so I used to be wanting into the implementation particulars of find out how to open a compiled .dylib file utilizing dlopen & dlsym from Swift. How does one create a .dylib file? Ah, I already know this! 👍

I at all times needed to grasp this matter higher, so I began to learn increasingly more each about static and dynamic libraries. Lengthy story quick, you possibly can create a dynamic (or static) library with the next product definition:

import PackageDescription

let package deal = Package deal(
    identify: "instance",
    merchandise: [
        .library(name: "myStaticLib", type: .static, targets: ["myStaticLib"]),
        .library(identify: "myDynamicLib", kind: .dynamic, targets: ["myDynamicLib"]),
    ],
    targets: [
        .target(
            name: "myStaticLib",
            dependencies: []),
        .goal(
            identify: "myDynamicLib",
            dependencies: []),
    ]
)

The necessary recordsdata are going to be situated contained in the .construct/debug folder. The .swiftmodule is mainly the general public header file, this accommodates all of the accessible API on your library. The .swiftdoc file accommodates the documentation for the compiled module, and relying on the kind you will additionally get a .dylib or a .a file. Guess which one is which.

So I might load the .dylib file by utilizing dlopen & dlsym (some @_cdecl magic concerned to get fixed names as a substitute of the “fuzzy” ones), however I used to be continually receiving the identical warning time and again. The dynamic loading labored nicely, however I needed to eliminate the warning, so I attempted to take away the embedded the lib dependency from my executable goal. (Trace: not likely doable… afaik. anybody? 🙄)

I used to be messing round with rpaths & the install_name_tool for like hours, however even after I succesfully eliminated my library from the executable, “libSwift*issues” have been nonetheless embedded into it. So that is the unhappy state of an unstable ABI, I assumed… anyway no less than I’ve realized one thing essential throughout the best way right here:

Importing Swift code into Swift!

Sure, you heard that. It is doable to import compiled Swift libraries into Swift, however not lots of people heard about this (I assume). It isn’t a well-liked matter amongs iOS / UIKit builders, however SPM does this on a regular basis behind the scenes. 😅

How on earth can we import the pre-built libraries? Nicely, it is fairly easy.

// utilizing swiftc with compiler flags

swiftc dynamic_main.swift -I ./.construct/debug -lmyDynamicLib -L ./.construct/debug
swiftc static_main.swift -I ./.construct/debug -lmyStaticLib -L ./.construct/debug

// utilizing the Swift Package deal Supervisor with compiler flags

swift construct -Xswiftc -I -Xswiftc ./.construct/debug -Xswiftc -L -Xswiftc ./.construct/debug -Xswiftc -lmyStaticLib
swift construct -Xswiftc -I -Xswiftc ./.construct/debug -Xswiftc -L -Xswiftc ./.construct/debug -Xswiftc -lmyDynamicLib

You simply should append a couple of compiler flags. The -I stands for the import search path, -L is the library search path, -l hyperlinks the given library. Examine swiftc -h for extra particulars and flags you will not remorse it! Voilá now you possibly can distribute closed supply Swift packages. No less than it was good to know the way SPM does the “trick”. 🤓

Please word that till Swift 5 & ABI stability arrives you should utilize the precompiled libraries with the identical Swift model solely! So in the event you compile a lib with Swift 4.2, your executable additionally must be compiled with 4.2., however it will change fairly quickly. 👏

The Swift Package deal Supervisor methodology

After 2 days of analysis & studying I actually needed to unravel this, so I’ve began to verify the supply code of SPM. The very first thing I’ve tried was including the --verbose flag after the swift construct command. Right here is the necessary factor:

/Purposes/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc 
--driver-mode=swift 
-L /Purposes/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 
-lPackageDescription 
-suppress-warnings 
-swift-version 4.2 
-I /Purposes/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 
-target x86_64-apple-macosx10.10 
-sdk /Purposes/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk 
/Customers/tib/instance/Package deal.swift 
-fileno 5

Whoa, this spits out a JSON based mostly on my Package deal.swift file!!! 🎉

How the hell are they doing this?

It seems, in the event you change the -fileno parameter worth to 1 (that is the usual output) you possibly can see the outcomes of this command on the console. Now the trick right here is that SPM merely compiles the Package deal.swift and if there’s a -fileno flag current within the command line arguments, nicely it prints out the encoded JSON illustration of the Package deal object after the method exits. That is it, fuckn’ simple, but it surely took 1 extra day for me to determine this out… parenting 2 children & coding is a tough mixture. 🤷‍♂️

Should you open the /Purposes/Xcode.app/Contents/Developer/ Toolchains/XcodeDefault.xctoolchain/ usr/lib/swift/pm/4_2 folder you will see 3 acquainted recordsdata there. Precisely. I additionally seemed on the supply of the Package deal.swift file from the SPM repository, and adopted the registerExitHandler methodology. After a profitable Package deal initialization it merely registers an exit handler if a -fileno argument is current encodes itself & dumps the outcome by utilizing the file handler quantity. Candy! 😎

Since I used to be just about within the end lap, I needed to determine yet another factor: how did they handle to place the swift package deal command below the swift command?

Swift toolchain

I simply entered swift lol into my terminal. That is what occurred:

tib@~: swift lol
error: unable to invoke subcommand:
/Purposes/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-lol
(No such file or listing)

Bought ya! The toolchain is the important thing to every thing:

1. Apple is compiling the PackageDescription library from the Swift Package deal Supervisor and places the .swiftmodule, .swiftdoc, .dylib recordsdata into the right locations below Xcode’s default toolchain library path.
2. The swift construct, run, check subcommands are simply one other Swift binary executables positioned contained in the toolchain’s binary path. (Named like: swift-package, swift-build, swift-run, swift-test)
3. The swift command tries to invoke the right subcommand if there may be any and it is a legitimate (Swift) binary. (Tried with a shell script, it failed miserably…)
4. SPM makes use of the PackageDescription library from the toolchain with a view to compile & flip the manifest file into JSON output.
5. The remaining is historical past. 🤐

Swift can resolve subcommands from anyplace “inside” the PATH variable. You simply should prefix your Swift script with swift- and also you’re good to go.

SwiftCI – a process runner for Swift

I had this concept that it would be good to have a grunt / gulp like process runner additionally a steady integration service on a long run by utilizing this method I defined above. So I’ve made the same extension wired into the center of the Swift toolchain: SwiftCI. ❤️

You may seize the proof-of-concept implementation of SwiftCI from GitHub. After putting in it you possibly can create your personal CI.swift recordsdata and run your workflows.

import CI

let buildWorkflow = Workflow(
    identify: "default",
    duties: [
        Task(name: "HelloWorld",
             url: "[email protected]:BinaryBirds/HelloWorld.git",
             model: "1.0.0",
             inputs: [:]),

        Process(identify: "OutputGenerator",
             url: "~/ci/Duties/OutputGenerator",
             model: "1.0.0",
             inputs: [:]),

        Process(identify: "SampleTask",
             url: "[email protected]:BinaryBirds/SampleTask.git",
             model: "1.0.1",
             inputs: ["task-input-parameter": "Hello SampleTask!"]),
    ])

let testWorkflow = Workflow(
    identify: "linux",
    duties: [
        Task(name: "SampleTask",
             url: "https://github.com/BinaryBirds/SampleTask.git",
             version: "1.0.0",
             inputs: ["task-input-parameter": "Hello SampleTask!"]),
        ])

let mission = Undertaking(identify: "Instance",
                      url: "[email protected]:BinaryBirds/Instance.git",
                      workflows: [buildWorkflow, testWorkflow])

The code above is a pattern from a CI.swift file, you possibly can merely run any workflow with the swift CI run workflow-name command. Every thing is 100% written in Swift, even the CI workflow descriptor file. I am planning to increase my CI namespace with some useful sub-commands in a while. PR’s are greater than welcomed!

I am very pleased with the outcome, not simply due to the last product (that is solely a proof of idea implementation), however largely due to the issues I’ve realized in the course of the creation course of.