Building a declarative router for Swift Vapor

As I’ve been using Swift Vapor 3, one of the things that I felt could be better was the way routes were declared. So I built a framework to meet my Vapor routing needs. If you’d like to try it out, go to the Github page and checkout the README. For the rest of this post, I’m going to talk about what my goals were, and highlight parts of the implementation that were interesting to me.

But first, to whet your appetite, here’s a tiny example of declaring routes using my framework:


import Vapor
import RoutingTable

public func routes(_ router: Router) throws {
    let table = RoutingTable(
        .scope("api", middleware: [ApiKeyMiddleware()], children: [
            .resource("users", parameter: User.self, using: UserController.self, children: [
                .resource("sprockets", parameter: Sprocket.self, using: SprocketController.self),
                .resource("widgets", using: WidgetController.self)
            ]),
            .resource("sessions", using: SessionController.self),
            .post("do_stuff", using: StuffController.doStuff)
        ])
    )
    table.register(routes: router)
}

Goals

I had four goals when creating my routing framework:

All routes in one file

One of the challenges I ran into was determining what all the routes in the app were and how they fit together. It is possible to print out all the routes by running a command line tool, but that didn’t help me with finding where the associated code was.

I also attempted to take advantage of RouteCollections at one point in order to make my routes() method less verbose. It did improve the verbosity, but at the expense of all the routes in one place. Ideally, I’d like to have my cake and eat it, too: all routes in one file, but expressed concisely.
Hierarchical declaration

Routes are hierarchical by nature, and I’d like to declare them that way. By that, I mean building a tree data type that is hierarchical in Swift syntax when declared, as opposed to calling a series of methods that build up a tree at runtime.

I see a couple of benefits from making the route declaration hierarchical. First, it’s easier for me to see how the endpoints fit together or relate to one another. I can see the hierarchy in the code syntax itself, instead of parsing method calls to build up the hierarchy in my head. Second, it can reduce boilerplate code by inheriting configuration from the parent to the child.
Re-usability of controllers

By re-usability of controllers, I mean a controller can be used in more than one place in the routing hierarchy. For example, maybe a controller implements managing sprockets. It could be exposed in one place in the routing hierarchy for normal users, but also in a different place for admin users. Part of making this useful would be allowing the routing declaration to specify which controller endpoints are available at each place in the hierarchy. e.g. the admin sub-hierarchy should allow the DELETEing of sprockets, but the normal user’s sub-hierarchy shouldn’t.

Being re-usable implies controllers don’t know where they are in the route hierarchy. To me, this makes sense because of my iOS/macOS background. In that context, view controllers don’t know where they appear in the app. Instead, app architecture relies on Coordinators (or a similar pattern) to know when and where to display the view controllers. Because of the separation of concerns, view controllers can be re-used in multiple places of the app. I think of API/web controllers in the same way.
Use higher level concepts

In my experience, few things reduce boilerplate code and increase readability like the introduction of higher level concepts. In the case of routing, I’m thinking about scopes and resources.

Scopes add the ability to group routes logically, so that the code maintainer knows they fit together to accomplish a common goal. It also means routes in a scope can inherit the same path prefix and/or middleware. Some examples of scopes could be an API scope or an admin user scope.

Resources allow me to describe REST-like resources in a succinct way. Although resource declaration can be succinct, the code maintainer can infer a lot about it. That’s because REST-like resources are known to support certain subpaths and HTTP methods that behave in specific ways. So although fewer things are declared about a resource, more is known about it than if I had declared each route individually.

The Results

Based on these goals, I came up with an idealized version of what I wanted route declaration to look like:


scope "api", [ApiMiddleware.self] {
    resource "users", parameter: User.self, using: UserController.self {
        resource "sprockets", using: SprocketsController.self
    }
}

What I like about the above is it has just enough punctuation and keywords to make it readable, but no more than that is unnecessary. It also makes use of curly braces to denote hierarchy. When I’m reading Swift code, curly braces make my brain think parent/child relationship in a way other punctuation doesn’t. It also seems to help make Xcode do sane things with indentation.

Here I’m going to admit that much of my inspiration for what routing could be came from Elixir’s Phoenix framework, and its routing library. I feel like its route declaration is very close to my ideal. In addition, it supports more features, including the ability to generate full URLs from a named resource.

Unfortunately, I couldn’t achieve my ideal in Swift. The example I gave above isn’t valid Swift, nor was there a way to extend Swift to support it. A lot of Phoenix’s elegance and power comes from Elixir’s hygienic macros, which Swift doesn’t have.

Instead, here’s the closest I could come in Swift:


.scope("api", middleware: [ApiKeyMiddleware()], children: [
    .resource("users", parameter: User.self, using: UserController.self, children: [
        .resource("sprockets", parameter: Sprocket.self, using: SprocketController.self)
    ]),
])

It has a lot more punctuation and keywords than is really necessary to convey what I want. It also uses square brackets for arrays to denote hierarchy, which are a bit clumsy especially when used in Xcode. But given Swift’s limitations, I feel like it comes pretty close.

Interesting Implementation Bits

When implementing my declarative router I ran into some implementation hurdles that I thought were interesting enough to write down.

Verbosity reduction

One of my stated goals was to reduce the verbosity needed to declare all my routes. Some of the reduction came for free just by using higher level concepts like scopes and resources, and making the declarations hierarchical so common configuration could be shared. But all that space savings could be undone if I messed up the declaration API. I paid particular attention to leveraging Swift’s type inference and default parameters.

I realized early on that I needed a tree that was homogenous in type, but polymorphic in behavior. There are three types that could appear in a routing declaration: a scope, a resource, and a raw endpoint (i.e. a GET, PUT, PATCH, POST, or DELETE). Each of those has its own properties, and handles registering routes differently. Of those, both scopes and resources could have children, which could themselves be scopes, resources, or raw endpoints. That left me with a couple of options.

The first option that I considered was using an enum to represent the three types (scope, resource, and raw endpoint). Since Swift enums can have associated data, each value could contain all their necessary properties. However, enums had a couple of problems. First, they don’t allow default values on construction. Which means each declaration would have to specify all values even if they weren’t used. Second, eventually each enum value would have to register all the routes it represented, and since each enum type had to do that differently, there would be a giant switch statement somewhere. That didn’t seem elegant to me, so I abandoned that approach.

The second option was to declare a common protocol (e.g. Routable) and have scope, resource, and raw endpoint types conform to that protocol. Then I had scopes and resources declare their children to be of type Routable so type homogenous trees could be built. That turned out to mostly work. The problem I ran into was the raw endpoints were more verbose than I wanted. For example:


Scope("api", middleware: [ApiMiddleware()], children: [
    RawEndpoint(.post, "do_stuff", using: StuffController.doStuff)
])

I felt having the typename RawEndpoint in the declaration was unnecessary and uninteresting. The important bit was the HTTP method, but that was obscured by the typename. My next thought was use the HTTP method name as the typename (e.g. Post, Get, etc). This worked, but at a cost. First, it meant I had five different types that all did the same thing, except for one parameter. Second, the HTTP method names are common words and had to exist at the top level scope. This made me worried about typename collisions.

I tried to fix those problems by adding static methods to my protocol as a shorthand way to create the raw endpoint type like so:


extension Routable {
    static func get<T>(_ path: String..., using controller: T) -> Routable {
        return RawEndpoint(.get, path, using: controller)
    }
}

However, when I tried to use the static methods in a declaration:


Scope("api", middleware: [ApiMiddleware()], children: [
    .post("do_stuff", using: StuffController.doStuff) // ERROR
])

Swift complained, seemingly because it couldn’t infer the type because the static methods were on a protocol. I could have specified the full type name to the method, but I felt that would have made the declaration too verbose. But I thought I was close to something that would work. I just needed the type inference to work on the static methods.

That lead me to the final option that I actually used. My hunch was I needed use a concrete type rather than a protocol in my declaration API. That would allow me to use static methods in the declaration and Swift’s type inference would work. To put it another way, I could make this work:


Scope("api", middleware: [ApiMiddleware()], children: [
    .post("do_stuff", using: StuffController.doStuff)
])

If children was declared to be an array of a concrete (i.e. non-protocol) type, and if post() were a static method on that concrete type.

The challenge now was I needed two seemingly opposed concepts. I needed each declaration item (i.e. scope, resource, raw endpoint) to be polymorphic since they each should act differently based on their type. I had achieved that via making them conform to a common protocol. However, in order to make Swift type inference happy, I needed a concrete type.

So I used type erasure, kind of. I wrapped the protocol Routable in a struct called AnyRoutable. It works like a type erasure datatype in that it implements the Routable protocol by calling the methods on the Routable instance it contains. This gave me a single concrete type while still allowing polymorphism.

To make this work, I essentially made AnyRoutable a facade to the rest of the framework. Every node in the routing tree would be declared as an AnyRoutable, which could, internal to the framework, wrap the correct declaration item type. To make building an entire tree from one type possible, I added static methods on AnyRoutable that created each declaration type: scope, resource, and each of the HTTP methods. For example, something like:


struct AnyRoutable {
    static func get<T>(_ path: String..., using controller: T) -> AnyRoutable {
        return AnyRoutable(RawEndpoint(.get, path, using: controller))
    }
}

The trick was I had the static methods deal only in AnyRoutable; children were declared as them, and the return types were AnyRoutable. Since all parameters and return types were a concrete type, Swift could easily infer the type, and the static methods could be called without them. Implementation wise, the static methods simply created the appropriate Routable subtype, then wrapped it in AnyRoutable. This had the added bonus of only needing to make AnyRoutable public in the framework. The Routable implementations for resource, scopes, and endpoints stayed hidden.

Although it took me a while to reach the final implementation, the pattern seems generally useful. It allows polymorphism in the implementation, while only exposing one concrete type to client code, which means type inference can be used. I suspect there might be other options for solving this problem. For example, I never tried a class hierarchy and I wonder if that could be made to work. However, I’m pretty happy with AnyRoutable since I got a facade pattern out of it as well.

Resource reflection

After designing the API interface, the next most difficult thing was figuring out how to implement controllers that managed REST-like resources. In a nutshell, if a router declares a resource, my framework should be able to look at the implementing controller and determine what verbs (i.e. index, show, update, create, delete, new, or edit) it supports, and automatically declare those routes. I wanted this to require as little boilerplate as possible. As a bonus, I wanted to determine the verb support at compile time.

I quickly realized that there would have to be some overhead no matter what, just because of the limitations of Swift. Swift has no way of asking “does this type implement this method” outside of having that type declare conformance to a protocol that requires the method. It doesn’t have a fully dynamic and reflective runtime like Objective-C, nor a structural type system. So I accepted that resource controllers would have to declare conformance to a protocol for each of the verbs it supported.

But even accepting that limitation, I still wanted to determine which verbs were available at compile time. Since the controller declared conformance to a protocol, and a generic method could require parameter conformance to a protocol, for a long time I held out hope this this was possible. For example, I could do this:


func indexRoutes<ControllerType: ResourceIndexable>(_ controller: ControllerType) -> [Route] {
    // generate route for index verb
} 

func indexRoutes<ControllerType>(_ controller: ControllerType) -> [Route] {
    return []
} 

class MyController: ResourceIndexable {
}

let controller = MyController()
let routes = indexRoutes(controller) // calls the correct method

The issue I ran into was trying to put all the verb routes together. The method that put all the routes together would have to know about all the resource verb protocols it conformed too. That’s because if the method didn’t require conformance to a verb protocol, Swift would act like it didn’t conform to that protocol regardless of it actually did or not. So for this to work, I would have to implement a generic method for every combination of the seven verb protocols that could occur. That seemed excessive to me. In the end, I simply had the code query at runtime which protocols the controller implemented.

This part was interesting to me because it seems like it should be solvable problem. However, in its current state, Swift doesn’t appear able to overcome it. I do wonder if a hygienic macro system would make it feasible.

Dependency injection woes

The next two implementation struggles relate to the implementation details of Vapor itself, and not something inherent to the issue of building a declarative router. But they still contained valuable learnings.

At a base level, the router connects routes with a controller that will handle requests to that route. To make the route declaration as concise as possible, I only required the type of the controller, as opposed to a live instance that might have be constructed and/or configured. I decided I would instantiate the controller later, when registering the route, using Vapor’s dependency injection system. The problem was when it came to register the route, Vapor’s dependency injection system was in the wrong state.

Vapor’s dependency injection effectively has two phases. The first phase is the setup phase, where it registers types it calls “services” which can be instantiated later, in the second phase. In the second phase, a “container” exists, and can be used to instantiate any of the service types registered in the first phase. When the router is being initialized, the system is in the setup phase, and can’t instantiate any of the controller types because there’s no container.

I considered using my own custom protocol on controllers that allowed them to be constructed without a dependency injection container. However, after trying that out, it seemed surprising in that everything else uses Vapor’s DI system. Plus my custom protocol would be more restrictive; it wouldn’t be able to allow custom init parameters to the controller (unless all controllers needed them), nor would it offer access to the DI system to allow construction of said parameters.

In the end, I was able to defer controller allocation until the first route was actually executed. By then, containers were available. Further, Vapor’s DI system took care of caching the controller for me.

This problem was interesting because I’ve run into it a few times in Vapor. I need to be on the lookout for different patterns to work around not having a DI container available when I need it.

Testing difficulties

The final issue I ran into was trying to unit test my framework. Since it’s implemented in Swift, my go to method is to define all dependencies as protocols, then for my tests to create fakes I can manipulate and observe.

Unfortunately, most of the Vapor types I had to interact with weren’t easily fakeable. The big one I needed to fake was Vapor’s Router protocol. Being a protocol, I thought it would be a slam dunk. Unfortunately, all the interesting methods on Router were in protocol extensions, meaning my testing fakes could not override those methods.

What I ended up doing was defining my own protocol for the methods on Router that I used, then using that everywhere in my code. Since the methods were declared in the protocol, they were overridable by my testing fakes. This allowed me to do the unit testing I wanted. That only left the issue of how to use Vapor’s Router in the non-testing version of the framework.

Indirection solves a lot of problems, this being another example. I declared a concrete type that conformed to my router protocol that my framework could use. I then had it wrap a Vapor Router, and implement the protocol by calling the corresponding methods on Router. Then, at the top level object, I hid the wrapping of Router behind a helper method, so it looked like my framework dealt with Routers natively.

The lesson I take from this is testing when relying on 3rd party frameworks is hard. Unit testing with UIKit and AppKit types is no different. Also, defining my own protocol and wrapping a 3rd party protocol in a concrete type to conform to it seems like a repeatable strategy.

Conclusion

When using Swift Vapor’s routing API, I discovered I wanted a few more things than it offered. Namely, I wanted all routes declared in one file, a hierarchal declaration, re-usability of controllers, and higher level concepts like scopes and resources. In building a framework to support these concepts, I ran into a few implementation concerns. The first was learning to design an API in such a way to reduce verbosity. The others were trying to determining protocol conformance at compile time, working around two-stage dependency injection limitations, and trying to test my code while not having fakeable protocols from 3rd party frameworks.

Safe from the Losing Fight

mac and ios development

About