I help maintain big OSS projects: from a third-party dependency manager used in most iOS apps (CocoaPods), to the most popular Objective-C testing framework and the most popular Swift networking API client. I’ve been doing this for years.
Projects with this much impact are big time-sinks. This time comes from ensuring infrastructure continues to work, support tickets need replies, new code needs reviewing and releases need coordinating.
Last September, almost a year ago, I started work on a new project, Danger. Danger fixes a problem we were seeing in the Artsy mobile team around adding “process” to our team.
As a part of discussing Danger internally, I’ve referenced that building CocoaPods has greatly influenced Danger. This blog post is about the way I’ve built Danger, knowing fully well that I cannot afford the level of time to maintain it at the scale it may get to.
Danger is a project that could end up with a lot more users than CocoaPods. So I want to be cautious about how I create the community around Danger. If you’re interested in some of the baseline setup required to run a popular project, the post “Building Popular Projects” by Ash is a great place to start, this builds on that.
My maintenance time on CocoaPods resolves around:
- Handling new issues
- Keeping infrastructure running
- Requests around user data
- Keeping disparate communities together
Issues
From the ground up, Danger could not end up as complex as CocoaPods, the domain they cover is different and CocoaPods sits atop of an annually moving (and somewhat hostile) platform.
However, get enough people using a product and you end up with three types of issues: Bug Reports, How Do I X? and Feature Requests.
I wanted to keep bug-reports down, as much as possible, and so I built a system wherein the default error reporting system would also search GitHub issues for similar problems. Knowing this was a generic problem, I built it with other large ruby projects in mind too.
How do I X? are issues that haven’t appeared much on Danger. For CocoaPods we request people use the CocoaPods tag on StackOverflow. That saves us from 5 to 6 issues a day, and provides others a great place to get internet points by responding instead.
Feature Requests issues are always fascinating, it gives you a chance to really see the difference between what you imagined a project’s scope is, and how others perceive it. One thing that helps here is that Danger has a VISION.md file. This helped vocalise a lot of internal discussion, and let contributors understand the roadmap:
The core concept is that the Danger project itself creates a system that is extremely easy to build upon. The codebase for Danger should resolve specifically around systems for CI, communication with Peer Review tools and providing APIs to Source Control changes. For example: Travis CI - GitHub - git.
As well as providing a heuristic for determining whether something should be added to Danger:
This means that decisions on new code integrated into Danger should ask “is this valid for every CI provider, every review system and source control type?” by making this domain so big, we can keep the core of Danger small.
Infrastructure
CocoaPods has about 6 web properties, 3 of which are critical. The others can go down, or be behind the Xcode update schedules and people’s projects will work fine. The 3 the critical projects are all simple, focused projects: trunk (provide auth, and submitting new libraries) cocoapods.org, and search. We control everything there.
Meanwhile the less critical ones like cocoadocs.org have dependencies all over the show: AppleDoc, CLOC, Xcode, Carthage, Jazzy - every one of these can, and has, been a source of unreliability for infrastructure that I maintain.
With Danger, I wanted to avoid building any infrastructure that does not sit on top of solid, mature projects. The website is statically created in Middleman, using boring old technology, this means no server to host.
To support dynamic content on the website, I have a rake command to use a decade old documentation formatter to pull content from a 13 year old dependency manager - that lets others describer their project’s. In order to let them keep it up to date, I have a tiny 35 line web-server that allows specific projects to trigger a new CI build.
Plugins
Summary of every big OSS project. Monolith -> Plugin support.https://t.co/7x4vuW4bRd
— Orta Therox (@orta) 30 June 2016
It’s almost inevitable that once a project becomes big, maintainers have to become a lot more conservative about how they introduce new code. You become good at saying ‘no’, but a lot of people have legitimate needs. So, instead you end up converting your tool into a platform.
Some of the most interesting ideas in the CocoaPods ecosystem come from plugins.
I wanted plugins to be a first class citizen within Danger from day one. It’s in the VISION file, and it’s applied into how I’ve designed a lot of the user-facing site. I was torn after a few months of development where things should go. Now the core of Danger is built as plugins.
Documentation
My second big project on CocoaPods was collating documentation and scoping different types of documentation. In CocoaPods I ended with:
- Highlight pages (intro pages, app pages, team pages)
- Guides (tutorials, overviews, FAQs)
- Reference (Command-line interface, APIs for developers)
These 3 buckets for documentation makes it pretty easy to separate where people should look depending on what they’re looking for. This pattern I’m stealing outright for Danger. Just not quite yet, it’s a blocker on 1.0 though.
One trick I took from CocoaPods is to have as much documentation as possible generated from the source code. With Danger, all of the work that’s gone into documenting the code is turned into public API documentation for end-users. This makes it really easy to ensure it’s consistent and up-to-date. The same tools used to generate documentation for Danger are used for plugins. Any improvements there helps everyone.
User Data
Not storing any, phew! Though if Danger as a Service happens, then it will.
People
People are hard, Ash said in Building Popular Projects:
The biggest existential threat to your library is this: you get burned out and stop working on it – and no one else contributes to it –
Understanding motivations, encouraging ownership and accommodating multiple viewpoints are vital parts of anyone who wants to make a project bigger than themselves. There are lots of times when I’m not the lead developer on Danger.
I owe a lot of this to the policy Ash and I created with Moya, the wordy “Moya Community Continuity Guidelines” which define the expectations of the maintainers of a project towards contributors.
It’s helped let a lot of other contributors make an impact. In the future, I hope those are the people that I get to hand Danger off to. Danger is bigger than me.
Maintaining big projects is a learned activity, for most people it’s a spectator sport, but it’s not too hard to jump from writing issues to helping out. It’s how I ended up contributing to CocoaPods.