Artsy Folio, our free iPad app for Gallery Partners, had been in the App Store for a couple of weeks before the iPad with a Retina display was announced. This had been something we expected internally and felt the application would be ready. We had all our image assets available in @2x versions and an image pipeline that would take scaling into account. With that in mind, we changed our artwork grid view to show a double resolution image. Finally, once we were happy that it worked fine on the simulator, we sent the build off to Apple for review.

The app passed review, and was Retina-ready before the actual release. But within hours of getting our hands on a real Retina iPad, we had to pull the app. This post will explain why, and what we did to work it out.

Scrolling the grid view was slow. Extremely slow. The reason why wasn’t obvious initially, but thanks to digging around using Instruments, we saw that a great deal of time was spent in Apple’s image processing libraries. This was a strong hint that the problem involved taking the file and getting it to the screen.

In our naiveté, Folio was originally using UIImage’s initWithContentsOfFile: to load (without caching) a jpg from the file system. Once the file was loaded into memory, we displayed it onscreen in an UIImageView. This was fast enough to deal with our small thumbnails of 240x240 but the moment that you start asking it to pull 3 or 4 480x480 jpg files off the filesystem, decompress them and then put them on the screen, you’re not going to have a smooth scroll.

As we knew that we were looking at an issue with getting images from a file, it made sense to start looking at ways to move image processing off the main thread. This Stack Overflow thread on UIImage lazy loading proved to be an essential start to dealing with our issue. We needed a thread-safe way to get the contents of a file and to pass them through once the images had been decoded. What we needed was initImmediateLoadWithContentsOfFile, a thread-safe way to go from a filepath to a UIImage.

Now that we had a way to get an image that was safe to go on a background thread, we gave our grid an NSOperationQueue and created a method to kick off a NSInvocationOperation with our the cell we’re looking at and the address it needs to load the thumbnail.

- (void)setImageAsyncAtPath:(NSString *)imageAddress forGridCell:(ARImageGridViewCell *)cell {
    NSDictionary *operationOptions = @{@"address": imageAddress, @"cell": cell};
    NSInvocationOperation *operation = [[NSInvocationOperation alloc] initWithTarget:self selector:@selector(asyncLoadImage:) object:operationOptions];

    [_operationQueue addOperation:operation];
}

When we had the simplest implementation of asyncLoadImage we found that scrolling would sometimes result in grid cells displaying the wrong image. It turned out that in the time it took to decode the jpg, the cell had already been reused for a different artwork. This one totally caught us off guard!

- (void)asyncLoadImage:(NSDictionary *)options {
    @autoreleasepool {
        NSString *address = options[@"address"];
        ARImageGridViewCell *cell = options[@"cell"];

        // don't load if it's on a different cell
        if ([cell.imagePath isEqualToString:address]) {
            UIImage *thumbnail = [[UIImage alloc] initImmediateLoadWithContentsOfFile:address];

            // double check that during the decoding the cell's not been re-used
            if ([cell.imagePath isEqualToString:address] && thumbnail) {
                [cell performSelectorOnMainThread:@selector(setImage:) withObject:thumbnail waitUntilDone:NO];
            }
        }
    }
}

This meant we could have our UI thread dealing with scrolling, whilst Grand Central Dispatch would deal with ensuring the image processing was done asynchronously and as fast as possible.However, this still wasn’t enough. We were finding if you scrolled fast enough, you could still see images pop in after the grid cell was visible. For this, we actually went back to the beginning, and made our image pipeline create a 120x120 thumbnail for each artwork that we use initImmediateLoadWithContentsOfFile to load on the UI thread. This is fast enough to smoothly scroll, and is replaced by the higher resolution image practically instantly.

The rest of the story is pretty straightforward. We wrapped all this up within a few days and got out a version of Folio for the Retina iPad, I ended up doing a talk about the issues involved in doing this in Leeds LSxCafé, and you got a blog post out of it.

At Artsy Engineering we encourage a culture of experimentation with something called labs.

A new feature released into production is usually only turned on for a handful of users. We get feedback from our own team and a tiny group of early adopters, iterate, fix bugs, toss failed experiments and work on promoting complete, well behaved features to all users. The labs infrastructure gives us a chance to sleep on an idea and polish details. It also allows us to make progress continuously and flip a switch on the very last day.

My favorite labs features push our collective imagination and give birth to productive brainstorms around coffee at a popular startup hangout around the corner from our Manhattan office. But the team’s favorite labs are, by far, those that ship as easter eggs. These are fun and sometimes useful features that don’t make much business sense. So, before I explain our rudimentary labs system, I want to invite you to our easter egg hunt. Check out https://artsy.net/humans.txt for instructions.

Read on →

A mean bug made it into our production environment. It wasn’t caught by our extensive test suite and caused thousands of emails to be sent to a handful of people. The root cause was an unfortunate combination of Devise, DelayedJob and, of course, our own code. It was an easy fix, but nobody ever wants this to happen again.

tl;dr DelayedJob says it’s possible to set Delayed::Worker.delay_jobs = false for your tests. Don’t do it.

Read on →

With the release of Xcode 4.4 I’ve taken a look back at our existing code standards and tried to come up with something that is cleaner and more elegant. Here are a few of the ideas I’ve been using to modernize the codebase.

Remove private method declarations and use class extensions to add ivars.

First to get chopped by the deletion button are private method declarations. After Xcode 4.2 came out we took to using the class extension feature to add private method declarations at the top of implementation files. This was a nice way of keeping private methods out of the header files. Now that the compiler will check for pre-existing method signatures within the same object there’s no need to define their interfaces.

Occasionally it’s necessary for subclass to know about private methods defined by its superclass, so we use a shared category to let them know what they respond to. Like Apple, we also quit using @private in header files.

Ivars now should go in class extensions, and be prefixed by an underscore. Apple advises that you don’t use method names with underscores but encourage underscored variable names. This also can free up method parameters from having ugly names such as anArtwork or aString.

Use object literals when possible.

Object literals are ways of adding syntacitcal sugar to the Objective-C language, they let you access keys and values easily on NSDictionarys and objects in NSArrays. There’s no reason to not be using them if you’re supporting iOS 4 and above. It’s simple a matter of _artworks[2] vs [_artworks objectAtIndex:2].

Dot syntax is OK for non-properties.

OK so, I admit it. I whined when properties came out. It was back in 2007 and the Objective-C was ranked 40th in the world, it’s now ranked 3nd most popular programming language. Within timeframe, my opinion on the subject of properties changed also.

Originally when properties came out they exclusively were given the right to use dot notation on objects. This makes sense as they were created to provide public access to ivars which normally you can only access internally using the dot notation. With Xcode 4.3, that also changed. Now, if a method doesn’t have any arguments it can be called using dot notation. I’m in favour of using this. For me a good rule of thumb has been if a method returns something, dot notation is OK. For example, _artworksArray.count is fine whilst _parsingOperation.stop isn’t.

Keep external code out of your project.

External, or vendored code should be kept out of the main body of your code. You can use CocoaPods to keep all that code in check and up-to-date. CocoaPods is a project that aims to be what bundler does for ruby projects, or npm for node. It will deal with your dependancies whilst you can concentrate on your own code. It will create a seperate Xcode project that handles all you dependancies leaving your project only as your own code.

Use umbrella imports.

To try and keep the amount of noise we have at the top of our implementation files we have started to reduce the number of #import "ARModel.h" lines we use. By creating a Models.h file and having that include all the models it means we can still have a look through the #imports at the top to get an idea of the connections between the objects as that will only show the important imports. These can optionally be moved into your precompiled header files.

Keep your code clean.

Whitespace can and does slowly accumulate at the line-endings of your code. You should make sure that the new preference for automatically trimming whitespace is turned on in the text editing section of Xcode’s preferences.

IBOutlets should probably go in your class extensions.

With modern versions of Xcode, it doesn’t matter that your IBOutlets are defined in places other than in headers. As Objective-C developers, we’ve come a long way from having to repeatedly drag a .h from Xcode to Interface Builder, so maybe it’s time to rethink the idea that our interface outlets should be publicly accessible. My opinion is that a controller’s views are for the most part a private affair and that if you want to expose their functionality you do it through custom methods in the controller. There are some downsides to this, in that initially you have to change to the implementation file when using hybrid view when connecting them.

These decisions have come from internal discussions and from watching many WWDC sessions on the topic. We highly recommend watching the following WWDC sessions.

WWDC 2011: 105 Polishing Your App, 112 Writing Easy To Change Code and 322 - Objective-C Advancements in Depth.

WWDC 2012: 405 Modern Objective-C and 413 Migrating to Modern Objective-C

The Artsy team faithfully uses Jenkins for continuous integration. As we’ve described before, our Jenkins controller and 8 agents run on Linode. This arrangement has at least a few drawbacks:

• Our Linode servers are manually configured. They require frequent maintenance, and inconsistencies lead to surprising build failures.
• The fixed set of agents don’t match the pattern of our build jobs: jobs get backed up during the day, but servers are mostly unused overnight and on weekends.

The Amazon EC2 Plugin allowed us to replace those agents with a totally scripted environment. Now, agents are spun up in the cloud whenever build jobs need them.

Read on →

Empathy with end users is critical when developing consumer-facing software. Many go even further and argue that you should be your own user to effectively deliver the best experience.

I’d encourage anyone starting a startup to become one of its users, however unnatural it seems.

— Paul Graham Organic Startup Ideas

In practice, though, this can be difficult:

• As a developer, you’re just not representative of the intended audience.
• You’re [appropriately] focused on the product’s next iteration, while your audience is occupied with the current state.
• You spend countless hours focused on product details—of course it’s a challenge to empathize with a casual visitor’s first impression.

Keeping it Real

We’ve tried some best practices to overcome these tendencies. User feedback is emailed to everyone in the company. Engineers share customer support responsibilities. But one simple tool has been surprisingly useful: we stole a page from the agile development handbook and built an information radiator. Like a kanban board, news ticker, or analytics wall board, our information radiator gives us an ambient awareness of end users’ experiences. How?

Read on →

The only constant is change, continuing change, inevitable change, that is the dominant factor in society [and web apps!] today. No sensible decision can be made any longer without taking into account not only the world as it is, but the world as it will be.

– Isaac Asimov

R.I.P #!

It did not take us long to discover we shared the concerns of Twitter’s Dan Webb on hashbang routes, but it was almost a year before we were able to remove them from Artsy. Here’s how it went down.

Artsy relies on the Backbone.js framework for our client application which offers a solid pushState routing scheme. This includes a seamless hashtag fallback for browsers that don’t support the HTML5 History API (looking at you IE 9).

The pushState routing is optional, but “the world as it should be” suggests we say “Yes!” (or true) to pushState.

Backbone.history.start({ pushState: true })

Read on →

Implementing server-side RESTful API caching is hard. In a straightforward API all the expiry decisions can be made automatically based on the URL, but most real world APIs that add requirements around object relationships or user authorization make caching particularly challenging.

At GoRuCo we open-sourced Garner, a cache implementation of the concepts described in this post. To “garner” means to gather data from various sources and to make it readily available in one place, kind-of like a cache! Garner works today with the Grape API framework and the Mongoid ODM. We encourage you to fork the project, extend our library to other systems and contribute your code back, if you find it useful.

Garner implements the Artsy API caching cookbook that has been tried by fire in production.

Read on →

The Jenkins CI project has grown tremendously in the past few months. There’re now hundreds of plugins and an amazing engaged community. Artsy is a happy user and proud contributor to this effort with the essential jenkins-ansicolor plugin, eliminating boring console output since 2011.

We are a continuous integration, deployment and devops shop and have been using Jenkins for over a year now. We’ve shared our experience at the Jenkins User Conference 2012 in a presentation. This blog post is an overview of how to get started with Jenkins for Ruby(-on-Rails) teams.

Read on →

E-mail is one of the most important ways to engage your users. And every time you touch a user’s inbox, it reflects on your brand. But getting email right has become increasing difficult due to the complexities introduced by the thousands of web-based, desktop and mobile mail clients. Email formatting is like the “Hunger Games” where the major players include online services such as GMail, Yahoo, Hotmail or AOL, desktop clients such as Outlook and a myriad mobile devices ranging from iPhone and Android to Blackberry.

To deal with this landscape, the MIME standard allows systems to send e-mail with multiple parts: plain/text for business-efficient devices such as the Blackberry, and text/html for web-based e-mail readers, such as GMail. Furthermore, ActionMailer supports multiple template formats: create an .html.haml template along with a .txt.haml template to generate both. We also know that text/plain email helps deliverability, but we believe a disproportionately small amount of text e-mails are actually read - the vast majority of devices are capable of parsing some HTML.

Is it possible to avoid having to maintain two separate templates without sacrificing deliverability? How can we inject a text/plain part into HTML e-mail that is both useful and “free”?

ActionMailer::Base defines an internal method called collect_responses_and_parts_order (#ref), which iterates over templates and renders them. Let’s override that method and examine the contents of the generated parts.

def collect_responses_and_parts_order(headers)
    responses, parts_order = super(headers)
    [responses, parts_order]
end

Each response is a MIME part with its boundary and the parts_order is the order in which the parts appear in the final e-mail. The MIME RFC 1341 says that the parts must be generated in the increasing order of preference, ie. text/html content-type part last, provided you want it to be the preferred format of your email.

We can find whether the generated e-mail contains a plain/text part and otherwise generate one.

html_part = responses.detect { |response| response[:content_type] == "text/html" }
text_part = responses.detect { |response| response[:content_type] == "text/plain" }
if html_part && ! text_part
  # generate a text/plain part
end

Generating the text part means stripping all HTML with links preserved. Nokogiri has a very convenient deep traverse iterator.

body_parts = []
Nokogiri::HTML(html_part[:body]).traverse do |node|
  if node.text? and ! (content = node.content ? node.content.strip : nil).blank?
    body_parts << content
  elsif node.name == "a" && (href = node.attr("href")) && href.match(/^https?:/)
    body_parts << href
  end
end

Once we have all the parts, assemble them, get rid of duplicate text and links, and re-insert into the email as a text/plain multipart block.

responses.insert 0, {
  content_type: "text/plain",
  body: body_parts.uniq.join("\n")
}
parts_order.insert 0, "text/plain"

This has been extracted into the actionmailer-text gem. Include ActionMailer::Text in your mailers.