The artsy.net website is a Backbone.js application that talks to a server-side RESTful Grape API sitting on top of a Rails app which serves minimal HTML markup. The latter includes such things as a page title, along with links to JavaScript and stylesheet packages. A page loads, scripts run, data is fetched from the API. The result is merged into a HAMLJS template and rendered client-side.

Building this kind of one-page apps allows for clean separation between the presentation and API layers. The downside is that it will slow page render times - fetching data after page load means waiting for an AJAX request to complete before displaying anything.

There’re many solutions to this problem, all involving some kind of server-side rendering. You could, for example, share views and JavaScript between client and server. This would be a major paradigm shift for a large application like ours and not something we could possibly maneuver in a short amount of time.

Without changing the entire architecture of the system, how can we bootstrap JSON data server-side and avoid the data roundtrip on every page load?

Read on →

In both my personal apps and Artsy Folio, I’m always after a deeper understanding of how people use the app. There’s three ways to do this: ask users, watch users and track usage. I’d like to talk about the third of these.

We’ve experimented with quite a lot of analytics tools for the Artsy website, and it seemed fitting to do the same for our mobile app. We wanted the freedom to change the analytics tool without having to change the code, and so ARAnalytics was born.

Read on →

Let’s implement an Easter egg that requires curl and is HTTP-compliant.

We accept access tokens on our API endpoints. These can come from an access_token query string parameter.

curl https://api.artsy.net/api/v1/system/up?access_token=invalid -v

< HTTP/1.1 401 Unauthorized
< Content-Type: application/json
< Content-Length: 24

{ "error" : "Unauthorized" }

So far, so good. Now try this:

curl https://api.artsy.net/api/v1/system/up?access_token=10013 -v

< HTTP/1.1 401 Broadway
< Content-Type: application/json
< Content-Length: 76

{ "error" : "Inspiration from the Engineering team at http://artsy.github.com" }

What?! 401 Broadway? See, our office address is 401 Broadway, 10013, New York, NY. We just tried to add a more developer-friendly way to find us in the New York grid. And here’s the view from our 25th floor office - that’s SOHO right below us and the Empire State Building a bit North.

Photo by @zamiang.

Easter egg implementation follows.

Read on →

At Artsy we make Artsy Folio. Folio is an awesome portfolio app that shows our gallery and museum partners their artworks in one place, allows them to easily get information about their inventory and to send works by email to their contacts.

Folio has to deal with large multi-gigabyte syncs in order to operate offline. That makes for a great user experience, but for the developer working on the sync, it’s not as pleasant. Combined with our use of Core Data, the app’s maturity, and dealing with data store migrations, things can get hairy. We needed a tool that could freeze and restore app data at will, obviating the need for constant syncing and resyncing.

That’s why I built chairs…

Chairs is a gem you can install via gem install chairs. It allows you to stash and replace your current iOS simulator application state. It will grab everything related to the app ( including the current NSUserDefaults) and store it in a named subfolder in your current working directory. No black magic, just lots of copying files.

The command line interface is based on git, so to bring in the current state you run chairs pull [name] and to replace the state you use chairs push [name]. The name is just a label so you can remember which version corresponds to that musical chair. You can get a list of these by doing chairs list, and delete them with chairs rm [name].

Besides the core functionality, chairs has a little bit of sugar to help you with related tasks. My personal favourite is chairs open; this will just open the folder of the most recently used app so you can go and have a snoop around. Amazing for making sure files are where they say they are or for opening your sqlite database in Base.

So gem install chairs or check out the README for some more information.

The Heroku’s Ugly Secret blog post went viral last week. I wrote in defense of Heroku, which has now responded with an official Routing Performance Update.

Random request queuing has been discussed in the past in Tim Watson’s post based on a response by Heroku’s Adam Wiggins. While the documentation may not have been accurate or even somewhat misleading, we, at Artsy, understood the strategy and the limitations of the routing mesh for quite sometime. Therefore, we have been making continuous efforts to improve our application’s performance and reduce the negative impact of random routing inside the routing mesh over the past few months.

One thing we didn’t do, was to measure the actual wait time inside a dyno. In restrospect, it seems obvious that we should have. In this post we’ll describe a middleware to do so. This is entirely based on the work of David Yeu, Jason R Clark and RG’s own Andrew Warner.

With this code in place, here’s a 12 hour graph of our website’s API performance. The dyno wait time for our application, in green, averaged 61.1ms for a total of 301ms average per request, which is 1/5th of the total request time. It’s certainly a lot, but we do spend a lot more time in our own code.

Note that the single peak on the right of the graph corresponds to a dyno auto-scale job. We double the number of dynos with early morning traffic, which causes new dynos to boot up and accumulate requests before they are “warm” enough to process requests at their normal rate.

Read on →

An infinite scroll can be a beautiful and functional way to present feed data. You can see ours on the homepage of artsy.net. It works by fetching a few items from the API, then fetching some more items as the user scrolls down the feed. Each API call returns the items along with a “cursor”, which marks the position of the last item retrieved. Subsequent API calls include the cursor in the query string and the iteration resumes from there.

Why use a cursor and not standard pagination? Because inserting an item on top of the feed would shift the existing items down, causing the API to return a duplicate item on the page boundary. Removing an item from the top of the feed would pull the remaining items up, causing an item to be missed in the next request on the page boundary.

Today we’re open-sourcing a small gem called mongoid-scroll, which implements this cursor-like behavior for MongoDB using mongoid or moped. Here’s how it works.

Read on →

We use MongoDB at Artsy as our primary data store via the Mongoid ODM. Eventually, we started noticing data corruption inside embedded objects at an alarming rate of 2-3 records a day. The number of occurrences increased rapidly with load as our user growth accelerated.

The root cause was not a HN-worthy sensational declaration about how MongoDB trashes data, but our lack of understanding of what can and cannot be concurrently written to the database, neatly hidden behind the object data mapping layer.

Read on →

Heroku users frequently run the heroku command-line tool that ships with the Heroku Toolbelt. It has two very convenient features: it will remember API credentials and default to the “heroku” GIT remote to figure out the application to connect to. Neither of these features are available in the Heroku client API, so it’s not unusual to find developers invoke the Heroku CLI from Rake tasks and other automation scripts.

There’re several problems with using the Heroku CLI for automation:

1. The exit code from heroku run is not the exit code from the process being run on Heroku. See #186.
2. Gathering console output from heroku run:detached requires an external heroku logs --tail process that will never finish.

The heroku-commander gem wraps execution of the Heroku CLI to overcome these common limitations.

Read on →

We use MongoDB as our primary store and have built a healthy amount of automation around various database instances and operational environments. For example, we backup databases to S3 using mongodump, mirror data between instances with mongorestore and often need to open a MongoDB shell with mongo to examine data at the lowest level.

Generating MongoDB command-lines is tedious and error-prone. Introducing a new gem called mongoid-shell to help with this. The library can generate command-lines for various MongoDB shell tools from your Mongoid configuration.

For example, connect to your production MongoDB instance from a db:production:shell Rake task.

namespace :db
  namespace :production
    task :shell
      Mongoid.load! "mongoid.yml", :production
      system Mongoid::Shell::Commands::Mongo.new.to_s
    end
  end
end

Read on →

Last year, we have open-sourced and made extensive use of two Ruby libraries in our API: mongoid-cached-json and garner. Both transform the procedural nightmare of caching and JSON generation into a declarative and easily manageable DSL. It was worth the investment, since our service spends half of its time generating JSON and reading from and writing to Memcached.

Today we’ve released mongoid-cached-json 1.4 with two interesting performance improvements.

Read on →