In both my personal apps and Artsy Folio, I’m always after a deeper understanding of how people use the app. There’s three ways to do this: ask users, watch users and track usage. I’d like to talk about the third of these.
We’ve experimented with quite a lot of analytics tools for the Artsy website, and it seemed fitting to do the same for our mobile app. We wanted the freedom to change the analytics tool without having to change the code, and so ARAnalytics was born.
Let’s implement an Easter egg that requires curl and is HTTP-compliant.
We accept access tokens on our API endpoints. These can come from an access_token query string parameter.
curl https://api.artsy.net/api/v1/system/up?access_token=invalid -v
< HTTP/1.1 401 Unauthorized
< Content-Type: application/json
< Content-Length: 24
{ "error" : "Unauthorized" }
So far, so good. Now try this:
curl https://api.artsy.net/api/v1/system/up?access_token=10013 -v
< HTTP/1.1 401 Broadway
< Content-Type: application/json
< Content-Length: 76
{ "error" : "Inspiration from the Engineering team at http://artsy.github.com" }
What?! 401 Broadway? See, our office address is 401 Broadway, 10013, New York, NY. We just tried to add a more developer-friendly way to find us in the New York grid. And here’s the view from our 25th floor office - that’s SOHO right below us and the Empire State Building a bit North.
Photo by @zamiang.
Easter egg implementation follows.
At Artsy we make Artsy Folio. Folio is an awesome portfolio app that shows our gallery and museum partners their artworks in one place, allows them to easily get information about their inventory and to send works by email to their contacts.
Folio has to deal with large multi-gigabyte syncs in order to operate offline. That makes for a great user experience, but for the developer working on the sync, it’s not as pleasant. Combined with our use of Core Data, the app’s maturity, and dealing with data store migrations, things can get hairy. We needed a tool that could freeze and restore app data at will, obviating the need for constant syncing and resyncing.
That’s why I built chairs…
Chairs is a gem you can install via gem install chairs. It allows you to stash and replace your current iOS simulator application state. It will grab everything related to the app ( including the current NSUserDefaults) and store it in a named subfolder in your current working directory. No black magic, just lots of copying files.
The command line interface is based on git, so to bring in the current state you run chairs pull [name] and to replace the state you use chairs push [name]. The name is just a label so you can remember which version corresponds to that musical chair. You can get a list of these by doing chairs list, and delete them with chairs rm [name].
Besides the core functionality, chairs has a little bit of sugar to help you with related tasks. My personal favourite is chairs open; this will just open the folder of the most recently used app so you can go and have a snoop around. Amazing for making sure files are where they say they are or for opening your sqlite database in Base.
So gem install chairs or check out the README for some more information.
The Heroku’s Ugly Secret blog post went viral last week. I wrote in defense of Heroku, which has now responded with an official Routing Performance Update.
Random request queuing has been discussed in the past in Tim Watson’s post based on a response by Heroku’s Adam Wiggins. While the documentation may not have been accurate or even somewhat misleading, we, at Artsy, understood the strategy and the limitations of the routing mesh for quite sometime. Therefore, we have been making continuous efforts to improve our application’s performance and reduce the negative impact of random routing inside the routing mesh over the past few months.
One thing we didn’t do, was to measure the actual wait time inside a dyno. In restrospect, it seems obvious that we should have. In this post we’ll describe a middleware to do so. This is entirely based on the work of David Yeu, Jason R Clark and RG’s own Andrew Warner.
With this code in place, here’s a 12 hour graph of our website’s API performance. The dyno wait time for our application, in green, averaged 61.1ms for a total of 301ms average per request, which is 1/5th of the total request time. It’s certainly a lot, but we do spend a lot more time in our own code.
Note that the single peak on the right of the graph corresponds to a dyno auto-scale job. We double the number of dynos with early morning traffic, which causes new dynos to boot up and accumulate requests before they are “warm” enough to process requests at their normal rate.
An infinite scroll can be a beautiful and functional way to present feed data. You can see ours on the homepage of artsy.net. It works by fetching a few items from the API, then fetching some more items as the user scrolls down the feed. Each API call returns the items along with a “cursor”, which marks the position of the last item retrieved. Subsequent API calls include the cursor in the query string and the iteration resumes from there.
Why use a cursor and not standard pagination? Because inserting an item on top of the feed would shift the existing items down, causing the API to return a duplicate item on the page boundary. Removing an item from the top of the feed would pull the remaining items up, causing an item to be missed in the next request on the page boundary.
Today we’re open-sourcing a small gem called mongoid-scroll, which implements this cursor-like behavior for MongoDB using mongoid or moped. Here’s how it works.
We use MongoDB at Artsy as our primary data store via the Mongoid ODM. Eventually, we started noticing data corruption inside embedded objects at an alarming rate of 2-3 records a day. The number of occurrences increased rapidly with load as our user growth accelerated.
The root cause was not a HN-worthy sensational declaration about how MongoDB trashes data, but our lack of understanding of what can and cannot be concurrently written to the database, neatly hidden behind the object data mapping layer.
Heroku users frequently run the heroku command-line tool that ships with the Heroku Toolbelt. It has two very convenient features: it will remember API credentials and default to the “heroku” GIT remote to figure out the application to connect to. Neither of these features are available in the Heroku client API, so it’s not unusual to find developers invoke the Heroku CLI from Rake tasks and other automation scripts.
There’re several problems with using the Heroku CLI for automation:
heroku run is not the exit code from the process being run on Heroku. See #186.heroku run:detached requires an external heroku logs --tail process that will never finish.The heroku-commander gem wraps execution of the Heroku CLI to overcome these common limitations.
We use MongoDB as our primary store and have built a healthy amount of automation around various database instances and operational environments. For example, we backup databases to S3 using mongodump, mirror data between instances with mongorestore and often need to open a MongoDB shell with mongo to examine data at the lowest level.
Generating MongoDB command-lines is tedious and error-prone. Introducing a new gem called mongoid-shell to help with this. The library can generate command-lines for various MongoDB shell tools from your Mongoid configuration.
For example, connect to your production MongoDB instance from a db:production:shell Rake task.
namespace :db
namespace :production
task :shell
Mongoid.load! "mongoid.yml", :production
system Mongoid::Shell::Commands::Mongo.new.to_s
end
end
end
Last year, we have open-sourced and made extensive use of two Ruby libraries in our API: mongoid-cached-json and garner. Both transform the procedural nightmare of caching and JSON generation into a declarative and easily manageable DSL. It was worth the investment, since our service spends half of its time generating JSON and reading from and writing to Memcached.
Today we’ve released mongoid-cached-json 1.4 with two interesting performance improvements.
A few days ago we have started seeing the Heroku deployments of one of our applications randomly hang during bundle install. The problem worsened with time and we were not able to do a deployment for days.
$ git push -f git@heroku.com:application.git FETCH_HEAD:master
-----> Deleting 12 files matching .slugignore patterns.
-----> Ruby/Rails app detected
-----> Using Ruby version: ruby-1.9.3
-----> Installing dependencies using Bundler version 1.3.0.pre.5
Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin
Fetching gem metadata from http://rubygems.org/.......
Fetching gem metadata from http://rubygems.org/..
/app/slug-compiler/lib/utils.rb:66:in `block (2 levels) in spawn': command='/app/slug-compiler/lib/../../tmp/buildpacks/ruby/bin/compile /tmp/build_1p6071sni4hh1 /app/tmp/repo.git/.cache' exit_status=0 out='' at=timeout elapsed=900.1056394577026 (Utils::TimeoutError)
from /app/slug-compiler/lib/utils.rb:52:in `loop'
from /app/slug-compiler/lib/utils.rb:52:in `block in spawn'
from /app/slug-compiler/lib/utils.rb:47:in `popen'
from /app/slug-compiler/lib/utils.rb:47:in `spawn'
from /app/slug-compiler/lib/buildpack.rb:37:in `block in compile'
from /app/slug-compiler/lib/buildpack.rb:35:in `fork'
from /app/slug-compiler/lib/buildpack.rb:35:in `compile'
from /app/slug-compiler/lib/slug.rb:497:in `block in run_buildpack'
! Heroku push rejected, failed to compile Ruby/rails app
Seeing bundler hang on “Fetching gem metadata from http://rubygems.org/”, my immediate reaction was to blame the RubyGems Dependency API for its poor performance and attempt the recommended workaround of switching to http://bundler-api.herokuapp.com. That didn’t work.
I also tried to reproduce the issue on a local environment, including a (what I thought was) a completely clean machine at no avail. My bundle install would always succeed.
Finally, everything pointed at an infrastructure problem with Heroku itself, so I opened a ticket (#72648), tweeted endlessly to Heroku devs, pinged a contact at Heroku on Skype and generally annoyed people for 5 straight days. It was a frustrating problem and I was getting no useful help.
Fast forward, this turned out to be an issue in Bundler. Narrowing it down would have been relatively easy if I had known where to look.
I hope this post helps you with similar issues.