Although some find infinite scroll to be a contentious topic at Artsy we’ve found it to be a useful element in many portions of our site such as filtering. However, we’ve run into a common and painful usability issue with infinite scroll. That is clicking on an item redirects to the next page, losing your scroll position, and losing your place when going back. To solve this we have come up with a clever little solution using an iframe.
When I joined Artsy, dB pitched me this idea: Open source as default.
I took this to heart. I genuinely believe the idea behind the philosophy. It’s cool that our real product isn’t our implementations on the web or native but the data which powers it - the Art Genome Project. Similarly, I spend a bunch of time on open sourcing solid abstractions from our apps, always taking the opinion if something is used in more than one place, it should be open sourced.
This week I pushed some libraries that were a bit different, read on to find out why.
I’m the kind of guy who thinks better tooling means better outcomes. But when good tooling isn’t available, it’s time to build it yourself. It’s this attitude that lead to my work on CocoaDocs.org, and then to CocoaPods.org & its documentation.
We’ve been trying to apply this to testing, and in order to pull this off I’ve had to extend Xcode to show off the results of failing tests in a more visual way. To that end, I’ve extended Xcode to show the results of failing view tests in a more visual way by building Snapshots for Xcode. Let’s go through the process of building an Xcode plugin so you can do this too. Screw stability.
The first time I released a patch release for Artsy Folio it crashed instantly, on every install. Turns out I didn’t understand Core Data migrations, now a few years on I grok it better but I’ve still lived with the memories of that dark dark day. Because of this I’ve had an informal rule of testing migrations with all the old build of Folio using chairs the day before submitting to the app store.
This time round, I’ve made vast changes to the Core Data models but skipped the manual work. Here’s how:
Whatever you have against monolithic architectures, at least they’re easy to test. And when those tests succeed, you can be reasonably confident the live app will work the same way.
Artsy began as one such monolithic app, but we’ve been refactoring into an ecosystem of related APIs and sites. Today, when you search for “cultural commentary” or visit Robert Longo on artsy.net, the page is rendered by a web app, sources data from an API, retrieves recommendations from a separate service, tracks trends in another, and records analytics in yet another.
This was a boost for developer productivity and scaling, but eviscerated the value of our tests. We repeatedly encountered bugs that were failings of the interaction between codebases rather than failings of individual ones. Test libraries and tools typically concern themselves with one isolated app. When you have services that consume services that consume services, those isolated tests (with their stubs of everything else) don’t necessarily reflect production’s reality.
So how should we develop our small, focused apps (or service-oriented architecture, or microservices…) with confidence? We set out to build a dedicated acceptance test suite that would run tests across multiple services, configuring and integrating them in a way that closely matches the production environment.
We recently launched a new personalized email here at Artsy that features content that a given user might find interesting. The goal of this post is to describe how we built a backend system that efficiently generates these e-mails for all our users. I’ll talk about the first, naive implementation that had performance problems right away, and how the second implementation (currently in production) solved those issues, and whose behavior at scale is well-defined and understood. I won’t go into the details of the design and layout of the mail itself and how we render the content - there are several earlier blog posts that deal with those: Presenters and Memoization, Pinterest-style Layouts and Email Layouts and Responsiveness.
Artsy ran several successful auctions over the past few months. The first, TWO x TWO, raised hundreds of thousands of dollars for amfAR (the AIDS Research foundation), and the Dallas Museum of Art. It was followed by Independent Curators International, at which Artsy launched on-site auction projection screens, which displayed competing bids coming in online from places around the world, like Oslo and Santa Monica, in realtime. Users could place bids on the website, via the iPhone app or with one of the Artsy representatives in the room carrying an iPad. All the auction lots sold, and Artsy helped ICI to raise 50% more than its target revenue goal. Other, recent Artsy auctions include Public Art Fund and the Brooklyn Artists Ball, benefitting the Brooklyn Museum.
The domain of auctions is a fascinating one, and includes everything from buying items on eBay to trading livestock and selling investment products on the stock exchange. For those interested in the large spectrum of auctions I highly recommend Auctions and bidding: A guide for computer scientists by Simon Parsons (CUNY), Juan A. Rodriguez-Aguilar (CSIC) and Mark Klein (MIT).
At Artsy we implemented a classic English auction with, so called, “book bids”. I spent a fair amount of time visiting engineering teams that have built internet auctions, most of which were transactional systems where taking a position on an item involved starting a transaction, running an auction round and committing the changes. In contrast, we chose to deliver a simpler, eventually consistent system on top of MongoDB, in which all data is immutable and where some level of serialization occurs within a single background process.
In this post we’ll go over some data modeling and examine the auction engine implementation details.
Refactoring usually describes chages to code. Specifically, small changes that bring a codebase closer to the desired state. By making these changes incrementally and without modifying the end-to-end behavior, we avoid risk and the intermediate broken states that usually plague large-scale changes. But refactoring need not be limited to code. It’s also an effective way to make infrastructure improvements.
Take the most common–and simplest–example: database schema changes. Environments that demand constant uptime have long had to chunk schema changes into steps that allow for a graceful transition. In the simple case of replacing a column, this might look like:
- Add the new column
- Redirect code references there from the old column
- Migrate data as necessary, and finally
- Remove the old column
The same sequencing applies to making larger infrastructure changes gracefully. Some recent examples from our own experience:
When dealing with rendering data for an email, one frequently has to make many database calls to assemble the required data. This can be slow, and depending on how you structure the code that is assembling the data vs rendering the data in a template, it’s very easy to be making repeated calls, which can significantly slow down your process. Additionally, whether you are using Haml, Mustache, Jade, or any other templating language, embedding too much logic in the template can making things hard to maintain (especially if some logic lives in the template and some elsewhere in your domain code). Of course some logic in the template (a conditional: should I render this section?, or loops: render this hash of data) is necessary, but I like to keep as much out of there as possible. It’s easier to optimize, debug and maintain that logic elsewhere, and also writing complex logic in Ruby is much more fun than in a templating language!
In this article I’ll present what I’ve been doing to keep my templates relatively logic-free, and how I make sure I don’t repeat any heavy database calls in assembling my data.
Something we’d like to do is email our users some suggested artworks as part of their personalized emails. The layout of those suggestions should look something like our newly re-designed Browse page, with a ‘salon’ style layout. Here’s some simple Ruby code that can group artworks into columns for you, that can then be directly rendered in an email (via Haml, Mustache, Jade, or your templating language of choice.)