Everything You Ever Wanted To Know About Authentication at Artsy (But Didn't Know How To Ask)

By Mykola Bilokonsky

Hi! Let's talk a little bit about user authentication. We'll discuss it through the context of authentication within Artsy's ecosystem of applications, but ideally the same concepts will translate into other systems as well. The goal here is to build an understanding of (1) what exactly authentication is, (2) how it works, and (3) how to use artsy's specific infrastructure to delegate authentication for a new application to the existing ecosystem.

There are two primary authentication flows that we use at Artsy. We support user authentication through OAuth to allow users to log into our applications by delegating authentication to Gravity, our primarily application API. Alternately, we support app authentication for those cases where an application will only be called by other applications. We don't care about user authentication in this context, there's no need to redirect to a login screen etc - we just need to establish permissions between services. In both cases you'll be working with JSON Web Tokens (JWTs), and the difference is how the token you're looking at is generated.

User authentication happens at login - when the user provides their credentials to our server, our server confirms that they are who they claim to be and then generates a cryptographically signed token that encodes a few facts about that user.

App authentication, by contrast, all gets done in advance. We create the token manually, and share it with whatever application we want to grant access to.

In this document we'll first develop an understanding of what OAuth is and how it works. Then we'll examine the tokens we're using to get a better sense of what kind of information we have to work with. Finally, we'll go into how to set up authentication for users and for applications, building on the knowledge we've established.

OAuth

OAuth is a paradigm that allows one service to delegate authentication to another one. You've probably used this before, even if you don't realize it. Whenever you use your Twitter account to log in to Medium, or your Facebook account to log in to Tinder, or whatever, you're relying on delegated authentication. Medium doesn't have to know much about who you are as long as they can trust Twitter to tell them, etc.

We use a similar approach here at Artsy. We've got dozens of applications, some outward-facing, some internal only. If every single application had to handle its own authentication then we'd have user objects stored all over the place and we'd be constantly trying to figure out if User 123 on service A is the same person as User 321 on service B, etc, right? So instead we delegate authentication to a single core service which we call Gravity. Gravity is Artsy's core API, and therefore is the canonical source of truth for user information.

So at a high level, OAuth works like this:

  1. An HTTP request is sent to your server.
  2. Your server checks for the presence of an Authorization HTTP header.
  3. If it cannot find one, your server redirects the request to login through Gravity.
  4. If the user logs in through Gravity, Gravity gathers a bunch of information about that user, cryptographically signs it, and gives it to the user to give your server by adding the token string to the user's Authorization header.
  5. The user is then redirected back to your server, this time with the header that represents your access token.
  6. Our access tokens are JWTs (they don't have to be - OAuth works with a simpler payload if you want - but by using JWTs we are able to include additional information in the payload.) so we can verify the key by decrypting the signature in the JWT using a key that your server shares with Gravity.
  7. If that private key works and the header is decrypted, you can trust the information inside the JWT

There's a slight difference in authenticating a user and authenticating a trusted application, as discussed above. If our incoming request was generated by another Artsy service then we don't redirect anyone to gravity for login - rather, we manually create the token in advance, and store it. Once this is set up, the above process exits after step 2. But for now, let's continue looking at how user authentication works, because it's important to understand the whole system. We'll circle back to app authentication at the end.

So!

Think about it this way: when the user is redirected to Gravity for login, Gravity gathers up a bunch of information about who this user is. What's their user ID? What roles has their account been assigned? It lumps these together to create a JWT, which is a short statement of identity verification. It sticks this JWT into the user's headers - conceptually, think of this as a legal document or a letter of introduction - "The bearer of this document is User 123. Sincerely, Gravity."

In fact, with OAuth the full header looks like this: Authorization: Bearer <Signed JWT>

So, what's in that JWT? That's a great question! First, let's talk briefly about how encryption works just to make sure we're all on the same page about how we sign JWTs. Then we'll dig into our JWTs and how to use them to provision access to parts of our applications.

Shared Secret

This document is not a guide to encryption, but a basic understanding of private key encryption is helpful to understand how this stuff works. Let's say you have some message MSG and you want to encrypt it. You can use some random secret key, let's call it ABC, to encrypt your message. There are a number of ways to do this, let's not worry about the implementation - assume that you have access to a function called encrypt(message, secret). So if we want to encrypt our message, we can say encrypt(MSG, ABC) and we'll get back a string of meaningless characters, let's call it ENC. The key here is that there also exists a function decrypt(encrypted, secret), such that as long as the secret is the same will convert your encrypted string back into its original state. decrypt(ENC, ABC) === MSG

This is basically how security works on much of the internet, with various twists that optimize for different things depending on what exactly is being secured and from whom. If you want to send a message out into the world but you only want a specific person to be able to read it, you can encrypt it with your secret and then share the secret with only those people that you want to allow to read your message.

This is what's going on with our Authorization token. Gravity generates a JWT which encodes some facts about the user or application under discussion. It then signs that JWT using a secret key, and attaches the signed token to the HTTP request using the Authorization header. You've got that same secret key in your application. This allows your app to verify the signature of the Authorization token and get at the JWT that it contains.

Note: the JWT itself is not encrypted. Anyone can copy the JWT string and drop it into a decoder. Do not put anything sensitive into your JWT. Just put basic facts about the user. The encryption is just used to sign it - you create an encrypted signature that can only be read by someone else who has the same private key. This allows them to verify that the token was created by a trusted source - but the token payload is always visible.

Let's dig a little bit deeper into what exactly JWTs are and how they work.

JWT Details

A JWT (or JSON Web Token) has three components. Each is a base-64 encoded string containing some information. The first part is the header - when you decode this back into a standard string you'll see that it just describes the mechanisms whereby the token was generated, which allows a reader to decode it. The second is a small JSON object that stores information about some subject. It has a small set of pre-defined fields, some of which we use, as well as support for arbitrary custom fields. The third part is the signature - anyone who has the secret key can decode the signature, which allows them to know for sure that the JWT was created by someone else who had that same secret key.

Parts 1 and 3 are infrastructure - they tell a reader how to decode and verify the authenticity of a token. Part 2 is the actual payload, where relevant information gets stored. We don't really have to worry too much about 1 and 3 - those parts get handled for us through the libraries we're using. But let's look a little closer at part 2.

When Gravity generates a JWT for a user it specifies four fields 🔒:

  1. sub is part of the JWT spec and is short for subject - this encodes the subject of the token, in other words the user on whose behalf it's been generated.
  2. salt_hash is a unique identifier for the given user with the given password within the gravity context.
  3. roles is a string representing a comma-delimited list of the roles that the user has within the Artsy authentication structure.
  4. partner_ids holds references to any Partner objects to which the user has access.

When Gravity generates a JWT for an application, on the other hand, it only specifies roles 🔒. There is no subject - the fact that it was encoded using a secret relevant to our application is sufficient grounds to know that it's about our application. Does that make sense? We don't need to specify a subject because if it was generated for any other application we'd never be able to decrypt it. So we just need to know what roles this application has been granted.

So when we decode the JWT in our application we get either a token that represents a trusted user or a token that represents a trusted application. We need to know what to do with this JWT next, but before we do that we need to set up our shared secrets so that our new app can verify the authenticity of decoded tokens.

Again, to reiterate: a JWT is not encrypted. It's publicly readable. Do not put anything too sensitive into your JWTs.

If you'd like to learn more about how Artsy thinks about and uses JWTs you can read this blog post

Generating Secret Keys

So now we understand that our application is going to receive cryptographically signed JWTs from Gravity, and we know that in theory as long as we have the same secret string that Gravity is using we should be able to verify that these JWTs are trusted. But how do we share that secret?

Through brute force banality: we copy the secret from Gravity into an environment variable in our application. First let's talk about how to generate the secret string on the gravity side, then we'll talk about how to set up the application to use it. Note that we can allow Users to access our application via the first step here, but if we want to allow access to trusted applications we need to use a second step, below.

Create a ClientApplication

The first thing we need to do is gain access to a Gravity console so that we can create a ClientApplication instance. Gravity keeps track of every application for which it handles authentication, and it models them as instances of the ClientApplication model. So if we're creating something called fooapp, we would have to do something like this (this part of this essay is specific to Artsy's implementation details):

1
2
3
  $ hokusai staging run --tty 'rails c'
  gravity:staging> ca = ClientApplication.create!(name: 'fooapp', access_granted: true, roles: ['artsy'], redirect_urls: ['http://localhost:5000','https://fooapp-staging.artsy.net'])
  => #<ClientApplication _id: 5bcf482dc95e4c0007bffeb7, created_at: 2018-10-23 16:11:25 UTC, updated_at: 2018-10-23 16:11:25 UTC, name: "fooapp", api_version: nil, app_id: "02a971b9401a317bf815", app_secret: "8b7d9b597428254f9305544b8cd4d710", internal_secret: "a111805fdfe2bb63b077ff65b71ebcd9", access_granted: true, redirect_urls: ["https://mydomain.artsy.net"], published_artworks_access_granted: false, roles: ["artsy"], user_id: nil>

Real quick, we passed a few important arguments into that create! method - we named our ClientApplication, we specified that we want to enable access to it, we gave it a role and we set a few redirect_urls - these are OAuth specific arguments that allow Gravity to redirect the user back to our app after they've authenticated (step 5 above).

(note we allow redirects to two urls, our staging instance and a local instance. A common workflow is to do local dev while pointing at the staging gravity instance - by allowing redirects to localhost in our ClientApplication we're allowing Gravity to support developers running their application locally in development.)

Once this client application has been created it's going to be used to generate a secret. There are two fields in the ClientApplication that we're going to need - app_id and app_secret. These combine to create the secret key that Gravity is going to use to sign our JWT, so we need to make sure that our app has access to these two values.

Go back to your app, open _config.rb and add the following:

1
2
3
  config.app_id=ENV['APP_ID']
  config.app_secret=ENV['APP_SECRET']
  config.gravity_url=ENV['GRAVITY_URL']

Then in your .env file (or however you're managing environment variables) go ahead and set APP_ID=02a971b9401a317bf815, APP_SECRET=8b7d9b597428254f9305544b8cd4d710 and GRAVITY_URL=https://staging-api.artsy.net. These values can now be accessed from yourapp.config.app_id, yourapp.config.app_secret and yourapp.config.gravity_url - we're going to need these values to configure ArtsyAuth in our new application.

This is enough, then, to enable user authentication! If we want to create application authentication we need to take one more step.

Create a JWT for Application Trust

For users, JWTs are generated when they log in. But if we are trying to send HTTP requests from barapp to fooapp we've established that no login is going to take place - so how does barapp get the JWT it needs to authenticate against our fooapp? We're going to create it manually and store it in barapp's environment. Let's go back to our Gravity console session - make sure ca is still defined. If you closed the session then before running the next code you should get a reference to that client application. ca = ClientApplication.where(name: 'fooapp').first should do it. Then:

1
2
  gravity:staging> expires_in = 10.years.from_now
  gravity:staging> token = ApplicationTrust.create_for_token_authentication(ca, expires_in: expires_in)

This should generate a long string that it dumps to the console. Congratulations - that string is your JWT. Any http request with the header Authorization: Bearer <JWT> will now be accepted by fooapp.

So to use this JWT, we copy the string into barapp's environment - often with a second environment variable for the specific host we want to connect to. So in barapp you'd have environment variables like FOO_JWT=<string> and FOO_HOST=http://fooapp.artsy.net or FOO_HOST=http://fooapp-staging.artsy.net or whatever. (We store the host as an environment variable rather than having it in the code precisely so that we can target specific instances of the application depending on the desired context).

Somewhere in barapp we'll have a service that sends HTTP requests to fooapp - we just have to make sure that those requests have the authorization header with the correct signed token, and bam! We're authenticated!

So now it's time to put it all together - we know how to generate tokens, we know how to attach them to requests as authorization headers, and we know how to decrypt them locally in fooapp. So now what? What do we do with those tokens once we have them? That's where a useful library called ArtsyAuth comes in, at least if you're writing a Ruby on Rails application here at Artsy.

ArtsyAuth

We've seen now how OAuth uses JWTs and private-key encryption to allow one service to vouch for a user to another service. But that's not quite enough - in our application, it's nice that we now have user data that we trust, but what do we do with it? How do we configure our application to allow trusted users? Come to think of it, where do we actually redirect unauthenticated users back to gravity to log in? OAuth is historically complex to set up correctly, but fortunately ArtsyAuth is here to help. It abstracts out most of the OAuth complexity for us, allowing us to focus on writing logic for what to do based on the JWT's actual values.

A simple ruby gem that handles all of this for you. Just bundle add artsy-auth and then create config/initializers/artsy_auth.rb. It should look something like this:

1
2
3
4
5
  ArtsyAuth.configure do |config|
    config.artsy_api_url = Osmosis.config.gravity_url # required
    config.application_id = Osmosis.config.app_id # required
    config.application_secret = Osmosis.config.app_secret # required
  end

Then, you include it in your application controller --

1
2
3
4
5
6
7
8
9
10
11
12
13
class ApplicationController < ActionController::Base
  # Prevent CSRF attacks by raising an exception.
  protect_from_forgery with: :exception

  # This will make sure calls to this controller have proper session data
  # if they don't it will redirect them to oauth url and once authenticated
  # on successful authentication we'll call authorized_artsy_token?
  include ArtsyAuth::Authenticated

  def authorized_artsy_token?(token)
    ArtsyAuthHelper.admin? token
  end
end

At this point, all you have to do is implement authorized_artsy_token? in the controller for the view you're trying to protect. In the example above, we're delegating the logic to the ArtsyAuthHelper's .admin? method (which checks for the 'admin' role in the JWT's roles field), but this can do whatever we want. We get a token and we return a boolean. If we turn true, then the application can go about its way secure in the knowledge that the person using it is allowed to do so. If we return false we either get a 403 or a redirection to Gravity to assert our identity.

You can see how rather than checking for the admin role we could, say, check for a specific partner ID - or we could restrict access to a specific subset of gravity users by manually checking the sub field against allowed ID's. Whatever. The point is, we've set all of these things up to allow Gravity to tell us details about the entity making an incoming request. It's still up to us to evaluate those details and provision access accordingly.

Wrapping Up

Ideally this document has explained how authentication works at Artsy. We've talked about how Gravity encodes user- and application-specific information into JWTs which it cryptographically signs. These JWTs get added to the Authorization header of requests that get sent to our protected service - for users, the JWTs are generated when they log in, and for trusted applications the JWTs are generated manually in advance, but either way requests end up with an Authorization token whose signature can only be verified by an application with access to the secrets used to sign it. We create a ClientApplication instance in Gravity specifically to get these secrets. Once we copy those secrets to our new application, our new application can decrypt the value of the Authorization header on any incoming requests bearing a token generated in Gravity for our application. We then add some logic to our controller to determine which parts of the JWT we want to treat as salient for the purposes of access control.