Verifying Replicate Signatures

Replicate is a platform for running AI models without having to worry about setting up infrastructure. A more simple definition would be “The Cloud but for AI”. There are a couple of advantages to doing this but the one that I think is pretty cool is the ability to run different models for different purposes with a sort of flexibility that is just crazy to think about.You could run one for transcribing tasks, another for embedding tasks while also getting the chatbot models e.g ChatGPT. There of course needs to be some setup for this, but it’s quite simple compared to running your own models especially if you’re just testing an idea (GPUs are quite expensive!).

There are a couple of ways you could configure your application to make use of the cloud-hosted models from Replicate and one such way is by the use of webhooks. Webhooks are regular HTTP post requests that are sent to a specific endpoint (webhook url) when a certain task is finished. Replicate uses webhooks to send prediction results to their users. Because a webhook is just a regular request sent to an endpoint, anyone can send that request (including malicious entitites) prompting the need to verify if indeed that request was from Replicate.

I’ll be explaining how you can verify those webhook requests, but as it turns out you will gain insights into how you can do the same for all sorts of scenarios.

It is important to note that Replicate uses an HMAC with SHA-256 to sign its webhooks.HMAC (Hash-based Message Authentication Code) is a type of message authentication code (MAC) that is acquired by executing a cryptographic hash function on the data that is to be authenticated and a secret shared key. The word ‘code’ seems confusing but I like to think of it as a ‘signature’.

To verify the signature or code that Replicate gave us, we are going to perform the same exact process that they performed on their end and then compare our computed MAC (signature) with theirs. It turns out that if by bad luck someone had intercepted the prediction and changed even a letter in the response body, the signature we produce on our end will be radically different from what we get from Replicate.

  defp verify_signature(body, signature_header, webhook_id, timestamp) do
    secret =
      :skeptic_bot
      |> Application.fetch_env!(:replicate)
      |> Keyword.fetch!(:webhook_secret)
      |> String.replace_prefix("whsec_", "")
      |> Base.decode64!()
    
    # Notice how they sign the content
    signed_content = "#{webhook_id}.#{timestamp}.#{body}"

    # compute the mac using this function 
    computed_mac = :crypto.mac(:hmac, :sha256, secret, signed_content)

    valid_signature =
      signature_header
      |> String.split(" ")
      |> Enum.any?(fn signature ->
        decoded_signature =
          signature
          |> String.split(",")
          |> Enum.at(1)
          |> Base.decode64!()
         
        # returns true if the 2 signatures are exactly the same
        :crypto.hash_equals(decoded_signature, computed_mac)
      end)

    if valid_signature do
      :ok
    else
      {:error, "Invalid signature"}
    end
  end

The secret is unique to each organization using replicate and looks something like :

whsec_9K6dddehHJuAjdf74dbn/u
# we only need this part 9K6dddehHJuAjdf74dbn/u decoded into a binary using *Base.decode64!()*.