Using public randomness from drand

In this blogpost we will run through some basic tenets of public randomness and then a brief tutorial on how to consume randomness from drand and use it in your applications.

Why public randomness?

Most developers are familiar with private randomness: we use randomness to generate keypairs, randomise backoff timings in distributed systems, choose stats for games, and a plethora of other things. This makes sense - if the randomness used to generate our keypairs was public, bad actors could intercept all our communications or steal our bitcoin! Some other applications use private randomness right now, but might be better served using public randomness: when I use an online casino or lottery, the casino generates a lot of random numbers to, for example, order the cards dealt or determine a winner. We must have faith that online casinos are using effective random number generators, but history has shown this is not always the case... (opens new window) If they were to commit to a random number generated by a third party (or third parties), or were using some kind of verifiable randomness, this could improve the fairness of such sites and increase the trust their users have in them!

For cases such as random distribution of welfare (e.g. universal basic income (opens new window) experiments), the government could use a third party such as an NGO to draw their random numbers but this still constitutes a weak link: a single third party could be unduly influenced (opens new window) to bias the randomness.

Enter drand

drand is built upon a threshold network (opens new window). Instead of relying on a single third party, multiple third parties work together to generate a random number. It exploits the fact that a hashed signature has all the properties of randomness as long as nobody holds the secret key. Due to the way drand distributes keys (opens new window), no party ever has the whole secret key, and as such no one can ever predict the signature generated - woohoo, randomness! These properties are the reason that projects such as Social Income (opens new window) are building on top of drand for their randomness!

Using drand

Presently, the drand network generates a random hex string every thirty seconds. It can be verified by using the public key of the network (the one corresponding to the secret key that nobody ever sees!), so you can even receive a piece of randomness from an untrusted party and verify that it's truly been created by the drand network. This is a very powerful property which simplifies your trust assumptions: you can receive randomness without having to trust the node that sent you the message because you can verify it for yourself.

In this tutorial we're going to fetch randomness using the official javascript client (opens new window), but you could also use the official go client (opens new window), one of the multiple unofficial rust clients (opens new window) or fetch it via curl/libp2p/some other channel and verify the BLS signatures yourself.

First off, make sure you have a relatively recent version of node (17+) and npm installed (instructions to do so can be found here: https://docs.npmjs.com/downloading-and-installing-node-js-and-npm (opens new window)).

Then create a new folder for your project by running mkdir drand-client-tutorial (or whatever name you wish to use), cd into it and run npm init, follow the prompts and finally install the latest drand-client package by running npm install drand-client. As the drand client is bundled as an esm module, you will need to add "type": "module" in your package.json to use it (lest we fall down the babel rabbit hole!).

From there, create a new file at the root of the project called index.js and open it in your editor of choice.

The most basic primitive is a Chain - it represents a single network instance hosted by a node. Networks can have different parameters such as frequency (how often it creates randomness), algorithm (what type of cryptography underpins it and how the beacons are related) and participants (the nodes that are actually working together to create the shared signature that constitutes randomness), but for our purposes the default network is enough!

We're going to connect to http API of the drand team’s nodes (https://api.drand.sh (opens new window)), but you could also use Cloudflare's API endpoint (https://drand.cloudflare.com/ (opens new window)), or StorSwift’s relay (https://api.drand.secureweb3.com:6875 (opens new window)), but beacons are also available through libp2p (/dnsaddr/api.drand.sh) or any CDN that hosts drand beacons! Provenance of the beacons is not important, since they are entirely verifiable.

Since we're using HTTPs, let's create a chain using the HTTPs implementation:

import { HttpChain } from "drand-client"  
const chain = new HttpChain("https://api.drand.sh")

From here we can see interesting metadata from the chain such as the public key by calling the info endpoint:

import { HttpChain, fetchBeacon } from "drand-client"  

const chain = new HttpChain("https://api.drand.sh")  
chain.info().then(info => console.log(info))

To run this, run node --experimental-fetch index.js in your console (node versions greater than 18 can skip the --experimental-fetch flag!)

It's suggested to verify the public key out of band if you're planning on using a single network longer term! In order to verify the beacon, each request actually uses the ChainInfo object to verify the payload against the public key, so a helper class HttpCachingChain is provided that caches the ChainInfo after the first call so you don't need to make an additional network request every time. Let's combine the HttpCachingChain with an HttpChainClient to request the first round from mainnet:

import { HttpCachingChain, HttpChainClient, fetchBeacon } from "drand-client"  

const chain = new HttpCachingChain("https://api.drand.sh")  
const client = new HttpChainClient(chain)

fetchBeacon(client, 1).then(beacon => console.log(beacon))

If you're sticking to a set chain, you could even consider creating a custom chain implementation that uses a hardcoded set of info, as it shouldn't change (other than the groupHash which changes every time a key refresh or resharing is done, but it isn't used for verification).

Okay, we've managed to retrieve a beacon by its round number - this could be useful if we want to commit to some specific future round (e.g. for a lottery), but plenty of uses cases want to retrieve the latest randomness instead. Out of the box, the drand client provides a few ways of doing that. Firstly, we could fetch the latest beacon for a given time and pass it the current time:

import { HttpCachingChain, HttpChainClient, fetchBeaconByTime } from "drand-client"  

const chain = new HttpCachingChain("https://api.drand.sh")  
const client = new HttpChainClient(chain)
fetchBeaconByTime(client, Date.now()).then(beacon => console.log(beacon))

This will nicely work out the round number last emitted on or before the time we provide (and will also fetch historical beacons for us). If we're creating a real-time game however, we'd prefer to get the beacons as soon as possible, without having to do the plumbing required to know exactly which time to request them at. Luckily the client provides an async iterator to do just that: (note if you're on some older versions of node, you might need to do some wizardry to get the async/await working as expected)

import { HttpCachingChain, HttpChainClient, watch } from "drand-client"

const chain = new HttpCachingChain("https://api.drand.sh")
const client = new HttpChainClient(chain)
const beacons = watch(client, new AbortController())

for await (const beacon of beacons) {
  console.log(beacon)
}

This will listen for all the new beacons being emitted and retrieve them at the relevant time. You will see the most recent beacon immediately and a pause before each subsequent beacon is emitted. watch takes an AbortController object so that we can manage cancellation of a request whenever we don't want to receive any further beacons. For more info on AbortControllers, you can read the docs from mozilla (opens new window). You can end the script with ctrl+c!

Great - now we can do logic such as repainting our roulette wheel or announcing our winner as soon as the latest randomness is in... but that's not all!

Using randomness effectively

If we only ever needed a 64 byte random value in our applications, we'd be finished now - just use the randomness from the beacon and be on our merry way. Unfortunately, real life applications are not so well aligned with drand's output! A common (and often mistaken!) way to reduce randomness into something we can use is by using modular arithmetic. Consider the following example: we want to use drand to flip a coin for us. A coinflip can have two outcomes: heads or tails. That maps conveniently to true or false for a programming paradigm, and in fact we can do that quite handily using drand randomness:

import { HttpCachingChain, HttpChainClient, fetchBeaconByTime } from "drand-client"  

const chain = new HttpCachingChain("https://api.drand.sh") 
const client = new HttpChainClient(chain)
fetchBeaconByTime(client, Date.now()).then(beacon => {   

	const flip = BigInt("0x" + beacon.randomness) % BigInt(2)

	if (flip === BigInt(0)) {      
		console.log(beacon.round+": HEADS!")
	} else {     
		console.log(beacon.round+": TAILS!")   
	} 
})

A small note: we have to use BigInt here, because javascript numbers only support 53 bit integers safely! Additionally, because drand mainnet emits randomness every 30 seconds, you will see the same output for multiple runs until the newest round is emitted.

As our 64 byte random hex string from drand is a multiple of 2, our coinflip will produce a uniform distribution of heads and tails (given enough flips) and works as expected. We could even take just one or two byte of output and still have an unbiased result. However this is not always a given: let's suppose instead of flipping a coin, we wanted to select a random winner from a lottery of 7 participants:

// DON'T DO THIS 
import { HttpCachingChain, HttpChainClient, fetchBeaconByTime } from "drand-client"  

const participants = ["alice", "bob", "carol", "dave", "edward", "fiona", "georgina"] 
const chain = new HttpCachingChain("https://api.drand.sh") 
const client = new HttpChainClient(chain)
fetchBeaconByTime(client, Date.now()).then(beacon => {   
  // there are 7 participants, we take 1 byte of output (!! bad !!)
  const randomNumber = BigInt("0x" + beacon.randomness.slice(0, 2)) 
  const winnerIndex = randomNumber % BigInt(participants.length)
  console.log(`the winner is ${participants[winnerIndex]}`) 
})

On the face of it, this might look fine, but in fact it's biased! As 7 does not factor evenly into 255, some output numbers are more likely than others to occur! For a deep dive into this, I suggest reading this in-depth blog post (opens new window) about it. Truth be told, if your desired range of random numbers is less than 2^64, using modulo will have only a negligible bias on the output - but let's continue for the sake of correctness!

Okay, so how should we reduce our drand randomness into something useful?

First off, the easy way - we can use the drand randomness to seed an existing PRNG implementation (opens new window). Most core implementations of PRNGs in common languages have already thought about this for us, so we can simply pass our drand random value to them and let them handle all the hairy parts. In Javascript though, the builtin PRNG Math.random doesn't provide a method for seeding it and is not a "secure" PRNG! Instead we must use what is called rejection sampling.

Rejection sampling is a method that is essentially using the modulo operator as before, but rejecting a number of the outcomes and choosing a new random number to maintain an equal distribution.

A small note that in some scenarios, rejection sampling can open software up to timing attacks (opens new window). It's unlikely that you, the dear reader, are writing such sensitive protocols without already being an expert in cryptography however, so we won't take it into account at all in the following code.

For simplicity, let's consider our coinflip again, but assume we have a black box function that returns a value from 0-2 inclusive. If we were to take a modified version of our code from before, we would end up with twice the number of tails than heads:

const zeroToTwoInc = fetchNumberFromZeroToTwoInclusive() 
const flip = zeroToTwoInc % 2  
if (flip === 0) {
	console.log("HEADS!") 
 else {   
	console.log("TAILS!") 
}

which is biased since both the value 1 and 2 would produce "TAILS!", but only the value 0 would produce "HEADS!". Adding rejection sampling would be a matter of redrawing a new number any time we got a 2:

let flip = fetchNumberFromZeroToTwoInclusive() 
while (flip === 2) {   
	flip = fetchNumberFromZeroToTwoInclusive() 
}  
if (flip === 0) {   
	console.log("HEADS!") 
} else {   
	console.log("TAILS!") 
}

Ahhh, finally we've restored balance to the force!

Implementing this on top of the drand randomness is an exercise left to the reader (and perhaps the team in due course!), but note that you can implement rejection sampling by just re-hashing drand signatures until you get a value passing your rejection sampling, since the randomness of a drand beacon is obtained by hashing its signature in the first place.

Thanks for reading and as ever we love to hear about anything you've built using drand, so feel free to join us on slack (opens new window) and tell us all about it or ask for help where you need it.

Until next time, adios!