Build a real-time leaderboard with Amazon Aurora Serverless and Amazon ElastiCache

Introduction

Background and setup

Provision a database

Plan your data model

Launch a cluster

Design your data model

Add authentication to your application

Deploy your application

Test your application

Clean up and next steps

Test your application

In this module, you use your application that you’ve deployed. Before you do that, let’s do a quick recap of the AWS components you’re using in your application:

Let’s see how each of these pieces tie together. In the following steps, you walk through your different application endpoints using these components.

First, you start with a Registration endpoint, where a new user signs up and creates their account.

Second, you use a Login endpoint where a user can use a client (such as a web application or a mobile app) to authenticate and receive an ID token.

Third, you use a AddUserScore endpoint to record a score for a user.

Fourth, you use the FetchUserScores endpoint to retrieve the top scores for a particular user.

Finally, you use the FetchTopScores endpoint to retrieve the global top scores for the current day and month as well as the top scores of all time.

Time to Complete Module: 20 Minutes

  • Step 1. Register a new user

    Step 1. Register a new user

    The first workflow you review is the Registration endpoint. At this endpoint, a new user signs up by providing login information, like username and password.

    As you look at these endpoints, the relevant snippets of code that are invoked by the endpoint are shown in this guide.

    Your registration endpoint is available by making a POST request to the /users endpoint. You can see the logic for this endpoint in the application/app.js file about halfway down:

    // Create user
app.post('/users', wrapAsync(async (req, res) => {
  await createCognitoUser(req.body.username, req.body.password, req.body.email)
  res.json({ username: req.body.username })
}))

    Your handler takes the request payload and creates a new user in Amazon Cognito.

    You already reviewed the createCognitoUser function in the Amazon Cognito module, so we won’t recap it here.

    Try invoking your Registration endpoint to create a new user. Run the following command in your terminal:

    curl -X POST ${BASE_URL}/users \
  -H 'Content-Type: application/json' \
  -d '{
	"username": "puzzlemaster",
	"password": "Mypassword1",
	"email": "test@hello.com"
}'

    You should see the following output in your terminal:

    {"username":"puzzlemaster"}

    Great! You’ve successfully created your user. Now, let's login and receive an ID token.

  • Step 2. Login to fetch user credentials

    Step 2. Login to fetch user credentials

    Your second endpoint is the Login endpoint. Users submit their username and password to this endpoint to receive an ID token, which is used to authenticate them on subsequent requests.

    To handle this authentication and token dispensing, your application has a /login endpoint. The handler is in application/app.js as follows:

    // Login
app.post('/login', wrapAsync(async (req, res) => {
  const idToken = await login(req.body.username, req.body.password)
  res.json({ idToken })
}))

    It expects the payload body to have username and password properties. It then calls your helper login function from your auth.js file. If the login is successful, it returns the ID token for the user.

    Let’s test it out with your first created user. Run the following command in your terminal:

    curl -X POST ${BASE_URL}/login \
  -H 'Content-Type: application/json' \
  -d '{
	"username": "puzzlemaster",
	"password": "Mypassword1"
}'

    You should see a response with an idToken property in your terminal:

    {"idToken":"eyJraWQiO…."}

    This ID Token is used in subsequent requests to authorize a user. Save the value of the ID token in your shell by copying the value of the token between the quotations, then running the following command:

    export ID_TOKEN=<idToken>

    Note: This identity token is a temporary credential that will expire over time. If you return to this tutorial after a period of time, you may need to re-run the steps above to generate a new token.

  • Step 3. Record score for user

    Step 3. Record score for user

    Now that your user has a way to authenticate, let’s record a score for a user. You can imagine this would happen after a user completes a level of the game on their mobile device. The application would upload the score and level to the backend for persistent storage.

    The endpoint for creating a new game is POST /users/:username, and the handler code in application/app.js is as follows:

    // Add new score
app.post('/users/:username', wrapAsync(async (req, res) => {
  const validated = validateCreateScore(req.body)
  if (!validated.valid) {
    throw new Error(validated.message)
  }
  const token = await verifyToken(req.header('Authorization'))
  if (token['cognito:username'] != req.params.username) {
    throw new Error('Unauthorized')
  }
  const score = await addUserScore(req.params.username, req.body.level, req.body.score)
  res.json(score)
}))

    This code is doing two things. First, it verifies the ID token that is sent over in the Authorization header. If this token is valid and the username from the token matches the username in the URL path, then the username is passed into the addUserScore function in your data package.

    The addUserScore function is in application/data/addUserScore.js and looks as follows:

    const { executeWriteSql } = require('./utils')
const redis = require('redis')

const client = redis.createClient({
  url: `redis://${process.env.REDIS_ENDPOINT}`
})
client.on('error', function(err) {
  console.log('Received Redis error:', err)
})

const addUserScore = async (username, level, score) => {
  sql = `INSERT INTO games (username, level, score) \
VALUES (:username, :level, :score)`
  parameters = [
    {
      name: 'username',
      value: { stringValue: username }
    },
    {
      name: 'level',
      value: { longValue: level}
    },
    {
      name: 'score',
      value: { longValue: score}
    }
  ]
  const result = await executeWriteSql(sql, parameters)
  const gametime = new Date()
  const key = `${username}|${gametime}|${game.level}`
  return new Promise((resolve, reject) => {
    client.multi()
      .zadd('Overall Leaderboard', game.score, key)
      .zadd(`Monthly Leaderboard|${gametime.getUTCMonth()}-${gametime.getUTCFullYear()}`, game.score, key)
      .zadd(`Daily Leaderboard|${gametime.getUTCDay()}-${gametime.getUTCMonth()}-${gametime.getUTCFullYear()}`, game.score, key)
      .exec((err) => {
        if (err) {
          reject(err)
        }
        resolve({ username, gametime, level, score })
      })
  })
}

module.exports = addUserScore

    The addUserScore function generates a current timestamp and saves the game score to your relational database for long-term storage. Then, it saves the game record in the leaderboard Sorted Sets for the relevant day and month, as well as the all-time leaderboard. This is the same strategy you used when loading all the sample data into your Redis instance.

    Let’s try using this endpoint. Run the following command in your terminal:

    curl -X POST ${BASE_URL}/users/puzzlemaster \
 -H 'Content-Type: application/json' \
  -H "Authorization: ${ID_TOKEN}" \
  -d '{
	"level": 37,
	"score": 6541
}'

    Note that you’re passing the ID token of the first user in the Authorization header with your request.

    You should see the following output in your terminal:

    {"username":"puzzlemaster","gametime":"2019-11-12T03:18:51.637Z","level":37,"score":6541}

    This new score has been recorded for your user!

    Try running this endpoint again but without an ID token. Run the following command in your terminal:

    curl -X POST ${BASE_URL}/users/puzzlemaster \
 -H 'Content-Type: application/json' \
  -d '{
	"level": "37",
	"score": "6541"
}'

    Because your request does not include a token, you should see the following output:

    {"message":"jwt must be provided"}

    You have used your Amazon Cognito tokens to protect an endpoint.

    Before moving on, load two more scores into your database to make the next step more exciting. Run the following two requests in your terminal.

    curl -X POST ${BASE_URL}/users/puzzlemaster \
 -H 'Content-Type: application/json' \
 -H "Authorization: ${ID_TOKEN}" \
  -d '{
	"level": "42",
	"score": "7142"
}'

curl -X POST ${BASE_URL}/users/puzzlemaster \
 -H 'Content-Type: application/json' \
 -H "Authorization: ${ID_TOKEN}" \
  -d '{
	"level": "48",
	"score": "9901"
}'

  • Step 4. Fetch the top scores for a user

    Step 4. Fetch the top scores for a user

    In this step, you can fetch all the scores for your user. This can be used in your application to show a personal leaderboard for a user. You can also use it right now to ensure your data is being persisted properly.

    The endpoint to fetch a user is GET /users/:username, and the handler code is contained in application/app.js as follows:

    // Fetch user scores
app.get('/users/:username', wrapAsync(async (req, res) => {
  const limit = req.query.limit || 10;
  const scores = await fetchUserScores(req.params.username, limit)
  res.json(scores)
}))

    This endpoint is not an authenticated endpoint as each user can see details about other users if desired.

    Invoke the endpoint and fetch your user by entering the following command in your terminal:

    curl -X GET ${BASE_URL}/users/puzzlemaster

    You should see the following output in your terminal:

    [{"game_id":303,"username":"puzzlemaster","gamedate":"2019-11-12 03:21:55","score":9901,"level":48},{"game_id":302,"username":"puzzlemaster","gamedate":"2019-11-12 03:21:47","score":7142,"level":42},{"game_id":301,"username":"puzzlemaster","gamedate":"2019-11-12 03:18:51","score":6541,"level":37}]

    Success! You retrieved your user, and it has the three scores you entered on the previous step. Notice that the scores are stored from highest to lowest so that you can show your users their top scores first.

  • Step 5. Fetch the top scores for a date

    Step 5. Fetch the top scores for a date

    In this final step, you fetch the top scores for a particular date. This endpoint returns the top scores for the date and month of the given date, as well as the top overall scores.

    The endpoint to fetch a user is GET /scores/:date, where :date is a date in the format of YYYY-MM-DD. The handler code is contained in application/app.js as follows:

    // Fetch top scores
app.get('/scores/:date', wrapAsync(async (req, res) => {
  const scores = await fetchTopScores(req.params.date)
  res.json(scores)
}))

    The handler is pretty simple as it passes the date parameter into the fetchTopScores data method. That method is contained in application/data/fetchTopScores.js. The contents of that file are as follows:

    const redis = require('redis')
const _ = require('lodash')
const client = redis.createClient({
  url: `redis://${process.env.REDIS_ENDPOINT}`
})

client.on('error', function(err) {
  console.log('Received Redis error:', err)
})

const parseKey = (key) => {
  const parts = key.split('|')
  return {
    username: parts[0],
    gamedate: parts[1],
    level: parts[2]
  }
}

const parseZRevRangeResponse = (resp) => {
  const result = _.chunk(resp, 2).map(([key, score]) => {
    const obj = parseKey(key)
    return {
      ...obj,
      score
    }
  })
  return result
}

const fetchTopScores = async (date) => {
  const gametime = new Date(date)
  return new Promise((resolve, reject) => {
    client.multi()
      .zrevrange('Overall Leaderboard', '0', '4', 'WITHSCORES')
      .zrevrange(`Monthly Leaderboard|${gametime.getUTCMonth()}-${gametime.getUTCFullYear()}`, '0', '4', 'WITHSCORES')
      .zrevrange(`Daily Leaderboard|${gametime.getUTCDay()}-${gametime.getUTCMonth()}-${gametime.getUTCFullYear()}`, '0', '4', 'WITHSCORES')
      .exec((err, resp) => {
        if (err) {
          reject(err)
        }
        const overall = parseZRevRangeResponse(resp[0])
        const weekly = parseZRevRangeResponse(resp[1])
        const daily = parseZRevRangeResponse(resp[2])
        resolve ({
          overall,
          weekly,
          daily
        })
      })
  })
}

module.exports = fetchTopScores

    The fetchTopScores method takes the date string and turns it into a Date object. Then it fetches the top scores from the relevant date and month Sorted Sets from Redis as well as the top scores from the overall leaderboard. It parses the results into a structure that’s easier for the client to handle before returning the results.

    Try fetching the top scores by running the following command:

    curl -X GET ${BASE_URL}/scores/2019-11-08

    You should see the following results in your terminal:

    {"overall":[{"username":"puzzlemaster","gamedate":"Tue Nov 12 2019 03:21:55 GMT+0000 (Coordinated Universal Time)","level":"48","score":"9901"},{"username":"debbieschneider","gamedate":"2019-11-09T18:41:27","level":"28","score":"9895"},{"username":"alicia39","gamedate":"2019-11-09T10:39:59","level":"47","score":"9824"},{"username":"rosecolleen","gamedate":"2019-11-10T07:09:51","level":"58","score":"9765"},{"username":"allisonsandra","gamedate":"2019-11-07T22:43:32","level":"62","score":"9760"}],"weekly":[{"username":"puzzlemaster","gamedate":"Tue Nov 12 2019 03:21:55 GMT+0000 (Coordinated Universal Time)","level":"48","score":"9901"},{"username":"debbieschneider","gamedate":"2019-11-09T18:41:27","level":"28","score":"9895"},{"username":"alicia39","gamedate":"2019-11-09T10:39:59","level":"47","score":"9824"},{"username":"rosecolleen","gamedate":"2019-11-10T07:09:51","level":"58","score":"9765"},{"username":"allisonsandra","gamedate":"2019-11-07T22:43:32","level":"62","score":"9760"}],"daily":[{"username":"terriross","gamedate":"2019-11-08T21:31:47","level":"26","score":"9386"},{"username":"alicia39","gamedate":"2019-11-08T15:45:30","level":"60","score":"9323"},{"username":"christopherrichardson","gamedate":"2019-11-08T09:51:28","level":"55","score":"9234"},{"username":"castilloanthony","gamedate":"2019-11-08T15:08:32","level":"77","score":"9175"},{"username":"rodriguezjonathan","gamedate":"2019-11-08T13:56:32","level":"46","score":"9155"}]}

    Nice! If you look closely, you can see that your puzzlemaster user has the top score for the monthly and overall leaderboard.

Conclusion

In this module, you exercised your working endpoints to see how your components worked together. First, you registered a new user, which involved creating a new user in your Amazon Cognito user pool. Second, you exercised the login endpoint to fetch an ID token that can be used by the client to authenticate the user. Third, you used this ID token to authorize the user recording some new scores for the user. Fourth, you retrieved the top scores for a single user. Finally, you retrieved the top scores around the globe.

In the next module, you clean up the resources you created.

Next: Clean up and next steps