AWS API Gateway

What's AWS API Gateway?

Amazon Web Services (AWS) API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. It acts as a "front door" for applications to access data, business logic, or functionality from your back-end services.

AWS API Gateway vs Express.js API Gateway

Express.js

An Express app could act as an API gateway, but it would require you to handle the management, scaling, and security of the API yourself, whereas with AWS API Gateway, it handles these things for you.

Logging

With an express.js app, you would have to manually set up logging for your API, whereas with AWS API Gateway, you can automatically log API requests and responses using Amazon CloudWatch.

import winston from 'winston'
import expressWinston from 'express-winston'
import responseTime from 'response-time'

// middleware for logging requests and how fast they get responded to
app.use(
  expressWinston.logger({
    transports: [new winston.transports.Console()],
    format: winston.format.json(),
    statusLevels: true,
    meta: false,
    msg: 'HTTP {{req.method}} {{req.url}} {{res.statusCode}} {{res.responseTime}}ms',
    expressFormat: true,
    ignoreRoute() {
      return false
    },
  })
)

// to forward requests to other microservices (APIs)
const { createProxyMiddleware } = require('http-proxy-middleware')

app.use(
  '/search',
  createProxyMiddleware({
    target: 'http://api.duckduckgo.com/',
    changeOrigin: true,
    pathRewrite: {
      ['^/search']: '',
    },
  })
)

// ... some other proxy middlewares for other microservices

Authentication

With an express.js app, you would have to implement authentication and authorization mechanisms on your own, whereas with AWS API Gateway, you can easily integrate with AWS Identity and Access Management (IAM) for authentication and authorization.

require('dotenv').config()

const session = require('express-session')

const secret = process.env.SESSION_SECRET
const store = new session.MemoryStore()
const protect = (req, res, next) => {
  const { authenticated } = req.session

  if (!authenticated) {
    res.sendStatus(401)
  } else {
    next()
  }
}

app.use(
  session({
    secret,
    resave: false,
    saveUninitialized: true,
    store,
  })
)

app.get('/login', (req, res) => {
  const { authenticated } = req.session

  if (!authenticated) {
    req.session.authenticated = true
    res.send('Successfully authenticated')
  } else {
    res.send('Already authenticated')
  }
})

app.get('/logout', protect, (req, res) => {
  req.session.destroy(() => {
    res.send('Successfully logged out')
  })
})

app.get('/protected', protect, (req, res) => {
  const { name = 'user' } = req.query
  res.send(`Hello ${name}!`)
})

Rate limiting

With an express.js app, you would have to implement rate limiting on your own, whereas with AWS API Gateway, you can easily configure rate limits on your API.

import rateLimit from 'express-rate-limit'

// rate limiting
const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // Limit each IP to 100 requests per `window` (here, per 15 minutes)
})

// or app.use('/api', limiter)
app.use(limiter)

// ...

const express = require('express')
const cluster = require('cluster')
const os = require('os')

// checks if the cluster is the master one, if it's
// it starts all the workers
if (cluster.isMaster) {
  // get the number of CPU cores
  const numCores = os.cpus().length

  // start a worker for each core
  for (let i = 0; i < numCores; i++) {
    cluster.fork()
  }

  // listen for workers that exit and restart them
  cluster.on('exit', (worker) => {
    console.log(`worker ${worker.id} died. restarting...`)
    cluster.fork()
  })
} else {
  // every time a worker is created, it'll execute this exact file
  // with (cluster.fork)
  // and since it's not the master cluster, it'll run this `else` block

  // create the Express app
  const app = express()

  // add routes
  app.get('/', (req, res) => {
    res.send('Hello, World!')
  })

  // start the server
  const port = 3000
  app.listen(port, () => {
    console.log(`Worker ${cluster.worker.id} listening on port ${port}`)
  })
}

AWS API Gateway

Other features

• Integration with AWS WAF for protecting your APIs against common web exploits.
• Integration with AWS X-Ray for understanding and triaging performance latencies.

Some important concepts

• API key: An alphanumeric string that API Gateway uses to identify an app developer who uses your REST or WebSocket API.

REST APIs vs HTTP APIs

The major difference, an HTTP API only works with Lambda and HTTP backends (some Express.js API), while a REST API works with these too, and other AWS services.

Usually, an HTTP API is sufficient for Lambda function or HTTP backends as it costs less and provides lower latency.

Creating an HTTP API for Pokedex with HTTP backend

First, we go to the AWS API Gateway panel and select an HTTP API. The page Create and configure integrations will show up:

Integrations: Type of the integration whether it's a Lambda function or HTTP endpoint

Method: Request method GET, POST, ...etc

URL Endpoint: In our case, it's https://pokeapi.co/api/v2/pokemon-form/pikachu

Next, Configure the routes page. We choose what resource path and method that when are requests, will target https://pokeapi.co/api/v2/pokemon-form/pikachu

We can set the resource path to anything, so I set it to /pokedex and the method is GET instead of ANY even though ANY would've worked.

Next, Configure stages. We can use the default stage name here, and with auto-deploy on.

Stages are associated with a specific deployment of an API. A deployment is a snapshot of an API's configuration and resources at a given point in time. You can create multiple deployment stages, each with its settings, and then use them to test different versions of your API before promoting them to production. They're like GitHub branches.

Once this step is done, we create the API Gateway and we get this link https://7m3eb19o5m.execute-api.us-east-1.amazonaws.com/pokedex that once accessed, redirects our request to pokeAPI and returns the response from it.

Creating a REST API for Pokedex with a Lambda function

For this project, we'll create a Lambda function written in JavaScript that sends a request to PokeAPI and return the response.

Our Lambda function will have the following code:

export const handler = async (event) => {
  const res = await global.fetch(
    'https://pokeapi.co/api/v2/pokemon-form/pikachu'
  )
  return {
    statusCode: 200, // essential
    body: await res.json(),
  }
}

Info: It's essential to return at least a statusCode when Integrating a Lambda Function with AWS API Gateway

To be able to call the lambda function from an endpoint, we'll create a new REST API gateway for it.

Choosing an API type

As we discussed in the REST APIs vs HTTP APIs section, there are 4 types of APIs that we can choose from. We'll pick a public REST API.

In Amazon API Gateway after hitting Create an API, we'll find this page:

We chose our API to be REST instead of Websocket, and create it from scratch. After creating the API, we'll land on this page for configuring the API methods, resources, authorizers, ...etc:

Creating a Resource

Before Integrating the Lambda Function with the API Gateway, we'll create a resource that we can Invoke the Lambda function from instead of the root resource /

Set the resource name and path as pokedex

After we create it, the resources will look like this:

Creating a method for /pokedex resource

We'll create a new GET method for the pokedex resource that Integrates with our Lambda function

Select the Integration type as Lambda Function and pokedex as the Lambda function:

Testing the resource endpoint

Once the method is created, the following page will show and we can adjust our configurations from it or test the API. It's really handy when we want to quickly debug an API that responds with an error:

If we tried to test the API by clicking TEST button and hit Test again in the next page, we'll notice that it throws the error: Execution failed due to configuration error: No match for output mapping and no default output mapping configured

This is because we didn't specify what kind of response the method should respond with. Even though it shows HTTP Status: 200 under Method Response, we still need to specify that.

A quick click to Method Response will open the following view where we can select the response type it should expect from the Integration response:

If we tried to test the API again, it'll work as expected:

Deploying the API

After creating a new API or applying changes to an exisitng one, we always have to deploy it.

We deploy the API from the Actions:

Stages

Stages refer to different versions of an API that can be created, managed, and deployed separately within the API Gateway service. Whenever we deploy an API, we can choose whether we want to overwrite an existing stage or creata a new one.

It's helpful if we want to:

1. Test out a version of an API without affecting the production one
2. Deploy the API to different regions
3. Apply different security settings to different APIs

Invoking the API from a public URL

After we deploy it, we can now invoke the API from the provided Invoke URL that we can find by going to Stages -> prod -> / -> GET:

For our example, it's https://mn9hd5qc51.execute-api.us-east-1.amazonaws.com/Prod/pokedex try to open this URL!

Creating a REST API for an AWS Service (DynamoDB)

AWS services do not typically provide an API for performing operations. However, you can use API Gateway to create an API endpoint that allows you to access an AWS service through an API call. This enables you to perform operations on the service using an API, rather than having to interact with the service directly.

Follow this guide to create a DynamoDB and the REST API resources and methods but generally, learn about what kind of Actions or APIs does the service has. In the case of DynamoDB, we'll use this document.

After following the steps mentioned in it, we'll have these two endpoints:

1. POST Endpoint for creating a new Pokemon
2. GET Endpoint that takes a Pokemon's name path parameter
   1. E.g, https://aqeebamk5e.execute-api.us-east-1.amazonaws.com/prod/item/pikachu

For the POST Endpoint, if we send a request to create a Pokemon named friesmeat and has the age 3, we'll send the following:

curl -X POST \
  'https://aqeebamk5e.execute-api.us-east-1.amazonaws.com/prod/item' \
  --header 'Accept: */*' \
  --header 'User-Agent: Thunder Client (https://www.thunderclient.com)' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "pokemonName": "friesmeat",
  "age": "3"
}'

And it responds with an empty object {} that indicates the operation was successful. The action PutItem doesn't return the created object by spec.

For the GET Endpoint, if we send a request to get the created Pokemon friesmeat:

curl -X GET \
  'https://aqeebamk5e.execute-api.us-east-1.amazonaws.com/prod/item/friesmeat' \
  --header 'Accept: */*' \
  --header 'User-Agent: Thunder Client (https://www.thunderclient.com)'

It'll respond with the item:

{
  "pokemons": [
    {
      "itemId": "f8630ede-a6e4-4528-99aa-933b42f963ae",
      "pokemonName": "friesmeat",
      "age": "3"
    }
  ]
}

Mapping templates

A mapping template is a way to define how incoming data from an HTTP request should be transformed before it is passed to a backend service. This can include changing the format of the data, extracting specific values, or adding default values.

{
  "TableName": "Items", // default value
  "Item": {
    "itemId": {
      "S": "$context.requestId"
    },
    "pokemonName": {
      "S": "$input.path('$.pokemonName')"
    },
    "age": {
      "S": "$input.path('$.age')"
    }
  }
}

In the example given, the mapping template is used to ensure that requests sent to the DynamoDB integration have the correct format.

The template sets the "TableName" to "Items" and defines the structure of the "Item" object, which includes "itemId", "pokemonName", and "age" attributes.

Using this mapping template, the user is only required to provide the necessary information to create the "Item" object, rather than manually constructing the entire request body. It could also be used to map the Integration response to a format that's easier for the user to work with.

#set($inputRoot = $input.path('$'))
{
    "pokemons": [
        #foreach($elem in $inputRoot.Items) {
            "itemId": "$elem.itemId.S",
            "pokemonName": "$elem.pokemonName.S",
            "age": "$elem.age.S"
        }#if($foreach.hasNext),#end
    #end
    ]
}

This mapping template is used to define the format of the response that is sent back to the user when a request is made to DynamoDB integration.

Learn about the mapping template syntax through these examples

Advanced API Gateway usage

Logging

In order to enable logging for an API in API Gateway, you need to create an IAM role that allows API Gateway to write logs to Amazon CloudWatch.

You can follow this guide provided by AWS to integrate logging in API Gateway: https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-cloudwatch-logs/

When sending a GET request, our logs will show details about the request, such as the request method, path, headers, and response status. An example of how the logs might look like:

Authentication

In an API, authentication can be implemented in two ways:

1. Using a Lambda function: You can create a Lambda function that handles the authentication process. This function would act as a middleware (C0D3) that verifies whether the user is authorized to execute the resolver or not.
2. Using a Cognito user pool: AWS Cognito User Pools provide an easy way to handle user sign-up, sign-in, and access control. (Cognito totally went over my head)

Rate Limiting / Request Throttling

API Gateway provides a feature called "Usage Plans" that allows you to specify rate limits and quotas for requests to an API. This feature can be used to control the number of requests per second, the total number of requests, and to track the usage of API keys.

You can configure usage plans for an entire stage or for individual methods. To do this, go to the API Gateway console, select an API, then go to the "Usage Plans" section. There, you can specify the rate limits and quotas for the API, and create API keys for developers to use. This allows you to keep track of how much quota each developer is using, and to prevent excessive usage that might affect the performance of your API.

Resources

• AWS API Gateway
  • Main docs
  • REST API VS HTTP API
  • Mapping templates & Models
• Express.js API Gateway
• Method response type