APIs

Mix.install([
  {:jason, "~> 1.4"},
  {:kino, "~> 0.9", override: true},
  {:youtube, github: "brooklinjazz/youtube"},
  {:hidden_cell, github: "brooklinjazz/hidden_cell"},
  {:finch, "~> 0.16.0"}
])

Navigation

Home

Report An Issue

Common Components

Pokemon API

Review Questions

Upon completing this lesson, a student should be able to answer the following questions:

What are the common HTTP request methods and what do they do?
How do we send a cURL request?
How do we send an HTTP request with an HTTP client such as Finch?
What is JSON and how can we parse it into an Elixir term?
What are Bearer Tokens and how can we use them to authenticate and authorize a request?
Why is it bad to expose access tokens and how can we prevent or recover from doing so?

Overview

API stands for Application Programming Interface. In broad terms, an API is a way to communicate between pieces of software.

Generally, API refers to web APIs, which are programs that run on the internet. The internet is a network of interconnected machines that know how to communicate with each other.

We won't go deep into networking or how the internet works, as that is beyond the scope of this course. However, Crash Course Computer Science provides an excellent overview.

YouTube.new("https://www.youtube.com/watch?v=AEaKrq3SpW8")

Client-Server Model

These APIs use a client-server model. Servers provide a resource, and clients request a resource.

You use APIs every time you open your browser (Chrome, Safari, Firefox, Edge, etc.). The browser is the client, and it requests information from a server. For example, when you search for a video on YouTube, your browser communicates with YouTube servers to retrieve the video file.

URL (Uniform Resource Locator)

A URL (Uniform Resource Locator) is a string of characters that specifies the address of a resource on the internet. A URL consists of different parts, each with its own meaning:

Protocol: The protocol specifies how the client and server communicate, for example, HTTP, HTTPS, FTP, etc.
Domain name: The domain name is the address of the server that hosts the resource. For example, in the URL "https://www.example.com", "www.example.com" is the domain name.
Path: The path specifies the location of the resource on the server. It comes after the domain name and starts with a forward slash ("/"). For example, in the URL "https://www.example.com/path/to/resource.html", "/path/to/resource.html" is the path.
Query string: The query string contains additional parameters that modify the request or provide data to the server. It starts with a question mark ("?") and includes one or more key-value pairs separated by ampersands ("&"). For example, in the URL "https://www.example.com/search?name=example&sort=date", "name=example" and "sort=date" are query string parameters.

Your Turn

Identify the Protocol, Domain Name, Path, and Query string (separated into separate query parameters and their values) in the following url:

https://www.fakedomaindoesnotexist.com/fake/domain?greeting=hello&fake=true

HTTP Request Methods

HTTP (Hypertext Transfer Protocol) is the protocol used for transferring data on the web. The HTTP request methods, also known as verbs, indicate the desired action to be performed on the identified resource.

flowchart LR
Client --HTTP Method--> Server

Loading

GET: The GET method requests a representation of the specified resource. GET is the most common HTTP method and is used to retrieve data. It is a safe method, which means that it should not have any side effects on the server or the resource, and it should only retrieve data.
POST: The POST method submits an entity to the specified resource, often causing a change in state or side effects on the server. It is used to create a new resource or to submit data to be processed by the resource identified by the URI.
PUT: The PUT method replaces all current representations of the target resource with the request payload. It is used to update a current resource with new data.
PATCH: The PATCH method applies partial modifications to a resource. It is used to update only a part of the current resource, rather than replacing the whole resource like PUT.
DELETE: The DELETE method deletes the specified resource. It is used to delete a resource identified by a URI.

When you use a browser, you're actually using HTTP under the hood. The browser hides the details of how we use HTTP to communicate with these APIs.

As developers, we want to interact with web APIs directly using the HTTP protocol to request and send information.

Response Codes

APIs use various response codes to communicate the status of a request. These are generally called response codes and are grouped into five classes.

Informational responses (100–199)
Successful responses (200–299)
Redirection messages (300–399)
Client error responses (400–499)
Server error responses (500–599)

For example, you might be familiar with seeing the 404 not found response code.

Web APIs

We use web APIs to communicate with servers on the internet. These APIs provide useful functionality that would otherwise be difficult to build on our own. Here are a few example APIs.

Stripe API handles Stripe payments.
OpenWeatherAPI provides live weather information.
JokeAPI returns a random joke.
OpenAI API generate text, images, and perform other AI related tasks.

cURL

cURL (pronounced like "curl") stands for "client for URL". cURL is a command-line tool used to transfer data to or from a server.

You can make a simple cURL request by typing the following in your command line.

curl https://www.example.com

It's important to understand cURL because most APIs include documentation using cURL requests examples and we can use these simple curl requests to test an API or know how to send a request in our chosen Elixir API Client.

Here, we use the System module to simulate running a curl request from your command line and print the response for the sake of example. Notice it returns the HTML document of https://www.example.com.

{response, _exit_status} = System.cmd("curl", ["https://www.example.com"])

IO.puts(response)

cURL Options

cURL options are used to customize and modify the behavior of curl commands. Here are some commonly used curl options:

-X: Specifies the HTTP method to be used, such as GET, POST, PUT, or DELETE.
-H: Specifies an HTTP header to include in the request, such as Content-Type or Authorization.
-d: Specifies data to include in the request body, such as form data or JSON data.

Your Turn

Run the following fake curl request in your command line. The \ character allows us to split up a command into multiple lines.

curl -X POST \
-H "Content-Type: application/json" \
-d '{"username":"johndoe","password":"password123"}' https://www.example.com/login

You should see a response similar to the following:

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
        <head>
                <title>404 - Not Found</title>
        </head>
        <body>
                <h1>404 - Not Found</h1>
                <script type="text/javascript" src="//obj.ac.bcon.ecdns.net/ec_tpm_bcon.js"></script>
        </body>
</html>

Authentication

APIs use different forms of authentication to check if a user is authorized to access certain resources. Typically most forms auth authentication use some kind of access token to identify the user accessing the resource. Public APIs will not require any kind of access token, while private APIs will.

Bearer Tokens

Bearer Tokens are a type of access token used to authenticate and authorize requests to access protected resources in APIs.

Typically APIs allow you to create an account and receive an API key. We use this key as a bearer token that allows us to access API resources.

For example, here's an example API request to the Open AI API that requires a bearer token to create an image of a siamese cat. $OPENAI_API_KEY would be replaced with the actual bearer token to make the request.

curl https://api.openai.com/v1/images/generations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "prompt": "a white siamese cat",
    "n": 1,
    "size": "1024x1024"
  }'

Your Turn

First, try copy/pasting the curl request above into your command line. You should see the following error:

{
  "error": {
    "code": null,
    "message": "Invalid authorization header",
    "param": null,
    "type": "server_error"
  }
}

Then, follow these steps to create a bearer token and make an authenticated request to generate an AI image.

Create a free Open AI Account: https://auth0.openai.com/u/signup/
Find or create an API key: https://platform.openai.com/account/api-keys
Send the cURL request above in your command line. Make sure to replace $OPENAI_API_KEY with the api key you created.

You should see a response that looks something like this:

{
  "created": 1683498991,
  "data": [
    {
      "url": "https://oaidalleapiprodscus.blob.core.windows.net/private/org-xZKFBB8qReE9rPvITFyOR6WR/user-bOWL7A3HQkffCs8ssJEx4gsI/img-nCmoC73zliZ9YuUCAxQ3043K.png?st=2023-05-07T21%3A36%3A31Z&se=2023-05-07T23%3A36%3A31Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-05-07T17%3A44%3A05Z&ske=2023-05-08T17%3A44%3A05Z&sks=b&skv=2021-08-06&sig=ovkn87xw2l981zy62Aio3GErj7bfRF7zUL8HJulfL5s%3D"
    }
  ]
}

Go to the URL created in your browser to see the generated image.

You will use this Bearer token later on in this lesson. If you run out of free credits or have any other issues, request a bearer token to use from your teacher.

Finch

Finch is one of many HTTP clients that allow us to send HTTP requests in Elixir.

We've chosen to teach Finch instead of other popular libraries such as HTTPoison or Req because it's included by default with Phoenix 1.7 projects.

To send a request with Finch we need to start the Finch process. This is typically done in the application's supervision tree, but we'll do it here for demonstration purposes. Finch starts under a named process. We've chosen MyApp.Finch but this name is arbitrary and typically done for you in a Phoenix project.

Finch.start_link(name: MyApp.Finch)

Once the Finch process has started, to make a request we build the request information in a Finch.Request struct using Finch.build/5.

request = Finch.build(:get, "https://www.example.com")

Then we provide this struct to the Finch.request/3 function using the name of the Finch process to send the request.

Finch.request!(request, MyApp.Finch)

Jason

JSON is a popular format for storing information in a key-value structure.

{
  "key1": "value1",
  "key2": "value2",
}

Elixir represents JSON as a string, not a key-value structure. For example, the above in Elixir would be:

"{\"key1\":\"value1\",\"key2\":\"value2\"}"

We can use the popular Jason to encode an Elixir term into JSON and decode JSON into an Elixir term such as a map.

Jason.decode!("{\"key1\":\"value1\",\"key2\":\"value2\"}")

Jason.encode!(%{"key1" => "value1", "key2" => "value2"})

APIs commonly return a JSON response. For example, here we make a request to a JokeAPI that returns a JSON response.

response =
  Finch.build(:get, "https://v2.jokeapi.dev/joke/Any?safe-mode&format=json")
  |> Finch.request!(MyApp.Finch)

The response body is a JSON string that we need to decode. We can get the body out of the response and then use Jason to decode it into a map.

decoded_body = Jason.decode!(response.body)

Now we can work with the decoded body like any Elixir map. For example we can access the "joke" field.

decoded_body["joke"]

Building A Finch Request

We can build a more complex request using Finch.build/5. It's an important skill to read documentation in cURL and convert it into an HTTP request using an HTTP client. For sake of example, we're going to convert our previous Open AI API request into a Finch request.

Take the following cURL request from the Open AI documentation.

curl https://api.openai.com/v1/images/generations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_TOKEN" \
  -d '{
    "prompt": "a white siamese cat",
    "n": 1,
    "size": "1024x1024"
  }'

There are a few key details we can gather from this request.

Method: The HTTP method of the request. Typically GET, POST, PATCH, PUT, or DELETE.
Headers: HTTP headers provide additional information about a request or response, such as the content type or authentication and are transmitted as key-value pairs.
- Content-Type: Specifies the format of the data being sent in the request body.
- Authorization: Includes authentication information in the request, proving the client has been authenticated and is authorized to access the resource.
data (body): Contains the JSON data to be sent in the request body.

We need to take the information from the cURL request and provide it to Finch.build/5. Here's the full spec for Finch.build/5 that we need to provide.

Finch.build(method, url, headers \\ [], body \\ nil, opts \\ [])

Converting Our cURL Request Into Elixir Terms

Here's the cURL request broken down into valid Elixir terms in the format the Finch expects.

You may replace the $OPENAI_API_TOKEN with a valid token to see the actual API response if you like. However, you should never expose your API tokens, which would happen if you store this Livebook on GitHub.

To avoid exposing your token, make sure to create a new token and revoke the token you used for this lesson: https://platform.openai.com/account/api-keys

method = :post

url = "https://api.openai.com/v1/images/generations"

# Replace The $OPEN_API_TOKEN With Your Token.
# Make Sure To Revoke The Token Later To Avoid Publicly Exposing It.

headers = [
  {"Content-Type", "application/json"},
  {"Authorization", "Bearer $OPENAI_API_TOKEN"}
]

body =
  Jason.encode!(%{
    prompt: "a white siamese cat",
    n: 1,
    size: "1024x1024"
  })

How Do We Know It's A Post Request?

Most of this information is taken directly from the cURL request. However, the Method can be assumed as a POST request, even though it isn't specified. That's because the cURL request includes the -d option. a GET request cannot be sent with data, so cURL inferes that it's a post request. It also says so on the Open AI API Reference for image generation.

If we wanted to make this cURL request more explicit, it could be written with the -X option, but that's not necessary and not how many APIs will be documented.

curl https://api.openai.com/v1/images/generations \
  -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_TOKEN" \
  -d '{
    "prompt": "a white siamese cat",
    "n": 1,
    "size": "1024x1024"
  }'

The Request

Putting all of this together, we get the following Finch request, which we can decode using Jason.

request = Finch.build(method, url, headers, body) |> Finch.request!(MyApp.Finch)

Jason.decode!(request.body)

Your Turn

Make a Finch request for the following curl request from the Open AI API Reference.

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Decode the response using Jason and print the response string from the Chat API using IO.puts/2.

Example Solution

We've used the fake token sk-1WHDb0NwRkq3mfyRVparT3BlbkFJ500axJFd8pZ2RKxGJ0x to demonstrate how to replace $OPENAI_API_TOKEN with your bearer token.

request =
  Finch.build(
    :post,
    "https://api.openai.com/v1/chat/completions",
    [
      {"Content-Type", "application/json"},
      {"Authorization", "Bearer sk-1WHDb0NwRkq3mfyRVparT3BlbkFJ500axJFd8pZ2RKxGJ0x"}
    ],
    Jason.encode!(%{
      model: "gpt-3.5-turbo",
      messages: [%{role: "user", content: "Hello!"}]
    })
  )
  |> Finch.request!(MyApp.Finch)

decoded_body = Jason.decode!(request.body)

[%{"message" => %{"content" => message}}] = decoded_body["choices"]

IO.puts(message)

Commit Your Progress

DockYard Academy now recommends you use the latest Release rather than forking or cloning our repository.

Run git status to ensure there are no undesirable changes. Then run the following in your command line from the curriculum folder to commit your progress.

$ git add .
$ git commit -m "finish APIs reading"
$ git push

We're proud to offer our open-source curriculum free of charge for anyone to learn from at their own pace.

We also offer a paid course where you can learn from an instructor alongside a cohort of your peers. We will accept applications for the June-August 2023 cohort soon.

Navigation

Home

Report An Issue

Common Components

Pokemon API

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apis.livemd

apis.livemd

APIs

Navigation

Review Questions

Overview

Client-Server Model

URL (Uniform Resource Locator)

Your Turn

HTTP Request Methods

Response Codes

Web APIs

cURL

cURL Options

Your Turn

Authentication

Bearer Tokens

Your Turn

Finch

Jason

Building A Finch Request

Converting Our cURL Request Into Elixir Terms

How Do We Know It's A Post Request?

The Request

Your Turn

Commit Your Progress

Navigation

Files

apis.livemd

Latest commit

History

apis.livemd

File metadata and controls

APIs

Navigation

Review Questions

Overview

Client-Server Model

URL (Uniform Resource Locator)

Your Turn

HTTP Request Methods

Response Codes

Web APIs

cURL

cURL Options

Your Turn

Authentication

Bearer Tokens

Your Turn

Finch

Jason

Building A Finch Request

Converting Our cURL Request Into Elixir Terms

How Do We Know It's A Post Request?

The Request

Your Turn

Commit Your Progress

Navigation