2021-02-28
In this tutorial, you will learn how to develop apps on Cloudflare workers and Cloudflare KV by building a URL shortening service. You can try a demo of the URL shortener you will build here!
In this tutorial, you will be building a URL shortener with Cloudflare workers and Cloudflare KV.
Cloudflare Workers is a platform for deploying serverless code to run on the same edge network that powers Cloudflare's CDN.
After following this tutorial, you will have a fully working URL shortener and be ready to build your own apps with Cloudflare Workers.
You can try a demo of the URL shortener you will build here!
To complete this tutorial, you will need
Install wrangler globally with the following command:
npm install -g @cloudflare/wrangler
Run the following command to confirm that it is successfully installed:
wrangler -V
You should get an output like the following.
wrangler 1.12.3
Now wrangler is installed, Authenticate wrangler with your Cloudflare account with the following command:
wrangler login
You will be asked to allow wrangler to open a page in the browser. Type Y and press ENTER.
After logging into your account and granting wrangler access, wrangler will now be authenticated.
In the next step, you will bootstrap your worker.
We will create our app from the Cloudflare Workers default template. Open a terminal at the directory where you want the app to be created.
Run this command in the terminal to create the Worker app:
wrangler generate short-linker https://github.com/cloudflare/worker-template
This creates a worker app called short-linker for you from the default Cloudflare Worker template on GitHub.
Enter the directory of the worker app with
cd short-linker
There are a number of files in the newly created short-linker folder. The two main files you will be working with will be the index.js file which is the entry point of the worker and the wrangler.toml which contains configuration data for the worker.
Before you can start a development server, you have to add your Cloudflare account id to wrangler.toml.
Run the following command to find your account id:
wrangler whoami
Your Account ID will be displayed beside your account name. Copy your Account ID from the console.
Edit the line with account_id in wrangler.toml, replacing my-account-id with the account ID you copied.
name = "short-linker"
type = "javascript"
account_id = "my-account-id"
workers_dev = true
route = ""
zone_id = ""
Start the development server with the following command.
wrangler dev
This creates a preview deployment that is accessible from http://localhost:8787.
Run the following command in the terminal to make a request your worker
curl http://localhost:8787
You should have this output
Hello Worker
An alternative way to test the worker is to visit http://localhost:8787 in your browser.
If you face any errors, make sure you added your Account ID to wrangler.toml.
Open index.js. The contents should look as below
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})
/**
* Respond with hello worker text
* @param {Request} request
*/
async function handleRequest(request) {
return new Response('Hello worker!', {
headers: { 'content-type': 'text/plain' },
})
}
At the top, we have an event listener listening for fetch events. In that event listener, we pass the event object to a handler handleRequest that returns a response based on the data contained in the object passed to it.
The handleRequest function simply returns a Response object with a body and some headers.
Replace Hello worker with Hi worker in index.js and save.
When you save, the development server will automatically detect the changes and redeploy the changed code.
Preview your changes by running
curl http://localhost:8787
The result will now change to this.
Hi worker
The request object that is passed to the handleRequest function contains all the data in the request body. This includes the URL that the request was made to, the method of the request, the body of the request, some special objects added at the Cloudflare edge, etc.
Some of the request object's most useful properties and methods are
request.method which is the method of the request e.g. "POST", "GET", "UPDATE"request.url which is the URL where the request was made to.request.text() which returns the request body as plain textrequest.json() returns the request body as a json objectYou can find a full list of the properties of the request object at the documentation page for Request.
All requests to our worker regardless of the path or method will pass through handleRequest. You will use these properties and methods of request to determine the worker's response. The default worker, for example, returns the same response regardless of the path you visit.
A request to http://localhost:8787/test-path, gets the same response as one to http://localhost:8787.
To vary the response based on the URL path, you will have to write design your code to give varying responses based on the value of request.url. Similarly, to run separate logic for POST and GET requests to the same URL, you have to write a conditional statement that handles the case when request.method == "POST" and when request.method == "GET".
You can test that the responses are the same, regardless of method or path by running these curl commands from your command line
curl -X GET http://localhost:8787/some/sub/folder
curl -X POST http://localhost:8787
curl -X POST http://localhost:8787/api
curl -X GET http://localhost:8787
To illustrate the idea of changing the response based on the request method, modify index.js to look as follows:
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})
/**
* Respond with hello worker text
* @param {Request} request
*/
async function handleRequest(request) {
if(request.method == "POST") {
return new Response('Hello POST worker!', {
headers: { 'content-type': 'text/plain' },
})
}
return new Response('Hi worker', {
headers: { 'content-type': 'text/plain' },
})
}
Run the following command from a terminal to make a POST request to the worker:
curl -X POST http://localhost:8787
You will have the following response
Hello POST worker
Run the following command too
curl -X GET http://localhost:8787
The response remains the same as before
Hi worker
The code modification checks if the request is a POST request, and if so, returns "Hello POST worker" and returns "Hi worker" instead. You should now have a good understanding of what the request object is used for.
You will now proceed to build the URL shortener.
We will start by defining some terms that we will be using:
http://localhost:8787/abc123 is abc123.When you make a POST request to http://localhost:8787, with the body of the request containing the URL you want to redirect to, the worker will generate a random string, for example abc123. We will store a map from this random string to the destination URL contained in the request body. On a visit to http://locahost:8787/abc123, we will extract the random string at the end, find what destination that string is tied to, and finally redirect to that URL.
We need some form of persistent storage to store the random strings and the URL the string is tied to. Cloudflare KV is a perfect fit for this need. Cloudflare KV is a key-value storage just like the Redis database. A key-value is similar to a big dictionary where you can assign values to keys, and also read values for keys.
In Cloudflare KV, a collection of keys and values is known as a namespace. A namespace is what you will call a table in a relational database, only that it has two columns - one for the keys and another for the values.
Create a namespace called URL_SPACE with the following command:
wrangler kv:namespace create "URL_SPACE"
You will be prompted to add some extra values to the kv_namespaces array in wrangler.toml.
Creating namespace with title "short-linker-URL_SPACE"
Success!
Add the following to your configuration file in your kv_namespaces array:
{ binding = "URL_SPACE", id = "943543435ac8e2a7a4c6b89fe343c0c13a02" }
Open wrangler.toml and update the needed lines. Replace namespaceid with the id that is given.
...
kv_namespaces = [
{ binding = "URL_SPACE", id = "namespaceid" }
]
...
What this does is that it binds a URL_SPACE variable in our worker to this namespace. This means we can write to the newly created namespace in our worker with URL_SPACE.put("key", "value") and read a value with URL_SPACE.get("key").
Since we are working on a preview deployment, we have to make an extra modification to kv_namespaces.
Edit the kv_namespaces array in wrangler.toml and add the preview_id.
name = "short-linker"
type = "javascript"
account_id = "my-account-id"
workers_dev = true
route = ""
zone_id = ""
kv_namespaces = [
{ binding = "URL_SPACE", id = "namespaceid", preview_id = "namespaceid" }
]
You will receive a warning for using the same namespace for production and preview deploys but you can ignore it for now. This is because the value of id is the namespace id of the production namespace and the value of preview_id is the id of the namespace that will be used during development.
The option to use different namespaces in different environments is there to allow developers to work locally with a separate namespace from the one deployed to production and avoid accidentally breaking production.
You will now create an endpoint to generate short URLs. To do this, we will create logic to handle POST requests in handleRequest.
When a POST request is made to the worker, we will create a key-value entry in the namespace where the key will be the randomly generated string, while the value will be a destination URL contained in the request body.
Modify index.js to create a KV entry on POST requests:
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})
/**
* Respond with hello worker text
* @param {Request} request
*/
async function handleRequest(request) {
if( request.method === "POST"){
const neededURL = await request.text()
const rand = Math.random().toString(30).substr(2, 5);
await URL_SPACE.put(rand, neededURL)
return new Response(rand)
}
return new Response('Hi worker!', {
headers: { 'content-type': 'text/plain' },
})
}
When we make a POST request, this gets the URL that we want to redirect to from the request body by calling request.text. It then generates a pseudo-random string with Math.random().toString(30).substr(2, 5) and stores the key-value pair in our namespace with URL_SPACE.put(rand, neededURL).
Make a post request to locahost:8787 by running:
curl -d "https://www.google.com" -H "Content-Type: text/plain" -X POST http://localhost:8787/
You should receive the random string as the response.
rrenb
Another thing you certainly want to do is to make the destination URL have the https protocol if no protocol is added to the URL.
For example, after make the following request
curl -d "www.google.com" -H "Content-Type: text/plain" -X POST http://localhost:8787/`
We want the destination to be stored as https://www.google.com instead of www.google.com.
Modify the handleRequest function:
async function handleRequest(request) {
if( request.method === "POST"){
const neededURL = await request.text()
let cleanUrl
if(!neededURL.match(/http:\/\//g) && !neededURL.match(/http:\/\//g)){
cleanUrl = "https://" + neededURL
}
else {
cleanUrl = neededURL
}
const rand = Math.random().toString(30).substr(2, 5);
await URL_SPACE.put(rand, cleanUrl)
return new Response(rand)
}
return new Response('Hello worker!', {
headers: { 'content-type': 'text/plain' },
})
}
The extra addition does a regular expression search for http:// and https:// in the destination URL, and if neither is found, it appends https:// to the destination URL before storing it.
You have now successfully generated and stored the random strings together with the corresponding destination. In the next step, we will redirect short link visits to the intended destinations.
To achieve this aim, you will use a regular expression search on the request URL to strip off the https://.../ or http://.../ part of the URL and get the random string. You will then search for the entry in the URL_SPACE namespace, whose key is the random string, and redirect the user to that destination.
Edit the handleRequest function:
async function handleRequest(request) {
if( request.method === "POST"){
...
}
if( request.method === "GET" ){
let shortCode = request.url.replace(/https:\/\/.+?\//g, "")
shortCode = shortCode.replace(/http:\/\/.+?\//g, "")
if(shortCode !== "") {
const redirectTo = await URL_SPACE.get(shortCode)
return Response.redirect(redirectTo, 301)
}
else {
return new Response( 'Welcome to shortlinker, make a post request to add a shortlink', {
headers: { 'content-type': 'text/plain' },
})
}
}
return new Response('Hi worker!', {
headers: { 'content-type': 'text/plain' },
})
}
The new code addition intercepts only GET requests. It uses request.url.replace(/https:\/\/.+?\//g, "") to remove the protocol portion of the URL and assigns the random part left to the shortCode variable.
When we visit http://locahost:8787, the above addition will intercept the request, and the shortCode variable will be an empty string. We wouldn't want to redirect a user that visits the homepage. That is why there is a conditional statement to only redirect non-homepage visits. The worker then searches URL_SPACE for the destination that the value of shortCode is tied to and makes the redirect with Response.redirect().
To test this new addition, generate a random string that redirects to https://www.google.com with the following command:
curl -d "https://www.google.com" -H "Content-Type: text/plain" -X POST http://localhost:8787/
You will get a random string as the response.
Visit the short link with the following command, replacing short-string with the random string generated in the previous step
curl -v http://localhost:8787/short-string
You will have an output like the following. You will notice that a 301 redirect is made to https://www.google.com
* Trying ::1...
* TCP_NODELAY set
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8787 (#0)
> GET /short-string HTTP/1.1
> Host: localhost:8787
> User-Agent: curl/7.55.1
> Accept: */*
>...
< HTTP/1.1 301 Moved Permanently
< date: Fri, 23 Apr 2021 19:17:54 GMT
...
< location: https://www.google.com
...
In this step, you set up the redirects for the short links. In the next step, you will add a web interface at the homepage to create new shortlinks directly from a browser.
In this final step, you will add a simple user interface that will be used for generating short links.
Modify index.js:
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})
const htmlBody = `
<html>
<head>
<style>
#url {
font-size: 2em;
width: 100%;
max-width: 500px;
}
#submit {
font-size: 2em;
}
</style>
<script>
const submitURL = () => {
document.getElementById("status").innerHTML="creating short url"
//await call url for new shortlink and return
fetch('/', {method: "POST", body: document.getElementById("url").value})
.then(data => data.text())
.then(data => {
console.log('ready')
document.getElementById("status").innerHTML="Your short URL: http://localhost:8787/" + data
} )
}
</script>
</head>
<body>
<h1 id="title">URL Shortener</h1>
<input type="text" id="url" placeholder="enter url here" />
<button id="submit" onclick="submitURL()">Submit</button>
<div id="status"></div>
</body>
</html>
`
/**
* Respond with hello worker text
* @param {Request} request
*/
async function handleRequest(request) {
if( request.method === "POST"){
const neededURL = await request.text()
let cleanURL
//add necessary protocols if needed to url
if(!neededURL.match(/http:\/\//g) && !neededURL.match(/https:\/\//g)){
cleanURL = "https://" + neededURL
}
else {
cleanURL = neededURL
}
const rand = Math.random().toString(30).substr(2, 5);
await URL_SPACE.put(rand, cleanURL)
return new Response(rand)
}
if( request.method === "GET" ){
let shortCode = request.url.replace(/https:\/\/.+?\//g, "")
shortCode = shortCode.replace(/http:\/\/.+?\//g, "")
if(shortCode !== "") {
const redirectTo = await URL_SPACE.get(shortCode)
return Response.redirect(redirectTo, 301)
}
else {
return new Response( htmlBody, {
headers: { 'content-type': 'text/html' },
})
}
}
}
We created a htmlBody variable that contains the template for the home page's interface. We return this string when a visitor visits the homepage. Since we now have a webpage where we can create short links, visit http://localhost:8787 in a browser. Type in a destination you want to link to in the space provided and click submit. A short link will be generated and displayed. When you visit that short link, you will be redirected to the desired destination.
Now you have successfully created a URL shortener that runs on Cloudflare's edge network. You can read this section on deployment in the Workers documentation to learn how to deploy the production. Before finally deploying the URL shortener, you will have to make a slight modification to the template for the homepage.
Go to the following line in the htmlBody string in index.js
...
document.getElementById("status").innerHTML="Your short URL: http://localhost:8787/" + data
...
Replace http://locahost:8787/ with https://subdomain.workers.dev/ where subdomain is the workers.dev subdomain where your app will be deployed.
You can try a demo of this URL shortener here!