three-o-none: Fixing redirect chains with Cloudflare Workers

Simon Thompson · December 14th, 2020

A few years ago I was involved in the migration of a large website over to a new CMS, which required finding a way to redirect 100k+ legacy URLs into their new equivalent.

We found a solution for this using a combination of PHP scripts and SQL lookups, however the task of updating references to old URLs within the site content is ongoing to this day. This, combined with subsequent site architecture changes, has meant that a large number of redirect chains now exist. For instance…

In an ideal world each hop would redirect straight to /landing-page/ - saving the client round-trips to the server, and the server the overhead of additional requests.

Whilst it's good practice, staying on top of this is't always possible. Tech constraints and higher priorities can cause chains to build up over time to the point where tackling it is a huge task requiring time and effort which is hard to justify.

Solving this issue seemed like a fun challenge and a perfect application for a technology like Cloudflare Workers, so I set to work building a proof of concept which I've dubbed "three-o-none".

How it works

At a top-level, Cloudflare allows you to route your website through their global network of servers. This allows them to serve as a CDN, prevent DDoS attacks, carry out performance optimisations and apply various other security measures.

One of their features is Workers, which allows developers to write code which runs on Cloudflare's servers and can modify requests on their way to a web server, and responses on the way back. This allows us to build platforms like Spark and experiments like this proof of concept petitions platform.

three-o-none is entirely Workers-based, meaning that the only requirement is that a site is routed through Cloudflare.

Mapping 301 chains

The first step is to build up a map of all of the redirects on a website. Thanks to it's position as a middleman, three-o-none watches for server responses with a 301 status code and keeps track of them using Workers KV.

A benefit to this approach is that no redirect data needs to be manually loaded in, and new redirects will be tracked automatically the first time they're requested.

Rewriting 301 responses

Once it's built up a knowledge of redirects, three-o-none begins to intercept responses with HTTP 301 status codes as they come back from the server.

It then inspects the Location header (which says where the redirect should go), and looks it up in the list of redirects it's tracked. It then looks at the location of that redirect, and follows the chain as far as it can.

If the final location of that chain doesn't match the location sent from the server, three-o-none will rewrite the response Location header to point to the final URL before sending it back to the end user.

Rewriting internal links

With this knowledge of redirects, three-o-none can also automatically rewrite the target (href attributes) of internal links (<a> tags) which point to URLs which redirect.

This means that when a user clicks (or a bot follows) a link, they'll go directly to the final URL rather than bouncing through redirects (flattened or not).

Dashboard

Once installed, all of this functionality is manageable via a simple dashboard, made available on the domain at /seo-cgi/redirects/

The dashboard allows the Redirect Flattening and Link Rewriting features to be quickly toggled off, and provides a reference table of all redirects which have been detected, where they originally pointed, and where they have been flattened to.

Additionally, when the dashboard is first loaded, it will prompt you to set a password to help keep it safe from prying eyes.

Using three-o-none

I'm currently open to working with people to help them try out three-o-none whilst I prepare it for general availability (it'll be open sourced ASAP).

If you think three-o-none may be able to help your website, please click the button below to register your interest, and I'll be in touch as soon as possible!

Register Interest

If you have any questions about three-o-none or have a tricky SEO issue you think Edge SEO might be able to solve, please also get in touch - I'd love to chat!