Why we migrated 😼 inboxkitten (77 million serverless API requests) from 🔥 Firebase to ☁️ Cloudflare workers & 🐑 CommonsHost

Get Mail Nyow! 📩

Since our original launch of inboxkitten.com, a free opensource serverless project for creating a disposable email service…

The Stack : Making a free open-source disposable email service (inboxkitten.com) in 14 hours 😼
Eugene Cheah

#opensource#serverless#javascript#vue

The 🐈 kittens have multiplied and gotten out of hand…

3rd on product hunt
February bill shock of: $143

221 GB of static website data (this is from cloudflare cache miss)
215 GB of email data
77 Million API requests
20k+ Unique Visitors (according to cloudflare)
Depletion of my steam summer sale funds 😭 – that $143 got to come from somewhere

Credit : Samantha Whitten

Ok cat team, we need a plan to protect the summer sale! 💳

One of the key limitations of existing AWS lambda or GCP/Firebase cloud functions is that it limits 1 request to an application instance at any point of time.
While that is awesome, if you are compressing images, or doing complex stuff.
Inboxkitten API only role is to make HTTP requests to mailgun, which holds the actual email, with the API key, and echo the result back.
As a result our resource consumption is well below 1% of a CPU, or <10MB of ram per request. Well, below the minimum of 128MB ram and dedicated CPU per request. So our original plan was that once these kittens grow in very constant load, would be to toss it onto a $5/month Linode Nanode 1GB instance. However, the downside is that it wouldn't "scale" automatically at peak workload or if it ever goes to a gazillion request a month, which would be beyond one instance. (Not that we would ever need to support such workload) So in keeping to the spirit of serverless fun (which made this project popular), we would put that option aside and ask... What if, there was a serverless platform, with an alternative billing model. What if it bills only per request, or only for the amount of CPU and ram used. What if we ... Use cat Cloudflare workers ☁️ Cloudflare worker, which is part of the growing trend of "edge" serverless computing. Also happens to simplify serverless billing down to a single metric. Turning what was previously convoluted, and hard to decipher bills from GCP... Billable Item Usage Unit Total Cost Avg price (per million invocations) Invocation 77,418,914 invocation $30.17 $0.390 CPU Time 4,677,797 GHZ-second $44.78 $0.578 Outgress 215.12 Gibibyte $25.21 $0.326 Memory Time 2,923,623 Gibibyte second $6.31 $0.082 Log Volume 61.16 Gibibyte $5.58 $0.072 Total: $112.05 $1.447 Like seriously, try to guess how much your API will cost for a million request in advance is kinda impossible on GCP / AWS, without historical data Into something much easily understood, and cheaper overall per request ... Billable Item Usage Unit Total Cost Avg price (per million invocations) Invocation 77,418,914 invocation $39 $0.5 Total: $39 $0.5 Note that there is a minimum of $5/mth 😸 That's enough net savings for 7 more $9.99 games during summer sale! And has a bonus benefit of lower latency edge computing! But, there are catches ... 1) <5ms CPU time limit Each request is limited to < 5ms of CPU time, this differs greatly for the request time (also known as wall clock time), in that it only counts the amount of time the CPU spent on our function itself and ignoring all its sleep/wait time. This is as opposed to how GCP or AWS measure the time taken from start to end of the function, including all sleep/wait time. In such a setup, serverless functions which fold DNA strands or cat images, with large CPU or RAM consumption. Will find Cloudflare unusable. However, our serverless functions spend 99.99% of the time waiting for mailgun API to respond. Making the <5ms limit purrfect. Also if we ever need to upgrade this to <10ms, it is an optional addon. This is your captain speaking: Jumping ship is a go! 2) Incompatibility with express.js (as it uses web workers pattern) Another thing to note is that Cloudflare workers are based on the web workers model, where it hooks onto the "fetch" event in cloudflare like an interceptor function. So instead of the following in express JS (as we currently do for firebase) const express = require('express') const app = express() app.get('/', (req, res) => res.send(‘Hello World!’))

const port = 3000
app.listen(port, () => console.log(`Example app listening on port ${port}!`))

In Cloudflare, it would be the following
addEventListener(‘fetch’, event => {
event.respondWith(fetchAndApply(event.request))
})

async function fetchAndApply(request) {
return new Response(‘hello world’)
}

As this is a huge fundamental change in the code structure. It can be a major no go for larger existing projects, due to the sheer amount of rewrite work involved. Despite the extremely close similarity.

Perhaps one day someone will come up with a Cloudflare to express.js adapter?

But for us, due to the simplicity of the project, it would be just a simple rewrite. You can compare the difference in the code between express js version here and the Cloudflare version here on github.
And while workers is an open standard, with Cloudflare being the only major provider, it is currently also a form of vendor lock-in.

I can’t squeeze an express cat, into a cloudflare worker, out of the box
3) one script limitation per domain (unless you are on enterprise)
While not a deal breaker for inboxkitten, this can be one for many commercial / production workload.
Because Cloudflare serverless packages cannot be broken down into smaller packages for individual subdomains and/or URI routes.
This greatly complicates things, making it impossible to have testing and production code separated on a single domain, among many more complicated setups.
However in our use case, as this is a hobby project…. it doesn’t matter…

Sadly we cant share multiple different cats across a single box domain easily

The scratches in summary…

<5ms CPU time limit
Incompatible with express.js
one script limitation per domain

3 problems with cloudflares workers, and we do not care!
And all it took was a quick one day rewrite, by @jmtiong
, and we are done.

That’s cool, but Why do I even need Inboxkitten disposable email for again?

One of the key use cases currently, and why we built this project, is to perform email validations as part of our automated test scripts. Such as the following…

// Lets goto inbox kitten
I.goTo("https://inboxkitten.com");
I.see("Open-Source Disposable Email");

// Go to a random inbox inbox
I.fill("email", SAMPLE.id(22));
I.click("Get Mail Nyow!");

// Check that its empty
I.see("There for no messages for this kitten :(");

// Testing for regular email
// (sent using a jenkins perodic build)
I.goTo("https://inboxkitten.com");
I.see("Open-Source Disposable Email");
I.fill("email", "ik-reciever-f7s1g28");
I.click("Get Mail Nyow!");

// See an email we expect, nyow
I.see("Testing inboxkitten subject");

With sharable test results such as
Catfeeding: Uilicious testing inboxkittens XD
Plus it’s simple, cool, and fun to tinker around with.

cat and sheeps can be friends!

Hey, how about the $31 firebase static file hosting?

The easiest free solution would be to throw the entire website onto github pages
However in the spirit of opensource…
We are gonna do a shout out for our friends from 🐑 commonshost.com, an open source static site hosting platform, being built out of Singapore 🇸🇬

And help them push their network with a real production work load test across their 22+ servers on its global network.
As to why commons host instead of GitHub… cause its cool, and I want to support the underdog of the CDN world, being an underdog of the testing world.
Oops, did I say underdog? I mean undercat 😼

What’s next?

Due to the project rather unique simplicity (quick to rewrite), and its heavy production load. I am thinking of potentially expending its deployment option to as many serverless options as possible, or docker based deployments even.
Exploring the various trade-offs with actual 24/7 production load.

Docker container deployment

ECS Fargate
Digital Ocean Kubernetes

AWS lambda
Others??

Let’s see where this cat ship will sail next… Till then we will be taking a cat nap

Happy Shipping 🖖🏼🚀

Link: https://dev.to/uilicious/why-we-migrated-serverless-inboxkitten-from-firebase-to-cloudflare-workers-and-commonshost-gd3