Welcome to May! We're now live on the new infrastructure for E2. There's some tweaking around the edges that I need to do around some of the scaling policies, but by and large we are there. It is
serverless, meaning that E2 is a cloud-native application running inside of a
Docker container in
AWS FarGate. Sounds futuristic, right? Well in many ways, it is, but here's what's great about it:
It's cheaper: Due to the dynamic scaling policies available, it is reducing our costs by about 30%. There's a few more tweaks left to do, but I suspect there's a few percentage points available there. E2 isn't run as a for-profit business, but being able to withstand a few bumps in the advertising market isn't a bad thing.
It is faster and runs on a modern platform: E2 is now pinned back to Ubuntu LTS and it's trivial to update. We were held back by the incredibly complicated web of libraries that E2 used, and the incompatibilities caused by mod_perl being an under-represented tech stack. By having those as a part of the container build process, we are free to move around more and experiment. I stubbornly wrote something that did what I wanted, but with some more experimentation, I think that Carton is almost certainly correct so we'll move to that whenever it is convenient.
It auto scales/auto-heals!: The last version of the virtual machine E2 auto-scaled, but it took minutes for the scaling element to notice and bring a newly provisioned e2 machine online. Now it takes about three seconds, and it does so in response to the load. This will let us scale up to the "needs a professional engineering team" size, and let me focus on refining the application now instead of maintaining things.
No more clock!: Well technically there is one, but it's fully managed so no time drift. There is still a database however.
Better development experience: This is not a major concern, but it is far faster for me to get items reliably into production from my local laptop. Docker Desktop is pretty polished and doesn't require any Vagrant bootstrapping that is confusing for other contributors to potentially set up.
Entirely different cron system: This can be a little bit subtle, but the way that the frontpage, New Writeups, and the chatterbox cleanout works is 100% different, but should not appear to behave differently.
Over the next month, the major focus is going to get the JavaScript under control. It's currently in several different places and relies upon several quirks of engine to make API calls. We need to get it over to React, so we can take advantage of modern ecosystem support. When I'm not beating my head against that problem, I'll be continuing to pull code out of the database, and put it into the legacy emulation rendering system or the fully templatized rendering system. I'll have another root log update soon about how those changes are going to effect how some site internals work for power users.
Lastly, I've turned on "auto-ads" for Google, which attempts to let them optimize ad unit placement instead of the static top banner. This doesn't impact logged in users, but it does make the logged out user experience more busy. This isn't a permanent change, but I am trying to understand what this does for both revenue and layout.
Thanks,
-jay