Building IBBE Storage

Every product I have worked on eventually runs into the same problem. Images pile up, videos need a home, documents scatter across different tools, and someone ends up hunting for a file that used to live somewhere obvious. I hit that point with ibbe and decided to build a system that fit exactly how I work, rather than bend my workflow around a tool designed for someone else's needs. That system became ibbe storage, and this post walks through how I put it together.

At a high level, ibbe storage sits at the center of everything I build. My websites, my CMS[1], and any other internal tool talk to one central service whenever they need to store or fetch a file. That service takes the request and routes it to the right place, whether that means writing a new object to storage, updating a record in the database, or pulling something that already exists. Every application connects through the same entry point, which keeps the whole setup predictable and easy to reason about, even as the number of things talking to it grows.

The stack itself stays fairly ordinary on purpose. The dashboard runs on Next.js with React and TypeScript, styled in dark mode and hosted on Vercel. The backend runs on AWS Lambda[2], written in Node.js, with the whole thing deployed through AWS SAM[3] and CloudFormation. The database is PostgreSQL, hosted through Supabase. Files live in Fastly object storage, which also handles delivery through its content delivery network. I use pnpm for package management, GitHub Actions for continuous integration, and CloudWatch alongside X-Ray for monitoring and tracing requests as they move through the system.

I chose a serverless backend because asset traffic moves in bursts rather than a steady stream. A product launch or a big content update can send hundreds of upload requests within minutes, followed by long stretches of quiet. Each Lambda function runs with 1024 megabytes of memory and a 29 second timeout, spinning up only when a request arrives and scaling automatically with demand. This means I pay for what I actually use and avoid maintaining a fleet of servers that sit idle most of the day.

For the actual files, I built a thin abstraction layer between my code and the storage provider, an interface that describes what I need: creating an upload, copying a file, moving it, deleting it, checking its size, generating a public link. My code talks to this interface, and the interface talks to Fastly underneath it. This gives me room to switch providers down the line if pricing or performance ever calls for it, with minimal disruption to the rest of the system.

Uploading a large file is where a lot of systems fall apart, so I spent real time getting this part right. Instead of sending a huge file in a single request, which risks timing out or failing halfway through, ibbe storage breaks it into smaller parts, uploads each part separately, often at the same time, and stitches them back together once every piece arrives. A client requests an upload, the system creates a pending record and hands back a set of temporary, short lived links for each part, the client uploads directly to storage using those links, and once every part lands successfully, the system checks the final file size against what was expected and marks the asset ready to use.

The numbers hold up well across file sizes. A ten megabyte file, split into two parts, finishes in a couple of seconds. A hundred megabyte file, split into around twenty parts, finishes in ten to twenty seconds. A full gigabyte takes roughly one to two minutes. Push it to ten gigabytes across two thousand parts and the upload finishes in ten to twenty minutes, and a twenty gigabyte file, the current maximum the system accepts, finishes in twenty to forty minutes rather than stalling on one enormous transfer.

Every asset lives inside a folder, and folders can nest inside other folders, the same way you would organize files on a computer. Every organization using the system sees only its own folders and assets, keeping content separate and organized on its own terms. Underneath, the data model stays simple: an organization holds folders, a folder holds assets and can hold child folders, and each asset carries its original filename, content type, size, and status, whether it sits pending, ready, or failed.

A small PostgreSQL[4] database, hosted through Supabase, keeps track of every organization, folder, and asset along with these basic details. The database only stores this kind of metadata, the actual files live in object storage, so the database itself stays small and fast even as the total volume of assets grows into the millions. Queries come back in well under a tenth of a second because the data footprint per asset stays tiny.

Once an asset finishes uploading, it becomes available through a public link almost immediately, served through the content delivery network rather than directly from storage. The link follows a predictable pattern based on the asset type and its id, and older single segment links still redirect correctly to the newer format. Images can be resized and converted to modern formats like webp on request, using a fixed set of allowed widths, so a single uploaded photo can serve a thumbnail, a mid sized preview, and a full resolution version, all generated on demand and cached afterward for a full year.

Original uploadOptimized delivery

Delivery performance benefits from a few extra details working together. The network supports HTTP/3 for compatible clients, handles byte range requests for large files like video, and switches to a streaming approach for anything over twenty megabytes so large files start playing before the whole thing finishes downloading. Combined with a full year cache on original assets, most requests after the first one come back almost instantly from the nearest edge location.

On top of the API, I built a dashboard using Next.js, styled in dark mode with Geist for body text and Geist Mono for anything that reads as code or an id, all laid out on an eight pixel spacing grid similar to the feel of a Vercel dashboard. It gives me a folder tree to browse, a drag and drop interface for uploads with progress tracking, a detail view for each asset with its metadata and public link, and a place to manage who can access what. Building this on the same API every other client uses means the dashboard behaves like any other consumer of the system, instead of a special case with hidden shortcuts.

Scalability comes mostly from letting each layer grow on its own terms. Lambda handles well over a thousand concurrent executions automatically, the API layer scales alongside it and comfortably handles ten thousand or more requests per second, and object storage scales with demand far beyond anything the system currently needs. The database remains the one part that needs active attention as volume grows, so I lean on connection pooling now and keep read replicas as an option once write volume actually calls for it.

Every part of this reflects a fairly ordinary engineering philosophy: keep the feature set narrow, keep the API as the single way to interact with the system, let each layer scale on its own terms, and add complexity only once a real need justifies it. I could have picked an off the shelf asset management product instead of building this myself, and for plenty of teams that remains the right call. My reasoning came down to fit. Off the shelf tools tend to carry a wide range of features built for a wide range of customers, and a lot of that surface area goes unused while still adding weight to the interface and the price. Building ibbe storage myself meant I got to keep exactly what I needed: reliable uploads, clear organization, fast delivery, and a simple way to manage access, with room to add more only when a real need shows up. It also stays internal by design, restricted to admin created accounts rather than open public signup, which fits a tool meant to serve my own products rather than the public.

ibbe storage keeps evolving from here. Video processing, versioning for assets that change over time, multi region support, additional storage providers, better search, asset tagging, and webhooks are all on the list for later, each one added only once a real workflow needs it rather than because it sounds impressive on paper.

For now, the current version already handles the daily work of storing and serving files across ibbe, and that was the actual goal from day one: a tool that stays out of the way and lets me focus on the content itself instead of the plumbing underneath it.