By Team Imran ·
· 12 min read
Blob and Object Storage: Storing Files at Scale
Object storage explained: the flat key-space and HTTP model, blob vs object naming, object vs block vs file storage, the s3 style of buckets and keys, and when data outgrows the database and becomes files.
The Databases cluster left two pointers for this article. Database replication called the file tier “the place where objects are their own unit of copying”; a replica streams the bytes you ask it to, and the design question is what should be a row and what should be an object. Database sharding put the same boundary in growth terms: “when the data outgrows the database and becomes files.” The data-model decision pointed here for context from its side; the file tier is not a NoSQL type, it is the tier beside the database, and the two compose by reference.
Object storage is a storage model in which data is stored as objects: the bytes, a blob of user-defined metadata, and a unique key inside a flat namespace, addressed over HTTP: put an object, get an object, delete an object, list the keys. There is no hierarchy the storage enforces, no in-place modification, and no limit that matters: the service is built so that a bucket holds effectively unbounded objects, and scale is the provider’s problem rather than the architect’s.
The object: bytes, metadata, and a key
The anatomy is worth naming precisely, because every property of the tier follows from it. An object is its bytes, immutable once written; there is no in-place edit, only a new version replacing the old, plus a block of user-defined metadata (content type, cache directives, application tags like the owning entity’s ID), plus a unique key inside its bucket. The bucket is the namespace and the policy boundary: access rules, quotas, and encryption settings attach to it, and the API is the whole interface: put an object, get an object, delete an object, list keys. Range reads exist at the HTTP layer, but modification never does: the object model is write-once-then-replace.
What is deliberately absent explains what is present. No enforced hierarchy: keys are flat strings, and slashes are a naming convention the UIs render as folders. No directory walk: listing is a flat, prefix-filtered listing with a continuation token, built for a billion keys, not for a tree traversal. No locks, no byte-level writes, no partial updates. Each absence removes a coordination problem, and removing coordination is the scaling strategy: a flat namespace of immutable objects can shard trivially, replicate cheaply, and grow without the metadata service that a real file system would need; the same trade this cluster keeps meeting, taken at the storage layer.
Durability is the tier’s other promise, and its mechanics explain the price advantage. Object stores keep data alive with redundancy across failure domains; replication between systems, or erasure coding, which splits each object into data and parity fragments spread across many independent disks so that several can be lost at once without losing the object. It is the same idea replication applies to rows, taken at the fragment level, and it is why a stored object prices far below a mounted volume: the store is built to survive hardware loss as a routine event, not an incident.
Blob vs object: the same model, two names
Blob storage vs object storage is a naming question, and the honest answer is short: they are the same model. “Blob” (binary large object) is the older term, inherited from the relational world, and the cloud vendors split on vocabulary: one provider’s product line says blob, the de facto standard says object. Both mean flat namespaces, immutable data, metadata, and HTTP. The differences that matter are product differences (storage tiers, lifecycle rules, pricing, regions) and they differ per vendor, not per word: the decision between two object stores is a procurement decision, not an architectural one.
The word blob does carry one real ambiguity worth disarming, because it appears in two tiers of the stack. Inside a database, a blob is a large column value: bytes stored in a row, replicated with the table, backed up with the database. In the file tier, a blob is an object: bytes in a bucket, addressed by key. The two are related only by size, and the section at the end of this article is about when the first one should become the second. Keeping the tiers straight is the vocabulary win; the naming overlap is history, not a signal.
Where the naming does matter is translation. Cloud APIs differ per vendor even on the same model; the de facto standard’s bucket is one provider’s container, its terminology maps cleanly once you know the model is shared, and switching SDKs is the whole of the migration. Reading a design document that says blob, or a job description that says object storage, the model is the same and only the client library changes; the reverse trap is assuming the model because the word matches; a legacy file server’s blob archive is not this tier, and the context that disambiguates is which API sits in front of the bytes.
Object vs block vs file: the three storage shapes
Object storage vs block storage is the comparison that orients the whole tier, and it needs the third shape to make sense. Block storage sells raw volumes; fixed-size blocks addressed by offset, mounted by an operating system, and holding whatever the file system or database writes there; it is the disk under a database, optimized for low-latency reads and writes at small granularities. File storage sells a hierarchy (directories, inodes, POSIX semantics) the shared drive. Object storage sells none of that structure and all of the scale:
| Shape | What it looks like | How you talk to it | What it is best at |
|---|---|---|---|
| Block | Raw volume, addressed by offset | Mounted; the OS or database owns the layout | Latency-sensitive work: databases, boot disks, anything transactional |
| File | Directory tree with POSIX semantics | File system calls: open, read, write, lock | Shared access for humans and legacy apps: NAS, home directories |
| Object | Flat keys over immutable data plus metadata | HTTP; put, get, delete, list | Scale, durability, and distribution: files, media, backups, data lakes |
The sorting rule falls out of the table: the database lives on block storage, because transactions need small-granularity, low-latency, overwritable writes; the archive lives in object storage, because scale and immutability are the requirements; and the file server lives in between, holding on to structure because humans still browse. The shapes compose (a database on block storage whose rows reference objects in a bucket is the standard modern pattern) and the composition is the subject of the last section.
The performance shapes differ as much as the interfaces. Block storage answers in small, fast units (the random reads and overwrites a transaction needs) which is why the database cannot live on objects: an object GET is a whole-object operation at network latency, tens of milliseconds, not a page read at microseconds. Object storage inverts the budget: cheap per byte, slow per small operation, excellent per batch; the throughput orientation that makes it right for files and archives and wrong for indexes and transaction logs. Choosing by latency shape and not by price alone is the honest version of the block-vs-object decision.
The S3 style: buckets, keys, and operations
The S3 style became the de facto standard, and its vocabulary is worth learning once. A bucket is the namespace and policy boundary. A key is any string you choose, orders/2026/42/receipt.pdf, and the slashes are yours, not the store’s; they make listings and prefix queries behave like folders without the store enforcing a tree. The operations are the HTTP verbs: PUT an object, GET it back, DELETE it, and LIST keys by prefix with a continuation token for pagination. Every object carries its metadata and an entity tag for versioned conditional access.
Two capabilities complete the operational picture. Versioning turns the delete into a soft delete (a new version is the default, and an old version is recoverable) which is the tier’s answer to the immutability it enforces. And lifecycle rules automate age: transition to colder, cheaper tiers after days, expire after years, and abort incomplete uploads so failed transfers do not linger as billing. Neither is S3-specific; both are the shape the whole ecosystem adopted, because the interface (not any one provider) is what the applications were written against.
One piece of history is worth disarming because older material still teaches it: early object stores were eventually consistent (a write followed immediately by a read could miss) and the modern S3-style stores made read-after-write the standard for object operations. The design assumption that survives is narrower: the tier is consistent about its own namespace, and an application composing a bucket with other systems still owns the cross-system coordination. Treat the store as strongly consistent for its objects and as a participant in eventual consistency everywhere else.
Key design deserves the same care schema design gets, because the key is the only structure the store enforces. Entity-prefixed keys (tenant/42/orders/2026/) make the prefix listing a real query; versioned keys make replacement safe; and a stable, documented key convention is what keeps a bucket navigable after the team that named the first object has left. The anti-patterns are file-system habits transplanted into a store with no tree: meaning encoded in undocumented slashes, state encoded in names, and an assumption that the UI’s folder view is the store’s structure.
When data outgrows the database
The pattern this article was promised for is the boundary between the tiers: rows in the database, bytes in the bucket, a key in the row pointing at the object. The reason the boundary exists is what a blob in a row costs at scale. Big values evict the buffer pool’s hot pages, so one image-heavy table slows every other table’s working set: the memory argument database indexing made against scans, restated at the row level. Every write flows through the log, and every replica streams it: storing bytes in rows means replication copies the files with the table, backup sizes grow with the media library, and the database pays its durability tax on data that is already immutable. Objects, being their own unit of copying, copy once, into the tier built for copying them.
The discipline of the pattern is ordering, and the asymmetry makes one order safe. Upload the object first, commit the referencing row second: a committed row pointing at a missing key is a broken read, but an uploaded object with no row is an orphan that a scheduled cleanup can collect. Keys should embed the owning entity and a version, orders/2026/42/receipt-v2.pdf, so re-uploads replace content without overwriting history, and so the row’s reference can name exactly what to fetch. The read side pays a two-fetch cost (the row, then the object) and the standard answers to it are caching, presigned URLs that send the client straight to the store for a limited window, and edge placement: public objects behind a CDN are HTTP caching applied to the file tier, with the same TTL and invalidation vocabulary the caching series built.
Large objects get two operational patterns of their own. Multipart upload splits a big write into independent parts, uploads them in parallel, and assembles them server-side, which turns a flaky, hours-long transfer into a set of restartable pieces, and is how multi-gigabyte objects are written at all. And the presigned pattern points both directions: a client can upload directly to the bucket for a limited window, keeping the application out of the byte path entirely. The tier scales best when applications move keys and the network moves bytes.
The tier boundary is also the data-model decision this cluster links here for: the file tier is not a fourth NoSQL type, it is the tier beside the database, and the two compose by reference rather than by containment. The composite’s failure mode is forgetting the ordering rule (rows committed before uploads, orphans accumulating silently) and its success mode is boring: a database whose tables stay lean because its bytes live where bytes belong.
Security in the tier is configuration, and the configuration is the failure story. Buckets are private by default in the modern era precisely because the public-by-misconfiguration breach became a genre: a policy changed to unblock one integration, a listing endpoint left world-readable, data enumerated by anyone who guessed the pattern. The discipline is least-privilege credentials, per-application keys, and a policy review that treats an open bucket as an outage in waiting, because it is.
FAQ
What is object storage?
A storage model of flat namespaces and immutable data: each object is its bytes, user-defined metadata, and a unique key, addressed over HTTP: put, get, delete, list. The absences (no hierarchy, no locks, no in-place edits) are the design: removing coordination is how the tier scales to unbounded key counts. The model is the S3 style: buckets, keys, and HTTP, standardized across every major provider.
What is the difference between blob storage and object storage?
Nothing architectural: the same model under two names, inherited from different vendors’ vocabulary. “Blob” came from the relational world and one provider’s branding; “object” won the wider ecosystem. The differences that matter (tiers, lifecycle, pricing) are product features, not model features, so choosing between them is procurement, not architecture.
When should you use object storage instead of block storage?
When the workload is files at scale (uploads, media, exports, backups) and the access is whole-object reads and writes. Block storage stays under the database, because transactions need offset-addressable, low-latency, overwritable volumes. The two compose: a database on block storage whose rows reference objects in a bucket is the standard pattern.
Can you update part of an object?
No; objects are immutable. Range reads exist at the HTTP layer, but a modification means writing a new version of the whole object and updating the reference. That immutability is a feature: it is what makes versioning, cheap replication, and trivial scaling possible.
Should files be stored in the database or in object storage?
Bytes that are small enough to be negligible in the row can stay inline; anything that would burden the buffer pool, the log, the replicas, and the backups belongs in the bucket with a key in the row. The test is what the bytes cost on every write, every copy, and every restore, and the file tier prices all three near zero.
Why is object storage cheaper than block storage?
Because it is built to different tolerances: throughput-oriented hardware, redundancy by erasure coding across commodity disks, and no small-write latency to optimize. A block volume answers a random small read at disk latency; an object store answers sustained whole-object traffic at network latency, and prices accordingly. The costs match the shapes; pay for speed where the workload needs it, and for scale where it does not.
Related articles
- Next read: search engines in system design; the store that reads this tier’s documents: indexing pipelines, the consistency question, and when search earns a system of its own.
- SQL vs NoSQL: the data-model decision that precedes the file tier, and the polyglot rule this tier composes under.
- database replication; objects are their own unit of copying: the boundary this article drew.
- database sharding, rows split across shards; objects don’t: the tier that scales beside the database. The two tiers compose: shards for the rows, buckets for the bytes.
- how a CDN works, objects at the edge: HTTP caching composed with the file tier. The cache treats every object as immutable, which is what made it cacheable in the first place.
