If you run a Spring Boot application at any meaningful scale, you almost certainly have a Redis cluster behind it — powering @Cacheable, session storage, or rate-limiting. Redis is excellent. It is also expensive, because it lives entirely in DRAM. At small dataset sizes that cost is invisible. Past 50 GB it starts to sting. Past 200 GB it dominates your infrastructure bill.

EloqKV is a Redis-protocol-compatible distributed database that uses NVMe SSDs and tiered object storage instead of pure DRAM. In its cache mode (WAL disabled), it achieves throughput and P99 latency comparable to Redis — while serving datasets far larger than available RAM at a fraction of the cost.

This post documents the benchmark methodology, results, and cost analysis we ran for our own Spring services. All benchmark code is published and reproducible.

Setup​

We wanted the benchmark to reflect a real Spring production workload, not a synthetic redis-benchmark run. Everything runs through Spring Data Redis — the same RedisTemplate, the same @Cacheable abstraction, the same Lettuce connection pool that any Spring application would use.

Both backends share the same Spring application configuration. The only difference between runs is the spring.data.redis.host property.

Spring Application Code​

This is the critical point: no application code was modified. EloqKV speaks the Redis Serialization Protocol (RESP), so Spring Data Redis connects to it identically to Redis.

The cacheable service​

@Service
public class UserProfileService {

    private final UserRepository repo;
    private final RedisTemplate<String, UserProfile> redis;

    @Cacheable(value = "profiles", key = "#userId")
    public UserProfile getProfile(String userId) {
        // Cache miss — goes to database
        return repo.findById(userId).orElseThrow();
    }

    public void warmCache(String userId, UserProfile profile) {
        redis.opsForValue().set(
            "profiles::" + userId,
            profile,
            Duration.ofMinutes(30)
        );
    }
}

Switching backends — the entire migration​

The Redis config:

spring:
  data:
    redis:
      host: redis-cluster.internal
      port: 6379
      lettuce:
        pool:
          max-active: 32

The EloqKV config (one line changes):

spring:
  data:
    redis:
      host: eloqkv.internal  # ← only change
      port: 6379
      lettuce:
        pool:
          max-active: 32

JMH benchmark harness​

@BenchmarkMode({Mode.Throughput, Mode.AverageTime})
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Thread)
@Warmup(iterations = 3, time = 5)
@Measurement(iterations = 5, time = 10)
@Fork(1)
public class EloqKVBenchmark {

    // Lettuce synchronous API — one connection per JMH thread
    private RedisCommands<String, String> commands;
    private StatefulRedisConnection<String, String> connection;

    @Setup(Level.Trial)
    public void setup() {
        RedisClient client = RedisClient.create(
            RedisURI.builder().withHost("10.128.15.205").withPort(6379).build()
        );
        connection = client.connect();
        commands   = connection.sync();
        // pre-populate read targets
        commands.set(testKey, "benchmark-value");
        commands.hmset(hashKey, tenFields);
    }

    @Benchmark public void stringGet(Blackhole bh)  { bh.consume(commands.get(testKey)); }
    @Benchmark public void stringSet(Blackhole bh)  { commands.set("jmh:w:" + UUID.randomUUID(), "v"); }
    @Benchmark public void stringIncr(Blackhole bh) { bh.consume(commands.incr("jmh:counter")); }
    @Benchmark public void stringSetex(Blackhole bh){ bh.consume(commands.setex("jmh:ex:" + UUID.randomUUID(), 60, "v")); }

    @Benchmark public void hashHget(Blackhole bh)    { bh.consume(commands.hget(hashKey, "field0")); }
    @Benchmark public void hashHset(Blackhole bh)    { bh.consume(commands.hset(hashKey, "f", "v")); }
    @Benchmark public void hashHgetall(Blackhole bh) { bh.consume(commands.hgetall(hashKey)); }

    @Benchmark public void zsetZadd(Blackhole bh)  { bh.consume(commands.zadd(zsetKey, Math.random() * 1000, UUID.randomUUID().toString())); }
    @Benchmark public void zsetZrange(Blackhole bh) { bh.consume(commands.zrange(zsetKey, 0, 9)); }
    @Benchmark public void zsetZrank(Blackhole bh)  { bh.consume(commands.zrank(zsetKey, "member0")); }

    @Benchmark
    public void pipelinedWrites(Blackhole bh) throws Exception {
        var async = connection.async();
        var futures = new ArrayList<>();
        String prefix = "jmh:pipe:" + UUID.randomUUID() + ":";
        for (int i = 0; i < 10; i++) futures.add(async.set(prefix + i, "v" + i));
        for (var f : futures) bh.consume(((CompletableFuture<?>) f).get());
    }
}

Throughput Results​

JMH measured single-connection synchronous throughput via the Lettuce client. A single Lettuce connection sustains 7,538–8,120 ops/s across all command types with remarkable consistency. Pipelined writes (10 async SETs per round-trip) yield 4,163 pipeline-ops/s — equivalent to 41,630 effective SET ops/s over a single connection. In production, a pool of 50 Lettuce connections projects to >390,000 ops/s aggregate.

JMH throughput — ops/s per connection (single-thread, synchronous Lettuce, higher is better)

Throughput is nearly flat across the entire command set — a 7.8% spread from SADD (7,538 ops/s) to ZRANGE (8,120 ops/s). This reflects EloqKV's uniform NVMe-backed storage path: there is no "slow command" outlier. The pipelined benchmark shows the Lettuce async API eliminating round-trip overhead, multiplying effective write throughput by 10× over the same single connection.

Latency Results​

We measured latency at two layers. At the protocol level, JMH's AverageTime mode reports how long each synchronous Lettuce call takes end-to-end. At the application layer, Gatling drove load through Spring Boot's REST endpoints (HTTP → Spring MVC → RedisTemplate → Lettuce → EloqKV), capturing the P50/P95/P99 a real client would see.

JMH — protocol-level average latency (ms/op)​

Gatling — HTTP end-to-end latency through Spring Boot REST​

Mixed load (SET/GET, HSET/HGET, ZADD scenarios) at ~95 requests/s, all routed through RedisTemplate in a Spring Boot REST layer. 3,790 requests, zero failures.

The HTTP layer adds roughly 0.9 ms median overhead vs. the raw protocol (0.116–0.133 ms JMH avg). Even so, P99 stays at 3 ms — well within the budget for any cache tier. Crucially, as dataset size grows beyond available DRAM, EloqKV spills to NVMe without observable latency degradation, while a pure-DRAM Redis deployment would begin evicting or run out of memory entirely.

Cost Analysis​

Throughput and latency tell half the story. The more important question for most engineering teams is: what does this cost to run?

Redis's fundamental constraint is that every byte of your dataset must live in RAM. On AWS, cache.r6g instances — the recommended family for ElastiCache — price DRAM at roughly $0.16–0.20 per GB per month. NVMe SSD on i3en instances costs roughly $0.015 per GB per month — more than 10× cheaper per byte of storage.

EloqKV exploits this gap by keeping only the hot fraction of your dataset in RAM and serving the rest from NVMe with sub-5 ms P99.9 latency — performance that was previously impossible on spinning or NAND-flash storage.

Scenario: 100 GB working dataset, production HA​

13.1× cost reduction​

$2,178 → $166 / month for a 100 GB dataset

How cost savings scale with dataset size​

The crossover point is around 20–30 GB. Below that, ElastiCache's managed convenience may justify the premium. Above 30 GB, EloqKV's cost profile is structurally superior — and the gap widens monotonically with dataset size because the underlying cost structure is different: EloqKV's compute cost is fixed while Redis's cost grows linearly with data volume.

The Migration Path​

Because EloqKV fully implements the Redis Serialization Protocol (RESP2 and RESP3), Spring Data Redis talks to it identically. There are no compatibility shims, no protocol adapters, no changes to serializers.

Commands used by Spring that we verified work identically include: GET, SET (with PX/EX), DEL, SCAN, HGET/HSET/HMGET, SADD/SMEMBERS, ZADD/ZRANGE, EXPIRE, TTL, MULTI/EXEC, and Lua scripting via EVAL.

Spring Cache configuration (unchanged)​

@Configuration
@EnableCaching
public class CacheConfig {

    @Bean
    public RedisCacheManager cacheManager(RedisConnectionFactory factory) {
        RedisCacheConfiguration config = RedisCacheConfiguration.defaultCacheConfig()
            .entryTtl(Duration.ofMinutes(30))
            .disableCachingNullValues()
            .serializeValuesWith(
                RedisSerializationContext.SerializationPair
                    .fromSerializer(new GenericJackson2JsonRedisSerializer())
            );

        return RedisCacheManager.builder(factory)
            .cacheDefaults(config)
            .build();
        // This bean works identically with Redis or EloqKV —
        // RedisConnectionFactory is provided by Spring Boot autoconfiguration
    }
}

Recommended migration checklist​

Conclusion​

EloqKV is a drop-in replacement for Redis in the Spring ecosystem. The benchmark shows comparable or better throughput and P99 latency at equivalent dataset sizes, with the critical advantage that EloqKV is not bounded by available DRAM.

For datasets under 30 GB, Redis on ElastiCache remains a reasonable choice if managed convenience is a priority. For anything larger, the cost math is unambiguous: EloqKV saves 10× or more on infrastructure spend while delivering the same sub-millisecond P50 latency your Spring services require.

Critically, the migration cost is near-zero. Spring Data Redis is fully protocol-abstracted — your @Cacheable annotations, your RedisTemplate wiring, and your cache configuration are all unchanged. The only edit is a hostname in a YAML file.

At 100 GB our team saved $24,144 per year on a single caching tier. At 500 GB, that number exceeds $127,000 per year — enough to fund engineering headcount.

JMH results: single-thread synchronous Lettuce client, 3 warmup × 5 s + 5 measurement × 10 s, 1 fork, GCP us-central1-a, Ubuntu 24.04, Java 21.0.10 (OpenJDK). Gatling results: mixed REST scenario (SET/GET, HSET/HGET, ZADD) at ~95 req/s, 3,790 total requests, 0 failures. AWS pricing as of May 2026, us-east-1 on-demand rates. EloqKV tested in cache mode (WAL disabled) to match the semantics of a Redis deployment without persistence. For durable workloads with WAL enabled, EloqKV provides full ACID transactions with some throughput reduction; see EloqData's persistence benchmarks for details.

In the world of high-performance data, many engineering teams hit a "memory wall." As data sets grow, the cost of keeping every single byte in DRAM becomes the primary bottleneck for scaling. This is particularly painful for memory-bound applications, where the performance requirements demand Redis-like speeds, but the sheer volume of data makes the cloud bills unsustainable.

Moving from Redis to EloqKV allows you to shift from expensive, memory-heavy instances to cost-efficient, SSD-optimized infrastructure—all while maintaining the extreme low latency your application requires.

Quick answer: Migrate from Redis to EloqKV with zero downtime using RedisShake. Mirror your data into EloqKV (full + incremental sync), divert read traffic and validate latency, then cut over writes and decommission Redis. The full checklist and a compatibility-validation step are below.

Migration checklist​

Use this checklist to run the migration end to end. Each later section expands a step.

Before you start

• Inventory the commands and data types your app uses, and confirm EloqKV command and client compatibility.
• Provision the target EloqKV cluster and confirm TLS, auth, and cluster mode.
• Define your latency SLO (for example, a P99 GET target) and set up monitoring on both systems.

Stage 1 — Mirror data

• Deploy RedisShake with the shake.toml below; run a full sync, then leave incremental sync running.
• Confirm replication lag is minimal and stable.

Stage 2 — Divert reads

• Point read clients at EloqKV and verify P99 latency meets your SLO under real traffic.

Stage 3 — Cut over writes

• Set Redis read-only / pause ingestion, verify the final incremental batch has shipped, repoint writers to EloqKV, then decommission Redis.

The Migration Driver: Escaping the "DRAM Tax"​

For memory-bound use cases—such as a Feature Store for recommendation systems—the challenge isn't necessarily request volume, but the sheer footprint of user profiles and item embeddings. In a standard Redis setup, you are forced to pay for peak DRAM capacity, even if much of that data isn't accessed every second.

EloqKV breaks this linear cost curve. By utilizing a sophisticated storage engine optimized for NVMe SSDs, it delivers the speed of an in-memory database at the price point of disk storage. This migration doesn't just save money; it allows your business to scale its data footprint without a proportional increase in infrastructure spend.

Validate compatibility before cutover​

EloqKV is Redis- and Valkey-compatible, but treat the migration as a validated cutover rather than a drop-in swap. Confirm each of the following before you move write traffic:

• Commands and data types: Verify the commands and types your application uses are supported. Review the command compatibility reference. In the shake.toml below we block command groups EloqKV does not target (STREAM, GEO, HYPERLOGLOG) and admin commands (SELECT, FLUSHALL, FLUSHDB) so the sync never ships them.
• Clients and libraries: Most Redis clients work unchanged. Confirm your driver and version against the client compatibility reference, paying attention to cluster-mode clients.
• Cluster behavior: The target runs in cluster mode (cluster = true). Validate key distribution, multi-key operations, and how your client handles MOVED and ASK redirects.
• Persistence settings: Decide between cache mode and WAL-backed durability for the target, and make sure it matches your recovery requirements. This choice is independent of Redis's own RDB/AOF settings.
• Latency SLOs: Define an explicit target (for example, P99 GET latency) and monitor it on EloqKV during Stage 2 before shifting writes. See the EloqKV on EloqStore benchmark for reference tail-latency numbers.

The 3-Stage Migration Plan​

To ensure a seamless transition with zero downtime, we use RedisShake, a versatile tool that treats EloqKV as a replica of your existing Redis cluster.

Stage 1: Full and Incremental Synchronization​

The goal of Stage 1 is to mirror your Redis data onto EloqKV without affecting your production environment. RedisShake connects to your source Redis, performs a "Full" sync of the current dataset, and then switches to "Incremental" mode to ship every new write to EloqKV in real-time.

Configuration: shake.toml​

To bridge the two systems, deploy RedisShake with the following configuration. This setup treats EloqKV as the target writer for all incoming Redis data.

[sync_reader]
cluster = false
address = "redis-production.example.com:6379"
password = "your_source_password"
tls = true
sync_rdb = true # set to false if you don't want to sync rdb
sync_aof = true # set to false if you don't want to sync aof

[redis_writer]
cluster = true
address = "eloqkv-cluster.example.com:6379"
password = "your_target_password"
tls = true
off_reply = false

[filter]
block_command = ["SELECT", "FLUSHALL", "FLUSHDB"]
block_command_group = ["STREAM","GEO","HYPERLOGLOG"]

[advanced]
dir             = "/data"
ncpu            = 1
status_port     = 8084
pprof_port      = 9094

log_file        = "/data/shake.log"
log_level       = "info"
log_interval    = 5
log_rotation    = true
log_max_size    = 512
log_max_age     = 7
log_max_backups = 3
log_compress    = true

rdb_restore_command_behavior  = "rewrite"
pipeline_count_limit          = 512
target_redis_max_qps          = 30000
empty_db_before_sync          = false

target_redis_oom_requeue = true
target_redis_oom_requeue_max_times = 3
target_redis_oom_requeue_delay_ms = 500

io_reconnect = true
io_reconnect_max_times = 100
io_reconnect_delay_ms = 10000

target_redis_writer_shards=4
target_redis_proto_max_bulk_len = 512_000_000

Stage 2: Diverting Read Traffic​

Once the synchronization is stable and replication lag is minimal (usually measured in microseconds), you can begin utilizing EloqKV.

Because recommendation systems are heavily dependent on fast lookups, we switch the Read Traffic first. By pointing your application’s read clients to EloqKV, you can verify that the SSD-based architecture meets your latency requirements under real-world conditions.

Pro Tip: Monitor your p99 latencies during this stage. Most users find that EloqKV’s SSD performance is indistinguishable from Redis DRAM performance for feature lookups, but at a fraction of the cost.

Stage 3: The Final Cutover (Write Traffic)​

With reads successfully validated on EloqKV, it is time to move the "Source of Truth."

1. Stop Ingestion to Redis: Briefly halt your data pipelines or set the Redis source to read-only.
2. Verify Buffer Flush: Ensure RedisShake has finished shipping the final set of incremental changes.
3. Point Writes to EloqKV: Update your ingestion workers (e.g., Flink, Spark, or custom API workers) to write directly to the EloqKV endpoint.
4. Decommission: Once the write path is stable, you can safely spin down the expensive, memory-bloated Redis instances.

Conclusion: Economics at Scale​

By migrating your memory-bound Feature Store to EloqKV, you effectively decouple your data growth from your DRAM budget. You gain the ability to store 10x the features on the same budget, providing your recommendation models with more context and your business with a lower TCO (Total Cost of Ownership).

The transition is low-risk, the performance remains "extreme," and the SSD-based cost model finally makes large-scale data sets sustainable.

Try it yourself​

• Product overview: EloqKV product page
• Estimate savings: Cost calculator and the Redis vs EloqKV cost breakdown
• Latency proof: EloqKV on EloqStore benchmark

Frequently Asked Questions​

Is there downtime during the migration?​

No. RedisShake mirrors your data into EloqKV with a full sync followed by continuous incremental sync, so reads and writes keep serving from Redis until you choose to divert them. The only brief pause is the final write cutover in Stage 3.

Which Redis commands and data types are not supported?​

EloqKV targets core key-value and common structures, not STREAM, GEO, or HYPERLOGLOG, which the sample shake.toml blocks from sync. Check your command and type usage against the command compatibility reference before migrating.

Do my existing Redis clients still work?​

Most Redis and Valkey clients work without changes because EloqKV is wire-compatible. Confirm your specific driver and cluster-mode behavior against the client compatibility reference.

How do I validate latency before moving writes?​

Define a latency SLO such as a P99 GET target, divert read traffic to EloqKV in Stage 2, and monitor P99 and P99.99 under real load before cutover. The benchmark article shows reference tail-latency numbers on NVMe.

How do I roll back if something goes wrong?​

Until the Stage 3 write cutover, Redis remains the source of truth, so rolling back is simply leaving reads and writes on Redis. Keep Redis running until EloqKV has served production write traffic and met your SLOs, then decommission it.

However, the definition of "scale" is shifting under our feet.

On January 22nd at the Unlocked Conf (shout out to Khawaja and Daniela at Momento and always smile Madelyn for hosting such an incredible event!), the industry saw what "real" scale looks like today. Experts from Uber (Yang Yang and Shawn Wang), AWS (Kevin and Sarthak), and Apple (Yiwen Zhang) shared mind-boggling numbers. Most notably, Uber is now handling 1 billion RPS using their cache layer. In that context, "millions of QPS" is just the starting line.

From 800 Million Users to 800 Billion Agents​

OpenAI’s current architecture of 50 read replicas is a masterpiece of efficiency for 800 million humans. But we are rapidly approaching the Agentic Era. We aren't just building for humans who type a few prompts a day; we are building for 800 billion autonomous agents that never sleep, never stop querying, and never stop generating data.

When we talk about a Trillion QPS in a major AI company, we have to ask: Can PostgreSQL read replicas actually survive?

Even with cascading WAL shipping, the infrastructure cost of 50,000+ replicas is a budgetary black hole. More importantly, the replication lag and the CPU overhead of a traditional RDBMS engine make it the wrong tool for the "Trillion QPS" future.

The "Cache Miss" Vulnerability​

OpenAI’s post highlights a critical defense: a cache locking and leasing mechanism to prevent "thundering herds" from crashing PostgreSQL during a cache-miss storm. It’s a brilliant safety net, but it proves that the database is a liability in a high-velocity environment.

Our question is simple: Why not just cache all the data?

Enter EloqKV: The "Full Cache" Reality​

OpenAI mentions that their cache only holds "hot data" because keeping everything in memory is too expensive. In the traditional Redis world, they are right. Keeping 800 million users' (or 800 billion agents) session data in RAM is a waste of capital.

EloqKV changes the math. By building on EloqStore, we’ve created a Redis-compatible KV which holds all your data on NVMe SSDs, creating a "Full Cache" architecture where cache misses simply don't exist.

• No More Cache Miss: By making the "cache" with full data as high-performance layer for agent state, you eliminate the risk of a "burst of misses" crashing a back-end database.
• NVMe Performance at Scale: On a 16vCore GCP instance, EloqKV delivers sub-millisecond P99 latency and under 2 millisecond P9999 latency at 100K QPS. Need more? A single 44vCore machine can push near 1 million QPS.
• 10x Cost Savings: PostgreSQL is great for relations, but for the raw key-value lookups required by 800 billion agents, it’s 10x more expensive than an SSD-optimized KV store like EloqKV.

How does EloqKV on EloqStore achieve it? Read our Tech: EloqKV on EloqStore Introdution

But What About Joins?​

The classic rebuttal to KV stores is the lack of relational joins. However, as OpenAI themselves noted, they are already moving away from 12-table joins and toward application-layer logic.

We believe in the DynamoDB Philosophy: Design your data schema based on your access patterns. If you need predictable, single-digit millisecond performance at any scale, you denormalize.

The problem has always been that DynamoDB is prohibitively expensive at high QPS. If you’ve ever looked at a bill for 1 million provisioned WCUs/RCUs, you know the pain. EloqKV brings that same philosophy to your own infrastructure, making a Trillion QPS future not just technically possible, but economically affordable.

Looking Forward​

We agree with OpenAI that PostgreSQL is an incredible tool. But for the next generation of agent-generated traffic—where QPS will reach billions—the "Primary + 50k Replicas + Hot Cache" model will be too slow and too expensive.

The future belongs to Full Cache architectures that treat SSDs like memory and eliminate the database bottleneck entirely.

Ready to scale for the Agentic Era?

• Read the Tech: EloqKV on EloqStore
• Explore the Source Code: EloqStore Repo & EloqKV Repo
• Try it Now: Install EloqKV in One Minute

At CES 2026, NVIDIA CEO Jensen Huang delivered a stark warning: the industry is facing a critical shortage of DRAM. While the explosive growth of AI models is the primary driver, there is another massive consumer of memory that often flies under the radar: Caching Services.

Traditionally, caching services like Redis and Valkey are purely memory-based. Even though people have tried to leverage fast SSDs for caching (e.g. Apache KVRocks), for latency-sensitive workloads, DRAM-based solutions remained the only viable solution because SSD-based alternatives often have significant tail latency issues. In mission-critical environments, a latency spike can easily disrupt real-time workflows and render a service unresponsive. Until recently, how to tame tail latency for IO intensive workloads has remained an unsolved challenge.

Quick answer: EloqKV on EloqStore serves Redis-style workloads from NVMe SSD while holding P99.99 latency to a few milliseconds — on datasets up to 2TB, where a DRAM-only Redis node cannot fit the data at all. You get memory-like tail latency at disk-like cost, with the 20X node reduction detailed below. See the EloqKV product page and estimate your own savings with the cost calculator.

Today, we're proud to introduce EloqKV powered by EloqStore. By leveraging the innovation of a modern data storage engine, EloqKV is the first Redis-compatible KV store designed to deliver DRAM-level P9999 latency with modern NVMe SSDs. By eliminating the performance tail that plagues traditional systems, EloqKV, when powered by EloqStore, offers up to 20X cost savings without sacrificing the millisecond-level latency your business demands.

As discussed in our previous blogs, EloqKV is not just a cache. It can work as a full persistent main data storage. In this blog, we re-examine the cache usage case and we will leave the full persistent evaluation in another post. In all our tests, we turn off the write ahead log (WAL) to relax the strong durability guarantee.

Traditional DRAM-based systems like Redis are strictly limited by physical memory capacity. As shown in the benchmark above, while Redis is restricted to smaller datasets, EloqKV breaks this barrier by leveraging NVMe without the "tail-latency penalty" typically associated with disks. In fact, at 20GB and 100GB, EloqKV actually provides higher throughput than Redis while maintaining a near identical P99.99 latency profile. When data size exceeds main memory size, Redis fails while EloqKV continues to scale. Even with a 2TB dataset, EloqKV maintains a stable P99.99 latency of a few milliseconds, a performance previously thought possible only in pure DRAM environments.

For comparison, we also tried KVRocks, a solution that supports SSD serving. As shown below, the P99.99 latency grows out of control as IO intensive operations interfere with the read latencies (notice the log scale).

Benchmark conditions​

These results reflect one specific setup. We share it in full so you can judge how well it maps to your own workload.

• Hardware: Single node, GCP Z3-16 instance — 16 vCore, 128GB RAM, 2 × 2.9TB NVMe SSD.
• Dataset sizes: 20GB, 100GB, 200GB, and 2TB (up to ~800 million KV pairs).
• Payload: Variable value sizes of 1–4KB.
• Read/write mixes: 95:5 (read-dominant), 50:50 (balanced), and 5:95 (write-dominant). The scaling table below uses the read-dominant (GET) workload; the balanced and write-heavy mixes appear in the charts above.
• Persistence mode: Write-ahead log (WAL) disabled — i.e. cache mode — to isolate serving latency. Durable, WAL-backed performance is evaluated separately in ACID in EloqKV: Durability.
• Benchmark tool: memtier_benchmark.
• Metric definitions: P99 is the 99th-percentile latency (slower than 99% of requests). P99.99 is the 99.99th-percentile latency — the worst 1 in 10,000 requests — the figure that exposes the tail-latency spikes SSD caches usually suffer.

The scaling test holds the read-dominant workload steady while the dataset grows. EloqKV keeps a flat P99.99 profile across all four sizes; Redis cannot hold the 200GB and 2TB datasets on this single node because they exceed its RAM.

By solving the P99.99 latency problem on SSDs, EloqKV allows businesses to scale their data footprint by orders of magnitude without sacrificing the sub-millisecond responsiveness their users demand. To manage a 2TB dataset with Redis, an organization would typically need a cluster of 20 nodes to provide enough RAM. EloqKV delivers the same "long-tail" latency reliably on a single NVMe-optimized node — a 20X reduction in nodes for this 2TB scenario.

The actual cost saving depends on your dataset size, hot working set, replica count, and durability needs, so the multiple will differ for your workload. Model your own numbers with the EloqKV cost calculator, or read the full Redis vs EloqKV cost breakdown.

The Long Tail Problem: Why SSD Caching is Hard​

At the upcoming Unlocked Conference 2026, engineers from Uber will present "Real-world cache lessons from 1B RPS in prod." This highlights the scale modern infrastructure must handle. While a standard PostgreSQL or MySQL setup works fine for low throughput workloads and moderate-sized hot datasets, it crumbles under the "Internet Scale" traffic processed by Google, Amazon, Uber, and Snap.

The challenge is compounding in the AI Age. We are moving toward a future of Agent-to-Agent communication, which will generate 100x more data and RPS than human interaction. We see this with our own customers: AI chat scenarios create massive data explosion because prompts are long, complex, and carry context payloads 100x larger than typical human chat messages.

Since DRAM is expensive, the obvious alternative is to use SSDs. However, for high-performance caching, standard SSD implementations have historically failed. The dealbreaker isn't average latency, instead, it's the long tail latency, especially when dealing with millions or billions of requests per second (RPS). When you throw this level of concurrency at a standard SSD-based database, traditional architectures fail. Relying on multi-threading and synchronous I/O simply cannot guarantee consistent low tail latency on commodity hardware. The threads block, the context switches pile up, and the tail latency spikes.

The Software-Hardware Gap​

Our research found that the issue with the unstable performance of current storage design is that the software stack, traditionally designed for hard-disk based storage systems, has not caught up with the times. Traditionally, disk is slow, and every effort should be made to increase the performance of disks. However, modern NVMe SSDs are incredibly fast, capable of delivering millions of IOPS and gigabytes per second throughput, even for random data accesses. To take advantage of such high performance disks, a few things need to be changed.

The first and most fundamental change is the overall database architecture, which we described in detail in our previous blog posts. In this blog, we discuss several implementation issues that allow us to fully exploit modern storage devices. In addition, two specific implementation techniques are also essential:

1. Coroutines: To handle massive concurrency without the overhead of OS threads.
2. io_uring: The Linux kernel interface that enables high-performance asynchronous I/O.

By combining these, we bypass the limitations of synchronous I/O. You can read more about our engineering philosophy on this topic in our deep dive: Coroutines & Async Programming.

Introducing Our Open Source EloqStore Engine​

We developed EloqStore, a modern storage engine that is optimized for and takes advantage of the Data Substrate architecture to achieve unparalleled performance and stability. We are thrilled to announce that it is now Open Source.

Check out EloqStore on GitHub

EloqData's Data Substrate technology is designed for plugable storage engines. Previously, EloqKV utilized RocksDB and RocksCloud as its main underlying storage engine. While robust, RocksDB relies on Log-Structured Merge-trees (LSM), which suffer from well-known issues that destabilize long tail latency:

• Write Amplification: Repeatedly rewriting data during compaction.
• Read Amplification: Checking multiple files to find a single key.
• Compaction Stalls: CPU spikes during background compaction tasks.

EloqStore addresses these issues head-on by abandoning the traditional Log-Structured Merge-tree (LSM) architecture in favor of a design optimized specifically for the high-parallelism nature of modern NVMe SSDs. While LSM-trees are designed to sequentialize writes for older spinning disks, EloqStore is built from the ground up to exploit the random-access strengths and massive IOPS of modern flash storage.

Deterministic Performance via B-tree Variant Indexing​

To achieve deterministic performance, EloqStore utilizes a specialized B-tree variant indexing strategy. It maintains a compact, memory-resident mapping of all non-leaf nodes, ensuring that the internal path to any piece of data is always held in DRAM.

This architecture guarantees exactly one disk access per read. By bypassing the multiple "level checks" found in RocksDB, EloqStore eliminates the read amplification that typically causes latency spikes during heavy lookups. Whether your dataset is 100GB or 10TB, a read request is always a direct, single-IOPS operation, providing the predictable latency required for mission-critical caching.

High-Throughput Batch Write Optimization​

The write path is equally optimized through Batch Write Optimization. Leveraging io_uring, EloqStore bypasses the overhead of traditional synchronous system calls, allowing the engine to group multiple incoming writes into large, aligned blocks. This approach:

• Reduces the frequency of expensive disk commits.

• Significantly lowers write amplification.

• Preserves the lifespan of the SSD while maintaining massive throughput.

Coroutines and Async I/O​

Under the hood, EloqStore is built with coroutines. This allows the system to handle thousands of concurrent requests without the heavy memory footprint or context-switching penalties of OS threads. When an I/O operation is pending, the coroutine simply yields, allowing the CPU to process other requests until the NVMe signals completion via io_uring. Coroutine-based language-level concurrency and kernel-level asynchronous I/O allow EloqKV to maintain sub-millisecond responsiveness under heavy load.

Eliminating the "Compaction Stall"​

Most importantly, this architecture eliminates the jitter caused by background maintenance. Traditional LSM-trees must periodically merge and rewrite files to maintain performance—a process that consumes massive CPU and I/O resources, leading to the dreaded "compaction stall." EloqStore's append-only design manages data reclamation more gracefully, ensuring that background tasks never interfere with foreground traffic. The result is a P9999 latency profile that stays flat, delivering a "memory-like" experience at disk-based economics.

Rethinking the Key-Value Store for the AI Era​

Beyond its performance as a cache, EloqKV is a full-featured key-value store. This architectural shift enables a new class of advanced features:

• Cloud-Native "Scale to Zero": Because data is persisted on NVMe and/or object storage rather than volatile DRAM, EloqKV can spin down to zero when idle.

• AI-Native "Quick Branching": For complex AI workflows and agent simulations, EloqKV supports rapid branching. You can fork your state to explore different prompt chains or agentic outcomes without duplicating massive DRAM footprints.

• Automatic Archiving: Archiving is no longer a separate operation and you do not need to choose a "cut-off" period. EloqKV does not even require you to have sufficient capacity to store all your data on local SSD. In fact, cold data can be seamlessly and automatically tiered to low-cost object storage such as S3. The only penalty is a few seconds of delay when accessing cold data.

Meet us at Unlocked 2026​

We are excited to announce that EloqData is a sponsor of the Unlocked Conference 2026, hosted by Momento and AWS.

Unlocked is the premier event for developers discussing the future of backend infrastructure. We are eager to discuss how the shift from DRAM to NVMe can unlock new possibilities for AI and hyperscale applications.

Come visit our booth, chat with our engineers, and see EloqStore in action. We look forward to seeing you there.

Try EloqKV​

• Product overview: EloqKV product page
• Estimate savings: Cost calculator
• Already on Redis? Follow the Redis-to-EloqKV migration guide
• Source: EloqKV and EloqStore on GitHub

Frequently Asked Questions​

How does EloqKV keep P99.99 latency low on SSD?​

EloqStore keeps every index's non-leaf nodes in DRAM, so each read is a single, direct NVMe access (one IOP) with no LSM-style level checks. It pairs this with coroutines and io_uring asynchronous I/O, and its append-only design avoids the compaction stalls that spike tail latency on other SSD stores. The result is a P99.99 of a few milliseconds even at 2TB.

What exactly were the benchmark conditions?​

A single GCP Z3-16 node (16 vCore, 128GB RAM, 2 × 2.9TB NVMe), 1–4KB values, datasets from 20GB to 2TB, read/write mixes of 95:5, 50:50, and 5:95, measured with memtier_benchmark and the write-ahead log disabled (cache mode). See the Benchmark conditions section for the full table.

Do these results require turning off durability?​

These specific numbers were measured in cache mode (WAL off) to isolate serving latency. EloqKV also supports WAL-backed durable persistence; that mode is benchmarked separately in ACID in EloqKV: Durability.

How much can EloqKV save versus Redis?​

In this 2TB test, EloqKV replaces a roughly 20-node Redis cluster with a single NVMe node. Real savings depend on your data size, hot working set, replicas, and durability requirements — estimate yours with the cost calculator.

Is EloqKV a drop-in Redis replacement?​

EloqKV is Redis- and Valkey-compatible, which keeps migration effort low, but teams should validate command coverage, cluster behavior, persistence settings, and latency SLOs before cutover. The migration guide walks through this.

MinIO has officially moved its Community Edition to a “maintenance-only” release model:

• No more prebuilt binaries from the community.
• No regular acceptance of new features or PRs.
• Only critical security patches will be handled when necessary.

In plain terms: If your production object storage is built on MinIO Community Edition, you are now carrying hidden operational risk and rising maintenance costs.

For SREs: Three Things To Do Immediately​

1. Classify your MinIO usage:

   • Mission-critical production
   • Non-critical environment
   • Test / dev only

2. For critical workloads, start a 48-hour PoC immediately.

3. Select candidates that are:

   • S3-compatible
   • Actively maintained
   • Commercially supportable (e.g., Ceph RGW, SeaweedFS, RustFS), and run compatibility regression tests.

Part I: Why This Is Not “Minor News” — Real Engineering Risks​

Many teams treat object storage as the foundation of their infrastructure: cold data, backups, logs, media, and user uploads all live there.

Unlike databases or message queues, the risk of an unmaintained object storage layer is not sudden downtime, but:

• Extended patch windows
• Growing compliance risk
• Rising operational burden

Once your S3 endpoint is public-facing and used with cross-region replication, IAM, and access control, even subtle incompatibilities in headers or signatures can become incident triggers.

Part II: Viable Alternatives in 2024–2025​

Selection criteria:

• S3-compatible
• Active releases or commercial support
• Ability to provide patches and vendor/community support

Below is a quick engineering-focused comparison:

Quick Engineering Rule of `Thumb

• Need full S3 features (versioning, lifecycle, ACLs, multipart)? → Ceph or commercial vendors first.
• Only need basic read/write? → SeaweedFS or RustFS are fast to PoC.

Part III: A 48-Hour PoC Using SeaweedFS (Copy-Paste Friendly)​

Goal: Within 48 hours, produce quantifiable results to support decisions:

• Compatibility gaps (API / headers / multipart / ACL)
• Throughput & latency (P50 / P95 / P99)
• Operational complexity (steps, recovery time, scriptability)
• Failure recovery behavior (node failure → consistency)

Recommended Time Allocation​

Environment Requirements​

• CPU: ≥4 vCPU per node (2 minimum)
• Memory: 8–16 GB per node (Ceph: 16GB+ recommended)
• Disk: 500GB–1TB per node (500GB enough for PoC)
• Network: 1Gbps or better
• OS: Ubuntu 20.04/22.04 or CentOS 8/9
• Nodes: minimum 6

Common Setup Script (Run on All Nodes)​

#!/usr/bin/env bash
sudo apt update
sudo apt-get install s3cmd

set -euo pipefail
WORKDIR="${PWD}/s3_distributed_poc"
mkdir -p "$WORKDIR"{/seaweed,/scripts,/data}

for cmd in s3cmd ; do
  if ! command -v $cmd >/dev/null 2>&1; then
    echo "Warning: $cmd not installed."
  fi
done

cat > "$WORKDIR/README.md" <<'EOF'
Directory layout:
- seaweed/: SeaweedFS deployment and benchmarks
- scripts/: General test scripts
EOF
echo "Init done"

Run:

chmod +x poc_setup_common.sh
./poc_setup_common.sh

SeaweedFS Deployment (3-Master + 3-Volume Example)​

Assumed nodes:

node1: 10.0.0.11
node2: 10.0.0.12
node3: 10.0.0.13
node4: 10.0.0.14
node5: 10.0.0.15
node6: 10.0.0.16

Create directories (run on all nodes):

mkdir -p /data/seaweedfs/{master,volume,filer,data}

Start masters on node1/2/3:

nohup ./weed master -port=9333 -mdir=/data/seaweed/master -defaultReplication=001 -ip=10.0.0.11 -peers=10.0.0.11:9333,10.0.0.12:9333,10.0.0.13:9333 >> master.log 2>&1 &

(Repeat for three nodes)

Start volume servers on node4/5/6:

nohup ./weed volume -port=9334 -dir=/data/seaweed/volume -max=30 \
  -mserver=10.0.0.11:9333,10.0.0.12:9333,10.0.0.13:9333 \
  -dataCenter=dc1 -rack=rack1 -publicUrl=10.0.0.14:9334 -ip=10.0.0.14 >> v1.log 2>&1 &

Verify:

curl http://10.0.0.11:9333

Run filer + S3 gateway:

./weed filer -s3 -ip=10.0.0.1x -master=10.0.0.11:9333,10.0.0.12:9333,10.0.0.13:9333

Benchmark Script​

The benchmark script can be downloaded at EloqData Objecrt Storage Benchmark

Benchmark with the following commands:

# generate local files
./s3_concurrent_test.sh prepare
# upload local files to object storage
./s3_concurrent_test.sh upload
# download files in object storage to local
./s3_concurrent_test.sh download
# test upload and download concurrently
./s3_concurrent_test.sh both
# cleanup object storage with prefix
./s3_concurrent_test.sh cleanup

Key Metrics​

Part IV: From EloqData's Perpective​

EloqData provides two production-grade databases:

• EloqKV: A Redis-compatible transactional KV database
• EloqDoc: A MongoDB-compatible document database

Both are designed to use S3-compatible object storage as primary storage, eliminating traditional disk overhead and unlocking elastic scalability. EloqData also offers a 25GB Free Tier cloud service at EloqCloud.

MinIO moving into maintenance mode does not mean the market is abandoning object storage — quite the opposite. Object storage is becoming the foundation of next-generation databases:

• From Snowflake/Databricks-style lakehouse systems
• To Kafka / Pulsar streaming platforms

The real challenge today is: How to unlock object storage for ultra-low-latency, high-concurrency OLTP workloads.

EloqData provides an answer:

• EloqKV: a fully Redis-compatible transactional KV database, achieving sub-3ms P9999 latency under million-QPS workloads using disk-only reads.
• EloqDoc: a MongoDB-compatible document database using a decoupled compute/storage architecture for elasticity and dramatic cost reduction.

Unlike traditional cloud-disk architectures, EloqKV and EloqDoc use object storage as primary storage and local SSD as an intelligent cache, reducing total storage cost to one-tenth of traditional solutions.

With native multi-replica object storage, EloqData enables a single-compute + multi-storage replica HA model, dramatically reducing CPU and memory overhead.

Object storage also enables second-level backup and branching, turning backup, environment cloning, and testing workflows from hours into seconds.

MinIO’s transition is not the end of object storage — it is the beginning of a new cloud-native infrastructure era. With a proprietary quad-decoupled architecture, EloqData breaks the performance limits of object storage in OLTP scenarios and is helping define the next generation of cloud-native databases.

EloqData — bringing object storage into the OLTP core battlefield.

Register at https://cloud.eloqdata.com and get 25GB free storage and 10,000 QPS free DBaaS.

Many people now argue that single-node databases are enough for almost everything. Hardware has become so powerful that even massive workloads can often fit on one beefy machine. OpenAI recently discussed how their main database runs on a single-writer PostgreSQL setup. Just one node handling all the writes, with many read replicas to scale out read traffic. That's a bold design, and it suggests that maybe we no longer need complicated distributed databases because modern CPUs, SSDs, and memory are making scale-out architectures look like overkill.

In this artcle, we discuss how we reached this state in the database landscape, and disucss the future for scalable databases. We draw many inspirations from history, and we believe that there is a very bright future for database community going forward as we entering the new era of growth and prosperity.

The Rise and Fall of the Scale-Out Mindset​

Twenty years ago, distributed databases were the hot thing. As the internet exploded and apps started serving millions of users, the old single-node giants like Oracle and SQL Server began to crack under the load. That's when companies like Google and Amazon came up with Bigtable and Dynamo — systems designed to scale horizontally across hundreds or thousands of machines. Their success kicked off a wave of distributed data systems: Cassandra, MongoDB, Redis, HBase and many others. These databases powered the modern internet and became the backbone of web-scale engineering.

The first wave of distributed databases came with trade-offs. To scale across machines, they gave up the ACID guarantees people took for granted in traditional databases. These "NoSQL" systems and early data lakes were fast, but they pushed the pain of handling inconsistency back to the application developers. About a decade later, around 2015, the industry figured out how to bring transactions back — both for online and analytical workloads. That's when the "NewSQL" and "Lakehouse" movements took off, with systems like Google Spanner, CockroachDB, TiDB, Snowflake, and Databricks started to appear around that time. For a while, it looked like distributed databases were going to rule the world. Snowflake went public at a staggering valuation, and companies like Databricks, Clickhouse, Cockroach Labs, and Yugabyte attracted huge funding rounds each valued in the billions.

However, just as distributed databases were hitting their stride, a new idea started to spread — maybe massive scale-out isn't necessary anymore. The first hint came from SQLite, the tiny embedded database that quietly became the most deployed software in the world. Then PostgreSQL surged ahead of MySQL and became the default choice for serious applications, from startups to enterprises. And now, DuckDB — a single-node analytical engine that runs right on your laptop — is reshaping how people do analytics. Suddenly, the towering systems built around Bigtable, Spanner, TiDB, Clickhouse and Snowflake don't look as untouchable as they once did. So what happened?

To understand this shift in mindset, it helps to look back at what changed. Before the internet era, databases like Sybase and Oracle were built for a single organization, usually serving tens of thousands of users. When the first wave of internet applications arrived in the mid-90s, those same systems were pressed into service to run websites like eBay and AltaVista. It worked for a while, until it didn't. By the early 2000s, companies like Google discovered that traditional databases simply couldn't keep up with millions of online users, the new "netizens". That realization gave birth to Google's internal infrastructure, and with it, the entire era of Big Data and distributed databases.

Back then, a typical server had two sockets, one core per socket, running around 1 GHz, and maybe a gigabyte of RAM. Google's famous early papers described clusters of thousands of such machines — an impressive scale at the time. Fast forward to today: a single high-end server can pack terabytes of memory, hundreds of CPU cores running at several gigahertz with much improved instruction-per-cycle, and dozens of SSDs each delivering gigabytes per second of throughput. In other words, the capacity that once required an entire data center now fits in a single box. Meanwhile, the total number of users hasn't grown exponentially. Even with mobile devices and global internet access, we're still bounded by the size of the human population. For most large-scale applications, serving a few million users, a single modern database can easily handle the load.

Distributed OLTP companies such as CockroachDB, TiDB, and Yugabyte have all felt the impact of this shift. Founded between 2015 and 2017, they entered the market with high expectations for explosive growth. Those expectations never quite materialized. By around 2020, their products had become stable and feature-complete, and their valuations reached their peak. Yet, the demand was far smaller than investors had imagined. These systems now face competition from two sides: from powerful single-node databases that can handle most workloads, and from partially distributed but more efficient systems such as Amazon Aurora, Google AlloyDB, and starup offerings such as NeonDB.

The OLAP world has fared better. Analytical workloads depend not only on the number of users but also on how much data is collected, how long the history is kept, and how many users or systems consume the results. As companies collect more signals over longer periods, and as machine learning models begin to consume analytical outputs, the demand for performance and scalability remains high. Snowflake and Databricks had the advantage of starting a few years earlier than the OLTP vendors, which helped them secure a strong position in the market. Much of the growth in the analytics market today comes from a new kind of consumer: AI training and inference systems that rely on massive amounts of analytical data. Still, newer systems such as MotherDuck, which builds on DuckDB, have gained attention from both developers and investors.

The New Users: AI Powered Applications and Autonomous Agents​

So what happens next? Hardware will keep getting faster, and there is no sign of that slowing down. Does that mean single-node databases like SQLite, PostgreSQL and DuckDB will take over completely, leaving little room for distributed systems? Probably not.

Due to the recent explosive AI innovation and the emergence of AI powered applications, the way we use software has changed. Every chat, prompt, or API call produces data. For instance, I use ChatGPT almost every day and generate several conversations daily. Each one is a few hundred kilobytes, stored as chat history that I can scroll through anytime. After less than a year, I already have more than a thousand conversations. Compare that with my Amazon account, where I might have made less than a thousand purchases in more than a decade, each only a few kilobytes of data. I have easily produced a hundred times more data with OpenAI than with Amazon, in a fraction of the time.

Programming tools such as Cursor go even further. They can generate tens of millions of tokens every day, and all that text needs to be stored for debugging, history tracking, or later analysis. This volume of data already puts serious pressure on backend systems.

More importantly, a new kind of user is emerging: autonomous agents powered by AI. This shift is unlike anything we have seen before. It is creating enormous demand for both data volume and query intensity. In the age of AI, we have to rethink our data infrastructure, just as the internet once forced us to do.

Soon, autonomous agents will become the dominant users of IT systems. Take something as simple as buying a T-shirt. A human might browse a few pages and make one request every half minute. An AI agent, given the same goal, could issue thousands of requests in seconds to find the perfect item. This kind of machine-driven activity will easily overwhelm infrastructure that was built for human speed and scale. Just like the netizens used to overwhelm infrastructures built for employees in a single company at the begining of the internet age.

And this is only the beginning. Other kinds of AI applications and physical robots will arrive soon. We are standing at the edge of a new wave of innovation. If you think back to 1999, it would have been almost impossible to imagine people posting daily videos on TikTok or calling a ride with a phone through Uber. The same thing will happen again. Many new AI-powered scenarios have not even been imagined yet. Any company that wants to stay ahead, whether a startup or a long-established enterprise, needs to prepare for the pressure that these new workloads will put on their data infrastructure.

The pain of non-scalable systems is not new. Engineers have shared many war stories about hitting a scaling wall and being forced to migrate their databases under fire. We believe it is time to think ahead. It is better to prepare for what AI applications will need, rather than wait until the pressure becomes unbearable. The pace of change may be much faster than people expect.

During the early internet era, Google gained a huge advantage from its internal data infrastructure. It gave their teams the freedom to run experiments, ship features, and index the web without worrying about system limits. I saw this gap first-hand while working on the early version of Bing. Competing with Google was extremely difficult because they could move faster, process more data, and try new ideas at a pace that others simply could not match. Their real-time indexing pipeline was built on top of Percolator, the large-scale distributed transactional database, which itself sat on top of Bigtable, the large-scale distributed NewSQL database. Reproducing that stack was almost impossible for anyone else at the time.

This kind of competitive advantage will matter even more as we enter the hyper-speed growth of the AI era. Companies that build strong data infrastructure early will have the freedom to innovate without being held back by system limits, while everyone else will be stuck fighting their own scalability problems. As mentioned before, OpenAI currently relies on PostgreSQL for its main database, and for now it works. But even at this stage, engineers have already talked about banning new tables creations in the main database and avoiding any write-heavy workload. They also may need hierachical replications as the replication workload of the many read-replicas are overwhelming the write replica. OpenAI is only two years into its hyper-growth phase. These kinds of infrastructure limits can quickly become a major liability as the company scales.

Looking Forward​

Going forward, there are certainly spaces for simple, light databases such as SQLite and DuckDB. PostgreSQL is certainly still going to be a major force in the database ecosystem and will continue to power many important applications, just as Orcale DB continues to be super relevant in the internet age. However, we must not ignore the challenges for scalability and performance as we entering the AI era. One cannot dismiss the need for Caterpillar Excavactors just because a shovel is sufficient for my backyard work 99.9% of the time. This is especially true when we are on the cusp of taking on the endeavor of transforming the entire world.

So where does this leave us? If AI agents become the main users of software, the demands on storage and compute will rise far beyond what a single machine can handle. We will need systems that combine the simplicity and performance of single-node databases with the elasticity and fault tolerance of distributed ones. We will also need databases that speak multiple "dialects" at once, since AI applications mix transactional work, analytical work, vector search, caching, and document storage in the same request path. The future will not be served by a pile of disconnected systems glued together with application logic. It will be served by a unified, scalable, multi-model engine that treats data as a first-class substrate.

This is the direction we are building toward at EloqData. Our Data Substrate architecture is designed for the AI age. It brings the speed of in-memory systems, the reliability of transactional databases, and the flexibility of multi-model APIs together in one platform. Our database offerings build on top of this technology, i.e. the Redis API compatible KV store EloqKV, the MongoDB API compatible document DB EloqDoc and the MySQL API compatible relational database EloqSQL all inherit the architectural advantages of Data Substrate to provide scalable, high-performance, low cost, and transactional databases with AI native features such as low cost snapshot and branching. They all can scales out when you need, scales in when you don't, and keeps storage costs low by using object storage as the primary durable layer. They allow developers build AI-driven applications without worrying about whether the next spike in workload will tip the system over.

We believe the next decade of software will not be defined by yet another wave of siloed databases. It will be defined by systems that make data infrastructure disappear into the background, allowing AI systems to read, write, learn, and act at machine speed. Our goal is to help build that foundation. Join the discussion on our Discord Channel, leverage our free tier cloud service to try out, or visit our GitHub repo to contribute. We would be happy to hear your thoughts on the future of databases.

Those "Postgres can replace Redis/Kafka" posts usually benchmark light workloads where serious tooling simply isn't necessary, and then stretch that into a sweeping narrative that "you don't need Redis or Kafka at all". But whether a workload is small or not depends entirely on the application. For a local grocery store website, twenty concurrent checkouts might qualify as high traffic; for Instagram or Twitter, a million queries per second barely registers. We're not here to debate whether people may ever need ten thousand queries per second. Generalizing one's workload to everyone's infrastructure needs is non-productive. It's like writing an article about how much grocery a new convertable can carry and conclude that U-Haul is no longer needed. No one would post that on a trucking forum expecting a heated debate. So why does the same kind of reasoning explode into controversy every time someone applies it to databases?

Sure, part of the drama is emotional. Developers get attached to their tools. Kafka, Redis, and Postgres all have cult-like followings, and any praise or criticism feels personal. But there's more to it than fandom. These debates touch on real technical trade-offs: complexity, durability, scaling, and the cost of managing multiple systems. In this article, we'll look at why it actually makes sense to use a real database for workloads that used to require specialized systems, and then tackle the elephant in the room: why this shift hasn't happened before and what's needed to make it possible.

The Advantage of Combining Workloads on a Real Database​

Back in the pre-internet days, "database" basically meant Oracle or Sybase: big relational systems that stored all of a company's data and powered every enterprise app. Then came the internet era, and data exploded in both volume and variety. The once-unified database stack shattered into a zoo of specialized systems: PostgreSQL and MySQL for transactional truth, Kafka for streams, Redis for caching, and Spark, Snowflake, or ClickHouse for analytics. We also have specialized tools for graphs, for vectors, for time-series, for tensors, and even just for account balances. Each tool solved a particular problem, but together, they created a monster of complexity. IT teams now spend countless hours wiring, tuning, and babysitting these silos just to keep data moving.

Lately, there's been a growing movement to consolidate around a single database (often PostgreSQL) as the backbone of data infrastructure. The obvious motivation is simplicity and cost: running one system is easier (and cheaper) than orchestrating a zoo of databases stitched together with duct tape and ETL jobs. But beyond operational convenience, there are deeper reasons this trend deserves attention. A converged data architecture unlocks advantages that go far beyond saving money and any systems architect can't afford to ignore.

1. Avoid Cascading Failures​

Every engineer who's operated a large-scale system knows this pain: one small failure snowballs into a full-blown outage. This phenomenon is called cascading failure. When you glue together multiple systems, say Kafka feeding MySQL with Redis caching on top, each layer becomes a potential failure point. A hiccup in Kafka can stall writes, which can then cause cache invalidations to fail, which can then cause a thundering herd of retries.

Example: Twitter's Cache Incident (via Dan Luu) In 2013, Twitter experienced an outage where a minor latency issue in the caching layer caused by interrupt-affinity misconfiguration led to a GC spiral on the tweet service. As cache latency grew, more requests bypassed the cache, overwhelming downstream services and eventually collapsing success rates to near 0% in one datacenter.

The database itself wasn't the root cause, but the incident shows how fragile multi-tiered data stacks can be: when layers depend on each other for throughput, even small delays can cascade into total failure.

This is exactly the kind of chain reaction that converged database architectures can avoid by keeping durability, caching, and data access under one coordinated system instead of scattered layers of dependency. In a converged architecture, where all data flows through a single database, node failures degrade capacity proportionally rather than triggering a domino effect. The system scales down gracefully instead of collapsing catastrophically.

2. Avoid Development Complexity​

Maintaining two code paths for the same data is a tax on every engineering team. A typical "read-through" cache setup means developers have to write one logic path for MySQL (cache miss) and another for Redis (cache hit). Over time, these two paths drift apart: schema changes, data formats, and business rules slowly go out of sync. Debugging that kind of inconsistency is pure misery. A single database eliminates that duality. Your application talks to one API, with one transaction model, and one source of truth. Simpler code, fewer bugs, faster iteration.

Here is an example code for reading user information. If Redis cache is involved, the code looks like this:

def get_user(user_id):
    # Try cache first
    try:
        user = redis.get(f"user:{user_id}")
        if user:
            return json.loads(user)
    except RedisTimeout:
        logger.warning("Redis timeout, falling back to DB")
    except RedisConnectionError:
        logger.error("Redis down, falling back to DB")

    # Cache miss or Redis failed, try database
    try:
        with postgres.cursor() as cur:
            cur.execute("SELECT * FROM users WHERE id = %s", (user_id,))
            user = cur.fetchone()

            # Try to update cache (but don't fail if Redis is down)
            try:
                redis.set(f"user:{user_id}", json.dumps(user), ex=3600)
            except:
                pass  # "It's fine, we'll cache it next time"

            return user
    except PostgresConnectionError:
        # Both Redis and Postgres down? Try the read replica
        with postgres_replica.cursor() as cur:
            # ... more error handling ...

47 lines of code to read one user. And this doesn't even handle cache invalidation, which was another 200 lines of scattered redis.delete() calls that everyone was afraid to touch.

With everything in one database, the code becomes trivial:

def get_user(user_id):
    with db.cursor() as cur:
        cur.execute("SELECT * FROM users WHERE id = %s", (user_id,))
        return cur.fetchone()

def update_user(user_id, data):
    with db.cursor() as cur:
        cur.execute("UPDATE users SET ... WHERE id = %s", (..., user_id))

3. Avoid Consistency and Durability Confusion​

Distributed systems are already hard to reason about. Add multiple data tiers like Redis, Kafka, and a transactional database and you multiply the number of durability and consistency scenarios. Each component has its own guarantees: Kafka offers a choice of "at-least-once", "at-most-once", "exact-once" delivery, and whether you need flush every writes to achieve durability is sometimes debated. Redis is eventually consistent, meaning data can be stale, and provide replication, write-ahead-log and snapshot as persistency mechanisms. Your SQL database is usually transactionally consistent with full ACID guarantee, but only within its own boundary. Stitch them together, and you're left trying to reason about correctness in a house of cards.

Example: Stale Writes and Inconsistent Cache at Scale (via Dan Luu) Dan Luu's review of real-world cache incidents includes multiple cases where caches went out of sync with the source of truth. In one Twitter incident, stale data persisted in the cache after the underlying value had changed, leading to users seeing outdated timelines. In another, duplicate cache entries caused some users to appear in multiple shards simultaneously: an impossible state from the database's perspective. The fixes required strict cache invalidation ordering and retry logic to prevent stale data from overwriting newer updates.

None of these were database bugs; they were integration bugs, symptoms of managing consistency across disconnected systems.

A single, converged database architecture eliminates this entire class of problems. There's no cache to drift out of sync, no queue to replay twice, no data race between "source of truth" and "derived truth." Everything from reads to writes to streaming happens within one transaction boundary and one consistency model. You don't just simplify the system; you make correctness provable instead of probabilistic.

What About Scalability and Performance?​

If converging everything into a single database simplifies architecture so dramatically, the obvious question is: why hasn't everyone already done it? The short answer: performance and scalability.

Specialized systems like Redis and Kafka were born because traditional databases couldn't keep up. Caches were faster, queues were more scalable, and analytics engines could crunch far more data than your average OLTP database. The trade-off was fragmentation and complexity, but at least things stayed fast. For decades, this was a necessary compromise. In early 2000 the godfather of database Mike Stonebreaker famously declared "One size does not Fit All" in database systems. Even as late as 2018, the CTO of Amazon Werner Vogels decalared that "A one-size-fits-all database doesn't fit anyone".

But is that still so? Clearly there is a demand for one size fit all database, as demonstrated by the blogs mentioned in the early part of this article. Indeed, only recently have database architectures evolved enough to make convergence a realistic alternative. In this section, we discuss the necessary capabilities of a database that must be satisfied to enable such convergence.

1. Scalable Transactions​

For years, transactions were the deal-breaker for scaling databases. Everyone wanted the simplicity of ACID semantics, but once workloads outgrew a single machine, something had to give. The conventional wisdom was: you can have efficiency, transaction, or scale, but not all of them. That's how we ended up with a zoo of specialized systems. NoSQL databases like MongoDB and Cassandra threw away multi-record atomicity in exchange for horizontal scale. Developers got used to compensating in application code: implementing retries, deduplication, or manual rollback logic. It worked, but it was painful and brittle. For relational systems, manual sharding became the necessary evil: once you hit the single-node ceiling, you split your data and your sanity along with it.

As workloads grew, people learned that sharding relational databases by hand was a nightmare. Every cross-shard join, every transaction that touched more than one key, became a distributed systems problem. You could scale reads, but writes were a different story. And once data started to overlap across shards, consistency slipped through your fingers.

That's why the new generation of distributed transactional databases, often grouped under the "NewSQL" label, was such a breakthrough. Systems like Google Spanner, TiDB, and CockroachDB showed that you could have global scale and serializable transactions, thanks to better consensus protocols, hybrid logical clocks, and deterministic commit algorithms. Even previously non-ACID systems have evolved: MongoDB added multi-document transactions in 4.0, and Aerospike introduced distributed ACID transactions feature in 8.0.0. The takeaway is that distributed transactions are already standard features in many systems.

This changes the calculus completely. When your database can handle distributed transactions efficiently, you no longer need to glue together different engines just to scale. You get scalability and correctness in one system that can become a foundation for building truly unified data architectures.

Yet, most existing distributed transactional databases are still less efficient than their single node counter part. This is an issue we need to address (see below).

2. Independent Resource Scaling​

Different workloads stress different parts of a system. Streaming ingestion (like Kafka) is IO-bound and needs massive sequential write throughput. Caching workloads (like Redis) are memory-bound, thriving on low-latency access to hot data. Analytical queries and vector search, on the other hand, are CPU-bound, demanding large compute bursts. Traditional shared-nothing databases where compute, storage, and memory scale together force you to over-provision one resource just to satisfy another. You end up paying for RAM you don't use or SSDs that sit mostly idle.

That's one of the reasons why people historically broke their pipelines into multiple specialized systems. You could put Kafka on write-optimized disks, Redis on high-RAM nodes, and MySQL on balanced hardware.

Now, cloud infrastructure is changing that equation. Modern cloud platforms allow elastic and independent scaling of compute, storage, and memory. Databases are starting to embrace this model directly. The separation of compute and storage, first popularized by cloud data warehouses like Snowflake and BigQuery, is now making its way into OLTP systems as well. This decoupling lets a single database scale ingest, query, and cache layers independently.

A truly converged database architecture must exploit this flexibility: scale write nodes when ingestion spikes, scale compute when CPU demands surge, and expand storage as data volumn grows.

3. Performance and the Zero-Overhead Principle​

C++ developers have a mantra: the Zero-Overhead Principle

1. You don't pay for what you don't use.
2. What you do use is just as efficient as what you could reasonably write by hand.

That's exactly the mindset a converged database must adopt. Existing databases rarely meet this bar. Even if you just want to run an in-memory cache, most engines still insist on running full durability machinery: write-ahead logging, background checkpoints, transaction journals, and page eviction logic. The result? You're paying CPU and latency overhead for guarantees you may not need on a given workload.

This is why Redis can outperform MySQL on pure in-memory reads even when both are running on the same hardware and the dataset is identical. The overhead isn't only in the query parser; it's in the architectural layers designed for persistence, recovery, and buffer management that never get out of the way.

A truly unified database must treat these mechanisms as pluggable, not mandatory. Durable writes, replication, and recovery logging should be modular: enabled only when needed, bypassed when not. The same engine should be able to act as a blazing-fast cache, a strongly consistent OLTP store, or a long-term analytical system, without paying unnecessary tax in the fast path.

In other words, unification can not come at the cost of overhead. It has to come from rethinking the core execution and storage engine so that the same system can be both flexible and ruthlessly efficient, depending on how it's used.

PostgreSQL May Not Be the Answer​

If "just use Postgres for everything" sounds too easy, that's because it is. PostgreSQL deserves its reputation: it's battle-tested, reliable, and absurdly extensible. But it was never designed to be the all-in-one database the modern world needs. Its architecture still reflects the assumptions of the 1990s: local disks, tightly coupled storage and compute, and a single-threaded execution model wrapped in process-based concurrency. The query pipeline, the storage system, the concurrency control mechanism are all designed with a single-node, row-based MVCC ACID relational database in mind.

You can retrofit Postgres with external layers: Citus for sharding, Neon for decoupled storage, TimeScale (now TigerData) and PGVector for handling time-series and vector data. But this is often not optimal, and often just recreates the complexity we were trying to eliminate (i.e. they do not compose). When serious modification is needed, such as making it fully multi-writer capable as in CockroachDB and YugabyteDB or streaming capable as in Materialize and RisingWave, a major rewrite is often needed and often only the wire-protocol can be reused. Extensions like logical replication, foreign data wrappers, and background workers are impressive engineering feats, but they're patches on a monolith, not blueprints for a unified data platform.

When people benchmark Postgres against Redis or Kafka, they're really comparing apples to power tools. Postgres can simulate a cache or a queue, but it isn't optimized for them. WAL writes still happen, visibility maps still update, vacuum still runs. The performance gaps show up not because Postgres is "slow," but because it's doing far more work than those specialized systems ever attempt. Its design trades raw throughput for correctness and compatibility, a perfectly valid choice, just not one that scales linearly to every use case.

Recently, there is another school of thought argues that as hardware keeps improving, scalability and efficiency becomes less relevant. If a single server can pack terabytes of RAM and hundreds of cores, why bother with distributed systems at all? Why not just keep everything in one big Postgres instance and call it a day? We fundamentally disagree with this line of thinking. Hardware growth helps, but data growth is exponential. User expectations for availability, latency, and global presence grow even faster. Scaling vertically might postpone the problem, it doesn't solve it. This is especially true in the AI age, when data is the lifeline of all applications. We'll dedicate another article to discuss this in detail.

PostgreSQL has earned its place as the default database of our time, but it's not the endgame. There is opportunity in databases that internalize the lessons of distributed systems, cloud elasticity, and modular design.

The Road Ahead: Toward a True Data Substrate​

The industry's obsession with patching and stitching different data systems together is a legacy of the past twenty years of hardware and software limits. But those limits are fading. The next generation of data infrastructure won't be defined by whether it speaks SQL or supports transactions. It will be defined by whether it can unify the data lifecycle without compromise: streaming, caching, analytics, and transactions under a single, coherent architecture.

That's exactly what we're building with EloqData's Data Substrate. Instead of bolting more features onto yesterday's database engines, we started from a clean slate: modular storage and compute layers, fully ACID-compliant distributed transactions, object storage as a first-class persistence medium, and elastic scaling across workloads. The same engine that serves as a durable operational database can act as a high-throughput cache, a streaming log, or an analytical backend without duplicating data or wiring together half a dozen systems.

This is the promise of a converged data platform: simplicity without trade-offs, scalability without fragmentation, and performance without overhead. The future of data infrastructure belongs to systems that treat data as a continuous substrate rather than a pile of disconnected silos. That's where we're headed. Of course, we still have a very long way to go, but we are working hard on this goal. Join the discussion on our Discord Channel, or visit our GitHub repo to contribute. We would be happy to hear your thoughts on the future of databases.

In this blog post, we re-examine this decoupled architecture through the lens of cloud storage mediums. We argue that this prevailing model is at a turning point, poised to be reshaped by the emerging synergy between instance-level, volatile NVMe and highly durable object storage.

Separate Storage Nodes​

The Aurora's design decouples a database into a stateless compute node and stateful, distributed storage nodes. The compute node executes queries and transactions, while storage nodes manage the transaction logs (WAL) and data pages. Internally, storage nodes apply log records to data pages, perform log truncation, and serve read requests for data absent from the compute node's buffer pool.

The influence of the Aurora’s decoupled architecture is unmistakable across the cloud database landscape. Successors like GCP AlloyDB and Azure SQL Database Hyperscale have all embraced similar architectures. The trend continues in open-source projects like NeonDB, which implements the decoupling architecture through its pageservers (data pages) and safekeepers (WALs).

The trend also extends to distributed databases like CockroachDB, TiDB, and YugabyteDB. Although they were originally architected as shared-nothing systems, it has become a common and recommended practice to deploy them on two sets of servers: one cluster for stateless compute and the other for stateful storage. This operational separation further refines the principle of independent scaling and management.

In essence, one could argue that compute-storage decoupling serves as a foundational pillar for cloud-native online databases, which rely on one or more separate storage nodes.

The Landscape of Cloud Storage Mediums​

It is a common misconception to view cloud storage as merely an extension of on-premises solutions. In fact, cloud platforms offer a diverse spectrum of storage mediums, each with performance and cost characteristics that differ drastically from traditional models. The three most common tiers in the cloud are instance-local storage (e.g., NVMe), network-attached block storage (e.g., Amazon EBS), and object storage (e.g., Amazon S3).

Block-level storage is a cloud service that emulates network-attached hard drives. As a resource provisioned and billed independently from VMs, it guarantees full data durability beyond the lifecycle of any attached VM. The combination of familiarity and resilience has made it the de facto storage medium for online databases.

Instance-level storage is temporary, high-performance block storage that resides directly on a VM's physical host. Physical proximity enables extremely low-latency access. Specifically, instance-level NVMe have pushed performance to unprecedented levels: up to 2 million IOPS. However, this raw speed comes with a critical trade-off: the storage is ephemeral, meaning all data is lost if the instance is stopped, terminated, or experiences a hardware failure.

Block-level and instance-level storage thus represent a fundamental trade-off for storing data in the cloud. Block storage guarantees durability at a steep premium, whereas instance storage delivers superior performance and lower cost but sacrifices all data persistence.

To understand the stark performance-price difference between instance-level NVMe and block-level storage, let's examine a concrete example from Amazon. Amazon EBS io2 is a premium block storage solution offering up to 256K IOPS. Provisioning an io2 volume with 150K IOPS and 1 TB capacity costs approximately $6,000 per month. By contrast, an Amazon storage-optimized instance i8g.xlarge provides comparable performance with 150K IOPS and 937 GB of local NVMe storage. This instance features 4 vCPUs and 32 GB of memory, and yet costs only $247 per month. When we subtract the cost of compute resources, i.e., $170 for an equivalent r8g.xlarge instance with identical CPU and memory but no local storage, the net cost of the high-performance NVMe storage becomes almost negligible compared to the premium EBS solution.

Apart from block and instance storage, object storage is a scalable storage service for organizing data as objects in buckets. It excels at handling unlimited data with high durability (e.g., 99.999999999%) at a very low cost, often cheaper than traditional on-premises storage. However, its access latency is unpredictable and can be as high as hundreds of milliseconds. This performance profile has historically precluded its use as the default storage for online databases.

NVMe + Object Storage​

Given the landscape of cloud storage, a growing consensus recognizes that block-level storage is ill-suited for primary storage of online databases, being both prohibitively expensive and slow. Instead, a combination of instance-level NVMe and object storage is emerging as a superior architectural pattern.

• Write path: Data modifications are first committed to local NVMe storage and then uploaded to object storage.
• Read path: Reads that result in in-memory cache misses are served directly from the local NVMe, ensuring low-latency access and high I/O throughput.
• Stateless compute: VMs can be freely stopped or migrated. A new instance simply reconnects to object storage and incrementally downloads data files to its local NVMe.

In this design, NVMe delivers exceptional performance at a fraction of the cost of block-level storage, while object storage provides the ultimate guarantee of data durability.

A legitimate concern with the combination is the potential for unpredictable latency. This could occur if an online query needs to download from object storage a data file not available on local NVMe, which has reached its capacity limit.

However, we contend that such scenarios should be rare for two reasons. First, the scale of local NVMe capacity available in the cloud is substantial. Consider Amazon's storage-optimized i8g instances: a 64-vCPU VM provides 15 TB of local NVMe storage. Second, if a dataset grows beyond the capacity of a single node, the storage engine should be scaled out horizontally across multiple VMs. A conscious decision to operate with smaller local storage than the total dataset size should only be made when the application's access pattern exhibits strong locality (e.g., focusing on recent data) and can tolerate occasional high-latency access to colder, historical data archived in object storage.

A second common concern regarding object storage is high write latency. However, this is effectively mitigated by a well-designed database architecture that leverages a transaction log. The transaction log absorbs all writes on the critical path, ensuring low-latency commitments. The actual data writes to object storage are then deferred to asynchronous, background checkpointing processes.

Since checkpointing is a throughput-oriented, background task, it is inherently tolerant of higher latency. Therefore, the performance characteristics of object storage writes have a minimal impact on online queries.

With NVMe and object storage serving as the storage mediums for database storage engines, block-level storage finds a new role in managing the transaction log. The transaction log demands high write throughput, moderate IOPS (which is easily satisfied by the batched, append-only write pattern), and relatively low capacity due to continuous log truncation. Crucially, it must also guarantee 100% durability the moment a query commits, while maintaining low write latency. These specific requirements align well with the performance and durability profile of block-level storage, allowing databases to leverage it for the transaction log at a remarkably low cost. For instance, an Amazon EBS gp3 volume providing 3,000 IOPS, 500 MB/s of throughput, and 100 GB of capacity can be provisioned for approximately $20 per month.

New Shapes of Decoupling​

The combination of instance-level NVMe and object storage will redefine the architecture of compute-storage decoupling. In this new paradigm, object storage serves as the decoupled storage layer, which offers unlimited capacity and permanent durability. Meanwhile, instance-level NVMe acts as a local storage cache, in addition to main memory, and becomes an integral part of the compute layer. Within this framework, block-level storage finds its optimal role as a dedicated, cost-effective medium for hosting the transaction log, which is attached directly to compute nodes where low-latency writes are critical.

Of course, the established decoupled architecture can be augmented by equipping storage nodes with local NVMe, preserving the original benefits of independent scaling while enhancing storage performance. However, we argue that the value of such an enhancement is diminished in many scenarios.

A key reason lies in the fixed resource ratios inherent to cloud instances. For example, on Amazon's storage-optimized i8g instances, each vCPU is paired with 8 GB of memory and 468 GB of local NVMe. While these CPU and memory resources are provisioned to handle cache-miss read requests, they often remain underutilized due to two factors: the intermittent nature of cache-miss reads depending on workload patterns, and the fact that accessing local NVMe may require much less CPU and memory than allocated.

Furthermore, this architecture introduces inherent inefficiencies. Each cache-miss read necessitates a network round trip, incurring not only additional latency but substantial CPU overhead on both ends for data serialization and network processing. The two-sided CPU cost of network-based access becomes an increasingly significant resource drain, making remote storage access less appealing compared to local NVMe operations.

Nevertheless, the original decoupling architecture maintains its relevance in some scenarios, e.g., when dealing with a large-scale dataset that they must be distributed across multiple storage nodes.

When dealing with large-scale datasets, we believe a more promising approach lies in horizontally scaling both compute nodes and their attached NVMe storage. This architecture avoids the resource overhead of the traditional decoupled model by transforming the entire system into a multi-writer distributed database, where compute, memory and storage resources are fully utilized for processing, not just for serving remote requests.

Viewing object storage as the decoupled persistence layer enables new variants of database architecture. This is evident in high availability designs: object storage natively provides cross-AZ or even cross-region reliability. By complementing this with replication of the transaction log across multiple zones, the entire database inherently achieves geo-distributed reliability. This approach eliminates the need for complex, resource-intensive storage nodes spanning multiple zones, replacing them with lightweight transaction log replicas that consume significantly fewer resources.

Conclusion​

In this post, we re-examined the decoupled architecture of online databases through the lens of emerging cloud storage technologies, particularly the powerful combination of instance-level NVMe and object storage. We believe that this new storage paradigm is poised to become the new storage backbone for future cloud databases, ultimately reshaping how Database-as-a-Service (DBaaS) platforms are designed and deployed in the cloud.

Cloud databases must constantly balance durability, performance, and cost. In modern cloud environments, there are three main types of storage available:

Let’s break down the trade-offs and why recent events place a spotlight on risky architectural choices.

Option 1: Block-Level Storage (EBS) - Durable but Expensive and Slow​

EBS is the default choice for reliability:

• It survives instance failures.
• It supports cross-AZ replication via multi-replica setups.
• It enables quick reattachment to replacement nodes.

But the downside?

• GP2/GP3 disks deliver modest IOPS and high latency.
• High-performance variants like IO2 are extremely expensive when provisioned for hundreds of thousands of IOPS.
• Scaling performance often means scaling cost linearly.

EBS gives you durability-but performance per dollar is disappointing.

Option 2: Local NVMe - Fast but Ephemeral (and Now Proven Risky)​

Instance families like i4i provide 400K+ to 1M+ IOPS from local NVMe, making them a natural fit for databases chasing performance.

So many database vendors recommend:

• Use local NVMe for primary storage
• Add cross-AZ replicas for durability

But here’s the problem: Local NVMe is tied to the node lifecycle. If the node restarts, fails, gets terminated due to spot interruption, or is impacted by a region-level failure such as the recent us-east-1 outage-you lose ALL the data.

During routine failures, cross-AZ replicas often protect you. But during region-wide degradation or cascading incidents, with local NVMe, there is nothing to recover. The storage is simply gone. What you can do is to recovery from recent backups - often lagging days. Write loss is guaranteed between last backup and crash.

In contrast, EBS volumes can always be reattached to a new node.

The AWS us-east-1 outage just validated that “local NVMe + async replication” is a high-risk strategy for mission-critical databases.

Option 3: Object Storage (S3) - Durable & Cheap, But Latency Is a Challenge​

Object storage is:

• 3x cheaper than block storage
• Regionally and cross-region durable
• Built to survive region-level failures
• Practically infinite
• A first-class citizen for modern cloud-native platforms

But the challenge remains: S3 latency is too high for OLTP if accessed synchronously.

This is why traditional OLTP engines avoid it.

So the question becomes: How do we get the cost & durability benefits of S3 without paying the latency penalty?

The Data Substrate Approach: Object Storage First, NVMe as Cache, EBS for Logs​

EloqData treats object storage (e.g., S3) as the primary data store, and architect the system to avoid the usual latency pitfalls:

Through Data Substrate, we decouple storage from compute and split durability between:

• Log: persists immediately to EBS
• Data store: periodically checkpointed to S3 (async + batched)
• NVMe: purely a cache layer, safe to lose at any time

This allows us to:

• Withstand node crashes seamlessly
• Recover fully even if local NVMe is wiped
• Handle region-level disruption by replaying logs and checkpoints
• Enjoy millions of IOPS from NVMe without durability risk
• Cut storage cost by 3x+ compared to full EBS-based systems

Check out more on our products powered by Data Substrate:

• EloqKV
• EloqDoc
• EloqSQL

The Larger Industry Trend​

We are not alone in this shift. The broader ecosystem is moving object-storage-first:

EloqData brings this model to OLTP with a transactional, low-latency engine powered by Data Substrate.

After the Outage: A Hard Question Every Architect Should Ask​

If my database node died right now, would I lose all my data?

If you're running a primary database on local NVMe, and relying solely on async replicas, the answer might be yes.

It’s time to rethink durability assumptions in the cloud era.

Summary​

AWS us-east-1 just reminded the industry: Performance is replaceable. Lost data is not.

With the right architecture, you don’t have to choose.

• Build fast.
• Stay durable.
• Be outage-proof.

That’s the future we’re building at EloqData.

Check out more on our open source databases:

• EloqKV
• EloqDoc
• EloqSQL

Hardware Trend​

Database implementations are closely tied to hardware development. If we look back at the past decade, the trend is clear: the performance improvement of I/O devices has far outpaced that of CPUs. And the trend is likely to continue for the foreseeable future. As of 2025, it’s common for a single NVMe device to deliver 2 million IOPS. With 6 to 8 PCIE lanes, a single server can achieve a staggering total of 12 to 16 million IOPS.

Such massive I/O gains, however, come at a cost. These staggering numbers can only be unlocked through massive parallelism. In the traditional multi-threading model, this means a proliferation of threads—each dispatching and waiting for an I/O request. In conventional engineering, dispatching an I/O request involves a synchronous syscall (e.g., read() or write()), and waiting for it incurs two context switches. Though these operations are fast in isolation, their aggregate CPU cost is significant in comparison to the needs of sustaining 16 million I/O operations per second.

What’s more, I/O requests don't appear out of thin air. In a database, every query initiates I/O requests and the average number of I/O requests per query remains relatively stable for a given workload. Pushing for greater I/O parallelism means handling more concurrent queries, which consumes additional CPU for query parsing and execution. The system quickly gets caught in a self-reinforcing downward spiral: escalating competition for CPU ultimately makes the database CPU-bound. Therefore, reducing the CPU cost, both for I/O and other core components, will be a primary focus for future database engineering.

Coroutines and Async Programming​

Coroutines are functions that can be suspended and resumed. Unlike threads, which require expensive OS context switches, suspending or resuming a coroutine (whether stackless or stackful) is a user-space operation and can complete in less than 10 nanoseconds. This represents a 10 to 100-fold performance advantage, making coroutines an ideal programming model for processing massive parallel I/O requests.

Coroutines can unlock their full potential when combined with modern asynchronous I/O interfaces like io_uring. Async I/O allows a coroutine to dispatch an I/O request without blocking or invoking a syscall. After issuing the request, the coroutine suspends itself, allowing another coroutine to take over and continue execution, until it dispatches an I/O request and yields.

The advantages of coroutines extend beyond I/O to concurrency control in databases. Online databases must handle lots of concurrent reads and writes while ensuring proper isolation and serialization. This often requires blocking certain operations when data conflicts occur and resuming them later. The pattern of frequent suspension and resumption makes database queries an ideal use case for coroutines.

Combing coroutines and async I/O, we run each database query as a coroutine bound to a CPU core. Each core has a dedicated worker thread that runs an event loop. When new network messages arrive, the async I/O notifies the worker thread, which creates a coroutine for each query and processes them sequentially. Whenever query execution is blocked—whether due to concurrency control or waiting for storage/network access—the current coroutine yields, allowing the next coroutine in the line to run.

At the end of each loop iteration, the thread makes one syscall (io_uring_submit) to notify the kernel all pending I/O requests accumulated during that cycle. The worker thread also calls a second syscall (io_uring_peek_cqe) for completed I/O requests from prior iterations or new incoming network requests. Coroutines whose requests have returned are scheduled to resume in the next loop iteration.

Engineering Considerations​

Coroutines are an old concept, but they have only recently gained popularity and achieved standardization in major programming languages. Throughout history, coroutines have appeared in various forms, such as

• Handwritten state machines for specific applications.
• Coroutines in programming frameworks, e.g., bthread in brpc.
• Stateful coroutines libraries, e.g., boost:context.
• Stackless coroutines libraries, e.g., std::generator.

These coroutine variants emerged at different times and vary widely in maturity. For instance, std::generator was only introduced in C++23 and lacked compiler support until the recent GCC 14 release. Databases are built over many years, spanning multiple language standard and compiler release cycles. At every stage of development, we must carefully evaluate the availability, maturity, and inherent trade-offs of each coroutine implementation. Our implementation as of today has used all the first three forms:

• Handwritten state machines for core transaction processing
• bthread in brpc for networking
• boost::context to transform third-party code of query processing into stackful coroutines

We’ve recently started adopting std::generator in some parts of transaction processing, where coroutine call chains are isolated and yield/resume states are complex to manage.

A second consideration when using coroutines is that they are susceptible to blocking: if a coroutine is blocked during execution, the entire worker thread is blocked. All other coroutines bound to that thread are also unable to make progress.

Blocking can arise from several sources. The first is I/O, which has been largely mitigated in our system through the adoption of async I/O. The second source is thread synchronization: when multiple worker threads access a critical section or a concurrent data structure, contention may cause one thread to block another. In our implementation, we adopt a shared-nothing memory architecture. Most thread synchronization is handled through lock-free concurrent queues. We still occasionally use locks (such as std::mutex), but we restrict them to very short, low-contention critical sections.

The third source of blocking originates from third-party code or libraries designed around the multi-threading model. Some code is straightforward to adapt. For instance, query processors, which involve minimal thread synchronization, require only minor code changes. Others, however, are inherently difficult or impossible to modify. One example is RocksDB, a widely used key-value store that serves as our default storage engine. As a multithreaded library, RocksDB relies extensively on internal locks and condition variables. Any coroutine invoking its APIs may block unpredictably. Hence, accessing RocksDB and similar libraries must be offloaded to a separate thread pool.

The final consideration is to carefully design internal states when using coroutines in distributed databases/systems. A key advantage of coroutines is their ability to preserve execution state—such as local variables and control flow—across suspension and resumption. However, this feature introduces a challenge in distributed environments, where failover is a common occurrence. Failover may not terminate the process but can invalidate node-local resources. Consider a scenario where a query holds a write lock, stores a pointer to it in its coroutine stack, and then yields. If a failover occurs during suspension, all locks on the node are deemed invalid. When the coroutine resumes, it must not blindly release the lock—which may now refer to an invalid memory address. To handle such cases, the system must incorporate a mechanism to validate coroutine-held resources that may be affected by failover. Only if the resources are confirmed to be alive should the resumed coroutine proceed to use them.

Conclusion​

Coroutines and async programming represent a paradigm shift in building high-performance online databases for modern hardware. Their adoption, however, demands careful engineering and thoughtful design. Our journey has just begun, and we believe significant work remains to fully unlock their potential. One area is storage engines. Currently, very few are fully built on coroutines and async I/O. We anticipate this will change soon.

Thread Per Core Execution Model and Async I/O​

The Data Substrate adopts a thread-per-core execution model, a strategy well-aligned with modern multicore hardware. In this model, each core is assigned a dedicated thread that manages a variety of tasks using cooperative multitasking, eliminating the overhead of frequent context switches. This design ensures high CPU cache locality, reduces synchronization complexity, and enables fine-grained control over task scheduling. By minimizing contention and maximizing predictable performance, the thread-per-core model lays the foundation for building scalable and efficient distributed systems.

In Data Substrate, the thread serves as the primary unit of abstraction, with data partitioned and processed independently by each thread. Inter-thread communication is handled exclusively through message passing, and synchronization is limited to lightweight producer-consumer queues. This design intentionally reduces reliance on shared memory within a server node, allowing threads to operate in isolation and avoid contention. Specialized threads are designated for tasks like RPC handling and storage access, but beyond that, threads remain largely self-contained. This approach not only reflects best practices for building high-performance single-node applications but also provides a clean and natural path for scaling out to distributed deployments.

To ensure responsiveness and efficient resource utilization, all potentially blocking operations in Data Substrate are handled asynchronously. By leveraging modern OS capabilities like io_uring, the system performs network and storage I/O without stalling worker threads. This asynchronous model is critical for managing latency in I/O-bound workloads, a common characteristic of databases. With non-blocking operations, regular worker threads can remain active and productive, eliminating idle CPU cycles and maintaining high throughput even under heavy load.

In Data Substrate, each thread is designed to fulfill multiple roles—it may execute operations while also managing one or more CCMap shards. When a thread needs to access data and it happens to manage the shard containing the target entry, it can skip message passing altogether and access the memory directly, achieving maximum efficiency for local operations. This optimization demonstrates one of the core principles of the Data Substrate design, which is to enable zero overhead in degraded modes of operation: if a distributed feature isn't required, it should not impose any performance penalty. In this case, such optimization ensures that no additional overhead is incurred when the system operates in a non-distributed, single-threaded configuration. A direct consequence of this design is the performance characteristics of EloqKV. When performing in-memory data access (i.e. when WAL and data store is disabled), its per-core performance is close to Redis, which is a single threaded implementation. In a multi-threaded setup, its performance is close to DragonflyDB, a modern purpose-built multi-threaded in-memory KV store with Redis API.

Decoupling of Compute, Memory, Log and Data Store​

Data Substrate achieves a high degree of modularity by decoupling compute, logging, and storage into separate threads, each responsible for its own function. This separation allows for clean isolation of responsibilities and avoids cross-component interference. While the current implementation may allow a thread to handle both query computation and buffer management, the architecture is designed such that these roles can be fully separated. The message-passing model abstracts away physical locality, enabling threads to interact seamlessly whether they reside on the same server or across the network. As a result, each subsystem can scale independently, allowing the system to adapt dynamically to changing workload patterns.

This decoupling is a foundational principle of the Data Substrate design, enabling the independent scaling of four critical resource types: compute (CPU cycles for query execution), memory (used as dirty data buffer and cache for hot data), logging (governing write throughput and durability latency), and storage (backing data capacity and cache miss performance). In cloud environments where resource provisioning is granular and on-demand, this separation offers a significant architectural advantage. It allows the system to allocate only the resources needed for a given workload, avoiding over-provisioning and improving both cost efficiency and operational flexibility.

Moreover, this architectural decoupling enables dynamic scaling—resources can be added or removed at runtime without interrupting ongoing operations. This is particularly powerful in cloud-native deployments, where workload patterns can shift rapidly and unpredictably. Data Substrate is designed to respond to these changes by elastically adjusting compute, memory, logging, and storage resources on demand. To our knowledge, it is the only database architecture that supports independent dynamic scaling of all four resource types, offering a level of elasticity and adaptability that traditional monolithic systems cannot match. (Note: dynamic scaling of logging is still under development.)

Share Everything Architecture​

Traditionally, the shared-nothing architecture has been the dominant model for building scalable distributed databases. In this approach, each server node operates as an independent unit with its own dedicated compute, memory, and storage resources. While this simplifies certain aspects of distribution, it also introduces rigid boundaries—when a node fails, its local data and state become immediately inaccessible, potentially disrupting the system. Moreover, dynamically scaling, though possible, is tedious and time consuming, since migrating data between servers often takes hours or even days. In contrast, modern cloud-native designs like Amazon Aurora adopt a shared-storage model, where data is stored independently from compute, allowing read replicas to be dynamically launched to handle fluctuating read workloads. This separation of data and storage introduces greater elasticity and fault tolerance but remains limited in certain aspects. For example, to avoid the overhead of distributed transactions, shared-storage system only allows a single writer node, and the readers generally can only see stale state.

Data Substrate takes a different approach by embracing a share-everything architecture, where every thread can access and interact with the full system state, regardless of its physical location. This is a deliberate departure from the traditional share-nothing and share-storage model.

In the share-everything model adopted by Data Substrate, each thread functions as an autonomous unit, capable of operating independently while still having access to the global system state. This abstraction simplifies system design by removing rigid node boundaries and enables a highly elastic and fault-tolerant architecture. Developers can think of programming Data Substrate as programming a large single-node machine, where the primary challenge is handling the failure and recovery of individual threads. When a thread fails, it is responsible for reconstructing its own state by replaying logs and retrieving data from the durable storage layer, ensuring that fault recovery is localized, efficient, and minimally disruptive to the rest of the system.

Of course, making a distributed system behave like a single computer requires careful handling of a number of complex challenges, such as avoiding split-brain scenarios and ensuring consistent state across failures. Fortunately, decades of research in distributed systems have produced a rich set of techniques to address these problems. Consensus protocols like Paxos and Raft ensure agreement among nodes, while replication protocols maintain data consistency and durability. Lease and failure detection mechanisms help coordinate access and detect faults promptly. Data Substrate builds on these foundational techniques, integrating them into its architecture to provide the illusion of a unified system while preserving the robustness and safety required in a truly distributed environment.

One common concern when treating a distributed system as a single large computer is data locality, which traditionally has a major impact on performance due to network latency and bandwidth limitations. However, in modern cloud environments, high-speed datacenter networks with sub-millisecond latency and multi-gigabyte bandwidth have significantly reduced this bottleneck. While data locality still plays an important role in optimizing performance, it is no longer a strict constraint. In a share-everything architecture like Data Substrate, data locality is a matter of policy rather than a concern for correctness. By adjusting policy to restrict thread placement and scheduling, Data Substrate can effectively mimic shared-nothing or shared-storage architectures as needed.

With Data Substrate, building a database on top of a shared-everything distributed "computer" becomes a natural extension of decades of work on single-node database systems. This architecture allows us to reuse well-understood mechanisms for concurrency control and recovery, while adapting them to operate in a distributed context. For instance, databases on Data Substrate can support a range of isolation levels—read committed, repeatable read, serializable or MVCC and snapshot isolation—just as traditional systems do. While higher isolation levels may carry greater performance costs on a large scale, their correctness remains straightforward to reason about. Thanks to the clean separation of concerns in the Data Substrate design, low-level distributed system challenges like node crashes and restarts are isolated from database-level logic, making the system easier to reason about, maintain, and extend.

Auto Scaling and Scale To Zero​

In the context of cloud-native databases, features like auto scaling and scale to zero have become essential for optimizing cost and performance, with support now seen in systems such as Neon and AWS Aurora. Auto scaling enables a database to dynamically adjust compute resources in response to workload fluctuations—scaling up during peak demand and scaling down when load decreases—without manual intervention. Scale to zero takes this a step further by releasing compute and memory resources entirely when the system is idle for a certain duration. Upon receiving new workload, the system can rapidly rehydrate and begin processing within seconds. This approach dramatically improves cost efficiency, as users incur charges primarily for persistent storage when the database is not actively in use, aligning resource usage directly with actual demand.

Historically, implementing auto scaling and scale-to-zero has been challenging due to the tight coupling of compute and storage in traditional database architectures. In such systems, compute nodes typically manage in-memory caches, transaction state, and local storage, making it difficult to release or reassign resources without interrupting availability or risking data loss. The emergence of decoupled cloud storage changes this equation—by separating persistent data from compute, databases can now shed or acquire compute and memory resources.

Databases built on top of Data Substrate can implement auto-scaling with minimal complexity, thanks to the system's inherently decoupled architecture. Since compute, memory, storage, and logging are managed independently, a lightweight monitor process can dynamically allocate or release compute and memory resources—typically as containers or VMs—based on current workload demands. In a cloud environment, the storage (such as S3 or DynamoDB) are generally regarded as fully scalable. Last but not least, as an independent service in Data Substrate, the logging service can easily spin up more logging devices to allow better write throughput and reduce write latency as needed. As far as we know, the databases built on top of Data Substrate, such as EloqKV, EloqSQL and EloqDoc are the only family of distributed databases that can independently scale all four types of resources.

As for scale-to-zero, when the database becomes idle, the monitor instructs it to take a final checkpoint, flush any in-memory state to durable storage, truncate the logs, and cleanly shut down the active components. Upon the return of workload, a new container can be launched, quickly recover the last state from storage, and resume processing with little to no delay—making this process efficient, cost-effective, and operationally seamless.

Object Storage as Primary Data​

A major trend in the cloud computing era is the widespread adoption of object storages such as Amazon S3 as the primary data store, driven by their compelling advantages in cost, durability, and scalability. While object storage was initially considered slow, this perception has changed with the evolution of software architecture that can effectively hide or amortize latency. OLAP systems were early adopters—databases such as Snowflake, and open formats like Apache Iceberg, support high-performance analytical processing on top of S3. The movement has since expanded beyond analytics: StreamNative's Ursa engine uses S3 as its core storage layer for streaming data, Confluent launched Freight Clusters to adopt a similar model. Even vector and full-text search engines, like Turbopuffer, are embracing object storage to decouple compute from data and support elastic scaling.

However, using object storage as the primary data store introduces non-trivial latency, which, while tolerable for OLAP workloads and asynchronous, event-driven architectures, becomes a bottleneck for OLTP systems that require consistently low-latency responses. This is why most OLTP databases still avoid relying directly on object storage. In cloud environments, fast and cost-effective NVMe storage is often available on virtual machines, offering excellent performance characteristics—but this storage is ephemeral and tied to the lifecycle of the VM, meaning all data is lost if the VM is terminated. As a result, most OLTP databases today rely on services like Amazon EBS (Elastic Block Store), which provide durable, VM-independent storage. EBS can be reattached to a new VM after a failure, ensuring data durability, but this reliability comes at a cost—particularly when high throughput or low-latency IOPS are required, making it an expensive choice for performance-critical applications.

Database systems built on top of Data Substrate are uniquely positioned to take full advantage of object storage without suffering its typical drawbacks. Thanks to Data Substrate's decoupled architecture, S3 can be used as the primary data store to absorb the bulk of data capacity, while durability is jointly ensured by both the Log and the Data Store, as detailed in our previous post. Logging—which demands low-latency, durable writes—can be handled by a small EBS volume, since logs are mostly append-only and frequently truncated. This minimizes cost while preserving reliability. Updates to the data store happen at checkpointing time, when dirty CCMap entries are cleaned. Therefore, updates are performed in batches and asynchronously, meaning they do not block critical paths and can be safely written to S3 in the background. Meanwhile, local ephemeral NVMe storage can serve as a high-performance cache for read operations, absorbing cache misses efficiently. By orchestrating these three tiers—object storage for capacity, EBS for durability, and NVMe for performance—Data Substrate achieves a balanced, cost-effective storage strategy that leverages the strengths of each medium while avoiding their respective limitations.

Introduction​

In traditional database systems, scale and consistency often come at the cost of complexity, rigidity, or degraded performance. At EloqData, we've taken a fundamentally different approach. Data Substrate reimagines the database as a unified, distributed computer—where memory, compute, log and storage are fully decoupled yet globally addressable. In a previous article, we explored the motivations and core inspiration behind this architecture. Today, we dive deeper into its technical foundations—highlighting the key design decisions, abstractions, and architectural choices that set Data Substrate apart from both classical and modern distributed databases. While this article is not exhaustive, we aim to offer a clear, grounded view of how Data Substrate works and why it matters. For those interested in exploring further, visit our GitHub repositories or join our Discord community.

A Retrospect Overview of Existing Database Designs​

Database systems have evolved over many years, and the canonical design remains the single-node, multi-threaded (or multi-process, in the case of e.g. PostgreSQL) architecture, as described in numerous textbooks. This classical design typically includes a page buffer manager for handling in-memory state, a lock manager for concurrency control, a Write-Ahead Log (WAL) for durability, and a B-Tree based data store for managing on-disk pages. These components collectively make up the state management layer of the database. Above this layer, the execution engine—which includes a SQL parser， query optimizer， and query execution engine—interacts with users and translates their requests into operations on the state management layer of the database.

As mentioned in our previous blog post, the Data Substrate design is heavily inspired by the textbook single-node database architecture. However, before diving into its specifics, it's important to examine the limitations of the traditional approach and the solutions that have emerged to address them.

One of the major challenge is scalability: classical designs rely on vertical scaling—using larger, more powerful machines—and do not support horizontal scaling across multiple servers, which severely limits their ability to grow with increasing workloads.

To handle data that cannot fit in a single server, various approaches have been proposed. The simple method is database sharding, by partitioning the database into multiple smaller database shards and split the workloads to be handled by multiple single-node databases. An example is the Vitess/PlanetScale approach that leverages multiple MySQL instances. While database sharding allows systems to scale out, it introduces significant operational complexity, often requiring manual planning, rebalancing, and custom tooling. Moreover, it offers poor support for cross-shard transactions and queries, which can lead to inconsistencies, degraded performance, or the need for complex coordination protocols that are hard to maintain and debug.

The alternative to database sharding is data sharding—building a truly distributed database system and distribute data among server nodes. However, this approach is inherently complex, particularly when it comes to supporting ACID transactions across distributed nodes. Many NoSQL systems like MongoDB and DynamoDB in the beginning chose to forgo strong consistency to simplify this challenge, offering eventual consistency instead. Though later versions of these systems almost always added ACID support, famously pointed out by a recent paper. NewSQL databases such as CockroachDB and TiDB, on the other hand, often adopt designs inspired by Google's Percolator or Spanner, layering distributed transactions over an underlying key-value store using protocols like two-phase commit. While this enables stronger consistency, it typically comes with substantial overhead, resulting in significantly lower per-node performance compared to well-optimized single-node databases.

In recent years, shared-storage databases like Amazon Aurora and Neon (now part of Databricks) have gained traction in the market. These systems offer virtually unlimited storage and support multiple read replicas, but they remain constrained by a single write node, which imposes fundamental limits on write scalability. Additionally, the read replicas often serve stale data due to replication lag, making them less suitable for workloads requiring strong consistency. This architecture also struggles when the hot data working set exceeds the memory capacity of a single node, causing performance degradation.

Another limitation of the classical single-node database design is the tight coupling between on-disk data storage and in-memory data representation. Historically, page–based B-trees have dominated as the primary on-disk storage format, leading many traditional databases to manage memory in terms of pages as well. This close coupling makes it difficult to support alternative data models like graphs or streams, which may require different memory and storage abstractions. While newer systems have adopted techniques like LSM trees to address some of these challenges, legacy systems such as MySQL and PostgreSQL remain largely constrained by their page-based memory architectures. Moreover, many storage related issues, such as garbage collection of free pages, are coupled with concurrency control of the databases or the query execution engines, making it difficult to swap out storage, adopt new concurrency control algorithms, or add new data models. In short, many of the existing databases are not cleanly modularized.

In this article, we discuss how EloqData's Data Substrate architecture addresses these issues.

Scalability and the CCMap​

As mentioned earlier, the Data Substrate architecture draws inspiration from classical single-node database design, but introduces key innovations to support diverse data models and enable true scalability.

One of the foundational changes lies in the redesign of the buffer pool. Instead of a traditional page-oriented buffer pool, we introduce an in-memory–optimized structure called the TxMap—a large, sharded key-value map. Each entry in the TxMap consists of a key-value pair, abstracting away the specific data model. The key could represent a page ID in a relational system, a node ID in a graph, or a timestamp in a time-series database, while the corresponding value might hold a page's content, a graph node's adjacency list, or a time-series payload. This abstraction allows the upper-layer computing engine to interact with data uniformly, regardless of the underlying model.

The TxMap is divided into multiple shards, each called a CCMap, or Cache and Concurrency control Map. A CCMap holds a subset of the TxMap's entries and is responsible not only for storing data but also for managing concurrency control metadata such as row-level locks. It tracks modification timestamps and marks entries as dirty when they have been updated. Crucially, each CCMap is exclusively accessed by a single thread, eliminating the need for latches and enabling highly efficient operations. Interactions with a CCMap entry are performed through message passing to the thread that owns the shard, ensuring safe and scalable concurrency management.

This abstraction provides a powerful foundation for scalability by offering a uniform mechanism for accessing sharded data through CCMaps. When a thread needs to access a data item, it simply issues a request to the corresponding CCMap, without needing to know whether the target shard resides on the local machine or a remote one. This decoupling allows the system to scale across multiple machines transparently.

Decoupling of Logging and DataStore​

Unlike many distributed database designs that adopt a "shared nothing" architecture, Data Substrate embraces a "shared everything" approach, treating the entire distributed system as a single, unified computer. In this model, the TxMap functions as a large, distributed memory accessible by any thread, regardless of where the data physically resides. In addition to TxMap, Data Substrate introduces two other major components: the logging service and the data store. These components are designed to be decoupled and shared across the system, enabling flexible coordination and efficient resource utilization at scale.

The logging service in Data Substrate fulfills largely the same role as the Write-Ahead Log (WAL) in traditional single-node databases, but it is reimagined for distributed environments. Unlike the monolithic WAL of classic systems, Data Substrate supports multiple distributed logging devices, each functioning as a durable, recoverable log accessible by any thread during recovery. Log durability is ensured either through replication or by leveraging highly reliable services such as AWS EBS.

When a thread needs to persist log entries during transaction execution, it can write to any available logging device. To maintain global order among distributed log entries—a critical requirement for consistency—traditional distributed databases often rely on external timestamp services (like time oracles) or specialized hardware (such as Spanner's atomic clock based truetime). Data Substrate, by contrast, adopts a hybrid approach that combines Hybrid Logical Clocks (HLC) with a TicToc-inspired time-travel mechanism to assign globally ordered timestamps. Because recovery threads can access all log devices, the system can reliably reconstruct the state of the database during failures. This flexible, decentralized logging model not only enhances fault tolerance but also improves scalability by avoiding bottlenecks associated with centralized log (and timestamp) coordination.

As in traditional single-node databases, a transaction in Data Substrate is considered committed once its WAL entry has been successfully written. The effects of the committed transaction are then applied to the in-memory buffer (i.e., the TxMap), marking certain entries as dirty. To manage memory usage, these dirty entries must be periodically checkpointed to a persistent data storage engine. Once checkpointing is complete, the corresponding log entries can be safely truncated to reclaim space. In Data Substrate, the data storage engine is abstracted to work seamlessly with any durable key-value stores, enabling flexibility in choosing the most appropriate storage backend for different workloads or deployment environments.

In the Data Substrate architecture, durability is jointly ensured by the WAL and the storage engine, while availability is maintained through the in-memory TxMap and the data store. A key invariant governs this design: any entry in the TxMap must exist either in the WAL or in the data store, ensuring its durability; conversely, a valid data entry must be present in the TxMap (if dirty), in the data store, or in both (if clean). Concurrency control is handled by CCMap, and very importantly is not the concern of the data store. This decoupling opens the door to significant performance optimizations.

Threading and Cooperative Multitasking​

Modern CPUs offer massive parallelism, with machines routinely equipped with dozens or even hundreds of cores. To fully utilize such hardware, software must be architected like a distributed system—even within a single server node. Data Substrate embraces this philosophy from the ground up. As discussed earlier, threads and CCMap shards are the primary units of execution and data ownership, and message passing is the dominant communication model. By avoiding global synchronization and minimizing shared mutable states, this design drastically reduces lock contention and enables highly scalable execution, even within a single physical machine.

Hardware advancements—such as NVMe storage and RDMA networking—have fundamentally shifted the performance ratios of modern computer servers. Newer hardware often offers unprecedented I/O throughput. This contrasts sharply with traditional database designs, where I/O was often the primary bottleneck. In Data Substrate, we fully embrace this shift by adopting cooperative multitasking. This allows threads to issue non-blocking operations and efficiently switch between tasks, minimizing CPU waits and ensuring that each thread remains productive, continuously processing meaningful work.

Fault Tolerance, Recovery, and Transactions​

Fault tolerance is a critical aspect of any distributed system, and Data Substrate is designed to gracefully handle both hardware failures and software crashes without sacrificing consistency or availability.

We categorize failures and recovery into two types. The first type involves traditional distributed systems failures, such as data loss due to hardware faults like disk crashes. To mitigate these risks, Data Substrate relies on standard distributed systems techniques—primarily data replication. In cloud environments, many storage services are already replicated for fault tolerance. For instance, AWS EBS and S3 are designed to withstand server and storage failures, allowing us to treat them as effectively fail-safe. In environments where such services are not available—such as on-premises deployments—we can achieve similar level of resilience by replicating data using protocols like Raft. When a compute node fails, orchestration tools like Kubernetes can automatically restart the node elsewhere. In the absence of such tooling, failure detection mechanisms based on lease protocols are well-understood and straightforward to implement.

The second type of failure concerns database transaction execution. For example, a transaction might be interrupted midway due to a node crash, resulting in a situation where the transaction is either uncommitted or its local updates have not yet been fully installed. In such cases, the system must determine the transaction's status and either roll it back or complete its execution. This decision is guided by the WAL, which records the transaction's intent and progress. If the log indicates that the transaction was not committed, all partial updates can be safely discarded. If the commit was recorded but the effects were not fully applied, the system can replay the transaction's updates to bring the state to consistency.

While transaction recovery is a well-studied topic, distributed environments introduce additional complexity—such as handling orphaned locks or coordinating partial transaction states across nodes. We will not go into all the details in this article as a separate paper is under submission to discuss in detail the transaction commit protocols used in Data Substrate. We will sure to discuss this paper in another article when the double blind review period passes.

It's worth noting that by cleanly decoupling the concerns of distributed systems and database transactions, failure handling in Data Substrate becomes much easier to reason about. Historically, the database and distributed systems communities have often taken overlapping but subtly different views on core concepts—such as the definition of consistency and the purpose of logging—which can make discussions around distributed databases unnecessarily confusing. By adopting a "shared everything" approach and treating the system as one large computer, the Data Substrate architecture aims to bridge these perspectives and offer a unified conceptual framework for reasoning about consistency, recovery, and fault tolerance in modern distributed databases.

What about the CAP Theorem?​

No discussion of distributed storage systems is complete without addressing the CAP theorem. The theorem famously states that a distributed system cannot simultaneously guarantee consistency, availability, and partition tolerance. However, as ]researchers have pointed out, the theorem's practical implications are most relevant in the presence of network partitions. In normal operation—when the cluster is healthy and there are no partitions—a well-designed distributed system should provide both consistency and availability. Therefore, understanding and clearly defining system behavior during partitions is key to building predictable and resilient distributed databases.

A better understanding of distributed system behavior comes from the PACELC model, which extends the CAP theorem by considering system trade-offs both during and outside of network partitions. PACELC categorizes how a system prioritizes consistency (C) or availability (A) during a Partition (P), and how it balances latency (L) versus consistency (C) Else (E) when no partition exists. Data Substrate generally aligns with the PC/EC category, similar to fully distributed databases like CockroachDB and TiDB. This means that during a partition, the system prioritizes consistency over availability—it will halt database operations if it cannot ensure correctness. Outside of partitions, the system still favors consistency over latency: transactions must be fully committed and durable (e.g., WAL entries replicated and fsync'ed to disk) before acknowledging success to the client. This model ensures strong correctness guarantees, even at the cost of increased latency in normal operations.

Here we'd like to clarify a technical nuance to preempt potential concerns from careful readers. The timestamping mechanism used in Data Substrate allows databases built on it to achieve serializability but not necessarily linearizability. This is due to potential clock skew and the use of time-traveling mechanisms, which may assign commit timestamps that do not strictly reflect real-time order for transactions that are not causally related (of course, the DB is linearlizable for any individual data item). This design is intentional—it reduces transaction aborts and improves throughput while still providing strong, well-understood consistency guarantees. In the context of traditional database workloads, serializability is typically sufficient to ensure correctness, and the so-called strict serializability (i.e. serializability plus real-time ordering) is usually not necessary. That said, like most components in Data Substrate, the timestamping subsystem is modular and can be swapped out—for example, by introducing a centralized time oracle—to support linearizability if required by the application.

While the current implementation of Data Substrate follows a PC/EC configuration, this behavior is not hardwired—it can be adapted to suit different application needs. For example, in high-performance caching scenarios where strict durability and consistency are less critical, the system can be reconfigured to operate in a PA/EL mode, prioritizing availability during partitions and favoring lower latency over strong consistency during normal operation. This flexibility allows Data Substrate to support a wide range of use cases, from transactional databases with strong guarantees to low-latency, high-throughput NoSQL-style workloads, all within a unified architectural framework.

Summary and Future Topics​

Data Substrate represents a new architectural approach to building databases for the cloud era, and we believe it has the potential to significantly influence future database design. That said, we do not claim to have invented all the techniques discussed in this article. In fact, nearly all of them—or some variation—have appeared in prior systems or research. What sets Data Substrate apart is the careful and deliberate selection of these techniques, which we believe is the essence of sound architectural design. In a field as mature and intricate as database systems—with over half a century of evolution—it is unrealistic to expect a single, never-before-seen breakthrough to resolve all challenges. The real value lies in the thoughtful integration of proven ideas and the meticulous attention to detail that brings a coherent and effective system to life.

This article has explored several technical aspects of the Data Substrate architecture, but many important topics remain for future discussion. For example, We have not addressed how existing compute engines, such as those from MySQL or MongoDB, can be adapted to run on top of the Data Substrate. We have also not yet explored how this architecture naturally enables cloud-native advantages like Scale to Zero—where compute resources can be fully released during idle periods, incurring only storage costs—and diskless storage -- where object stores like S3 serve as the primary storage layer. These features offer significant reductions in total cost of ownership (TCO) in cloud environments. We will cover some of these topics in later posts.

This series of articles explore the motivations, technical foundations, and benefits of Data Substrate, providing a comprehensive understanding of how this architecture addresses the critical challenges facing modern data infrastructure in the AI age.

Some of the topics covered are rather heavy in technical jargons, and require a good understanding of database internal mechanisms to appreciate. We apologize in advance.

1. Data Substrate: Motivation and Philosophy​

This article introduces the core philosophy behind Data Substrate. We explore why traditional database architectures fall short in the AI era and present our vision for a new approach that treats the entire distributed system as a single, unified computer.

2. A Deeper Dive Into Data Substrate Architecture​

This technical deep-dive explores the architectural foundations of Data Substrate. We examine the key design decisions, abstractions, and technical choices that set Data Substrate apart from both classical and modern distributed databases.

3. The Benefits of Data Substrate Architecture​

This article examines the practical benefits and real-world implications of Data Substrate. We discuss how our design choices translate into concrete advantages for modern applications, particularly in cloud environments.

Why Data Substrate Matters​

Traditional database architectures were designed for a different era—one where data volumes were smaller, workloads were more predictable, and the demands of AI applications were unimaginable. Data Substrate represents a fundamental rethinking of database design, built from the ground up for the challenges and opportunities of the AI age.

By treating the distributed system as a single, unified computer, Data Substrate eliminates many of the complexities that have traditionally made distributed databases difficult to build, operate, and reason about. This approach enables:

• Modular architecture enables community collaboration and avoid reinventing the (many) wheels
• True scalability without sacrificing consistency
• Independent resource scaling for compute, memory, logging, and storage
• Better performance through optimized hardware utilization and innovative algorithm design
• Cloud-native features like auto-scaling and scale-to-zero
• Simplified development through familiar single-node programming models

Get Started with Data Substrate​

Ready to explore Data Substrate in action? Our open-source implementations are available on GitHub:

• EloqKV: A high-performance key-value store built on Data Substrate
• EloqSQL: A MySQL-compatible distributed SQL database
• EloqDoc: A document database for modern applications

Join our Discord community to connect with other developers and stay updated on the latest developments in Data Substrate technology.

The rise of autonomous, agentic AI applications has dramatically increased demands on underlying data infrastructure—highlighting the need for scalable, flexible, and cost-efficient solutions. While scalability and flexibility are widely discussed, cost-efficiency is harder to achieve. Many organizations only recognize the high cost of databases when faced with unexpectedly large cloud bills.

EloqData's Data Substrate architecture is built from the ground up to address these challenges. In this post, we explore how our innovative approach delivers substantial cost savings. To understand the context, let’s first examine the pain points of traditional cloud data infrastructures, particularly those relying on EBS or local NVMe storage.

Challenges with Traditional Cloud Architectures​

Traditional database architectures often assume that disk storage is durable and networking is free—assumptions that don't hold true in the cloud. Cloud environments introduce unique economic and operational characteristics. Key issues include:

• Expensive and Slow Persistent Storage: EBS offers durability, but at high cost and with limited performance.
• Ephemeral Local Storage: Local NVMe is fast but lacks durability, making it unsuitable for critical data.
• High Cross-AZ and Cross-Region Costs: Conventional architectures incur substantial network expenses due to required replication and frequent cross-zone or cross-region data transfers.

Because of these differences, traditional architectures often perform poorly after migrating to the cloud. The most widely adopted storage class in the cloud is now object storage, exemplified by AWS S3—a format that traditional databases were not designed to support.

Object Storage: The New Storage Standard in the Cloud​

Object storage provides key benefits that enable cost-effective scaling and separation of compute and storage resources. Its advantages include:

• High Durability: Object storage offers cross-AZ and cross-region replication by default, with SLAs up to 99.999999999% durability.
• Reduced Network Costs: Eliminates costly cross-AZ replication fees.
• Low Bandwidth Cost for Access: Even in failover scenarios, retrieving terabytes of data costs around $0.10 per million requests—far lower than other storage solutions.
• Lowest Cost Per GB: Object storage is approximately 3.5× cheaper than standard EBS.

Next-generation databases are already leveraging object storage. OLAP systems were early adopters, with technologies like Snowflake and formats like Iceberg and Parquet making full use of S3. More recently, streaming platforms such as StreamNative's Ursa engine have adopted object storage as the primary storage layer. Recognizing this trend, Confluent acquired WarpStream and introduced Freight Clusters that also utilize object storage. Vector and full-text search engines like Turbopuffer have followed suit.

However, using object storage directly introduces additional latency. While acceptable for OLAP workloads and asynchronous event-driven architectures, this latency is less suitable for OLTP workloads that demand low-latency responses. As a result, object storage is rarely used in OLTP database systems—until now.

EloqDoc: A Cost-Efficient, High-Performance Document Database​

Leveraging EloqData’s groundbreaking decoupled architecture, EloqDoc—a MongoDB-compatible JSON document store—effectively solves this challenge. EloqDoc fully embraces object storage, extending its benefits to document databases and unlocking elasticity, performance, and cost-efficiency. Unlike MongoDB Atlas, which incurs high costs from multiple replicas and cross-AZ traffic, EloqDoc uses object storage as its persistent layer.

Key advantages of EloqDoc include:

• Stateless Compute with Single Replica: Minimizes CPU and memory overhead while ensuring high availability and zero data loss.
• Object Storage as Primary Storage: Batches writes asynchronously to mitigate object storage latency, while recent data is safeguarded with EBS for durability.
• Local NVMe Caching: Delivers high-throughput, low-latency reads that outperform EBS IOPS by caching object storage locally.
• Scalable Redo Logs: Improves write performance cost-effectively using additional EBS volumes.
• MongoDB Compatibility and Distributed Transactions: Seamless migration from MongoDB, with robust transaction support powered by Data Substrate.

EloqDoc addresses the latency challenge by using local NVMe storage as an intelligent cache. Modern cloud-optimized instances offer significant local NVMe capacity (e.g., 3.5TB per 16-core server), enabling high-performance caching of active data while reducing costs.

Summary of the Video​

In Hubert’s presentation, we demonstrate how EloqDoc and StreamNative’s Ursa can be combined to build scalable, efficient data pipelines. Ursa is an innovative streaming storage engine that disrupts traditional models like Kafka, which rely on costly persistent storage and cross-AZ replication.

Key innovations of Ursa include:

• Diskless Architecture: Eliminates the need for expensive EBS storage.
• Stateless Brokers: Enables instant scaling and seamless failover without data movement.
• Leaderless Design: Avoids Kafka’s partition leader bottlenecks, reducing latency and cross-AZ costs.

Together with EloqDoc, Ursa enables the construction of a modern data pipeline that uses object storage as the primary data layer—fully realizing the potential of cloud-native infrastructure.

Tour Begin​

Imagine you’re building an agentic AI application—a chatty, autonomous assistant that roams the digital world, helping users with everything from travel tips to life advice. Your agent is smart, proactive, and powered by a large language model (LLM), but there’s one catch: it needs to remember everything. Every witty remark, every user request, every "Aha!" moment—it all needs to be stored somewhere reliable. That’s where EloqKV comes in, and today, I’m taking you on a wild tour of how this decoupled architecture saved my AI agent (and my sanity) as it grew from a humble prototype to a bustling, scalable powerhouse.

Starting Simple with a Memory Cache​

When I first launched my agent—let’s call it "TourBuddy"—it was a lightweight conversationalist, guiding users through imaginary trips. I needed a quick way to store chat history, so I turned to EloqKV as a simple memory cache. Deploying it was a breeze with a single-node setup using this Docker Compose file:

version: '3.8'
networks:
  eloqnet:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16
services:
  eloqkv:
    image: eloqdata/eloqkv_full:latest
    container_name: eloqkv
    networks:
      eloqnet:
        ipv4_address: 172.20.0.4
    ports:
      - '6379:6379'
    command: /EloqKVRocksDB/bin/eloqkv --config /EloqKVRocksDB/conf/eloqkv.ini --ip=172.20.0.4 --port=6379 --ip_port_list=172.20.0.4:6379
    working_dir: /EloqKVRocksDB
    tty: true
    stdin_open: true

Boom! TourBuddy’s chat history was cached in memory, lightning-fast and ready to recall at a moment’s notice. It felt like giving my agent a short-term memory upgrade—perfect for its early days.

Scaling Up with Persistent Storage​

As TourBuddy gained fans (who doesn’t love a chatty travel guide?), the chat history ballooned. Storing everything in memory started to feel like stuffing a suitcase with too many souvenirs—expensive and impractical, especially since older messages were rarely accessed. Luckily, EloqKV has a persistent storage trick up its sleeve. With one tiny tweak to the config, I enabled RocksDB as a backend:

version: '3.8'
networks:
  eloqnet:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16
services:
  eloqkv:
    image: eloqdata/eloqkv_full:latest
    container_name: eloqkv
    networks:
      eloqnet:
        ipv4_address: 172.20.0.4
    ports:
      - '6379:6379'
    command: /EloqKVRocksDB/bin/eloqkv --config /EloqKVRocksDB/conf/eloqkv.ini --enable_data_store=true --ip=172.20.0.4 --port=6379 --ip_port_list=172.20.0.4:6379
    working_dir: /EloqKVRocksDB
    tty: true
    stdin_open: true

Now, historical chats lived happily on disk, freeing up memory for hot data while keeping costs down. TourBuddy could still reminisce about that epic Tokyo itinerary from months ago without breaking the bank.

Ensuring Durability When Disaster Strikes​

Everything was smooth sailing until one day, TourBuddy forgot my latest adventure—my new location had vanished into thin air! After some frantic debugging, I realized the cache had lost data during a hiccup. Durability became my new obsession. Thankfully, EloqKV’s got a Write-Ahead Log (WAL) feature to make data loss a thing of the past. Another quick tweak:

version: '3.8'
networks:
  eloqnet:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16
services:
  eloqkv:
    image: eloqdata/eloqkv_full:latest
    container_name: eloqkv
    networks:
      eloqnet:
        ipv4_address: 172.20.0.4
    ports:
      - '6379:6379'
    command: /EloqKVRocksDB/bin/eloqkv --config /EloqKVRocksDB/conf/eloqkv.ini --enable_data_store=true --enable_wal=true --ip=172.20.0.4 --port=6379 --ip_port_list=172.20.0.4:6379
    working_dir: /EloqKVRocksDB
    tty: true
    stdin_open: true

With WAL enabled, every chat message was safely logged before being committed. TourBuddy never forgot a destination again, and I slept better knowing my data was durable.

Decoupling for Scalability​

TourBuddy’s fame grew, and so did the traffic. Users flooded in, asking for tips from Paris to Patagonia, but the app’s response time crept up. Customers grumbled, and I traced the culprit to a rising cache miss rate—my memory buffer was too small. Scaling the entire cluster felt like overkill (moving terabytes of disk data? No thanks!). Enter EloqKV’s decoupled architecture: I could scale the compute/memory cluster separately from storage. Here’s how I set up a 3-node compute/memory cluster with a Cassandra storage backend:

Below is a snippet of the docker-compose.yaml file. For the complete decoupled_storage configuration, please refer to the GitHub repository.

services:
  cassandra:
    image: cassandra:4.1.8
    container_name: cassandra
    networks:
      eloqnet:
        ipv4_address: 172.20.0.2
    ports:
      - '9042:9042'
    environment:
      - CASSANDRA_CLUSTER_NAME=eloq_kv
      - CASSANDRA_DC=DC1
    healthcheck:
      test: ['CMD', 'cqlsh', '-e', 'describe keyspaces']
      interval: 10s
      timeout: 5s
      retries: 20
  eloqkv1:
    image: eloqdata/eloqkv_full:latest
    container_name: eloqkv1
    networks:
      eloqnet:
        ipv4_address: 172.20.0.3
    ports:
      - '6379:6379'
    depends_on:
      cassandra:
        condition: service_healthy
      eloqkvbootstrap:
        condition: service_completed_successfully
    command: /EloqKVCassandra/bin/eloqkv --config /EloqKVCassandra/conf/eloqkv.ini --ip=${NODE1_IP} --port=${NODE1_PORT} --ip_port_list=${IP_PORT_LIST} --enable_wal=${ENABLE_WAL} --enable_data_store=${ENABLE_DATA_STORE} --checkpoint_interval=${CHECKPOINT_INTERVAL} --cass_hosts=172.20.0.2 --cass_password=cassandra --cass_user=cassandra
    working_dir: /EloqKVCassandra
  eloqkv2:
  eloqkv3:

In seconds, I had an elastic memory pool to handle hot chat data, slashing read latency. TourBuddy was back to its zippy self, delighting users with instant responses.

Boosting Write Throughput on a Budget​

Success brought a new challenge: write latency spiked as users chatted up a storm. My 3-node compute cluster was maxed out, and my budget screamed "No more high spec nodes!" EloqKV’s decoupled WAL log service came to the rescue. I added standalone low spec log nodes, each with its own cheap disk, to boost write throughput:

Below is a snippet of the docker-compose.yaml file. For the complete decoupled_log_storage configuration, please refer to the GitHub repository.

services:
  cassandra:
    image: cassandra:4.1.8
    container_name: cassandra
    networks:
      eloqnet:
        ipv4_address: 172.20.0.2
    ports:
      - '9042:9042'
    environment:
      - CASSANDRA_CLUSTER_NAME=eloq_kv
      - CASSANDRA_DC=DC1
    healthcheck:
      test: ['CMD', 'cqlsh', '-e', 'describe keyspaces']
      interval: 10s
      timeout: 5s
      retries: 20
  eloqkv1:
    image: eloqdata/eloqkv_full:latest
    container_name: eloqkv1
    networks:
      eloqnet:
        ipv4_address: 172.20.0.3
    ports:
      - '6379:6379'
    depends_on:
      cassandra:
        condition: service_healthy
      eloqkvbootstrap:
        condition: service_completed_successfully
    command: /EloqKVCassandra/bin/eloqkv --config /EloqKVCassandra/conf/eloqkv.ini --ip=${NODE1_IP} --port=${NODE1_PORT} --ip_port_list=${IP_PORT_LIST} --enable_wal=${ENABLE_WAL} --enable_data_store=${ENABLE_DATA_STORE} --checkpoint_interval=${CHECKPOINT_INTERVAL} --cass_hosts=172.20.0.2 --cass_password=cassandra --cass_user=cassandra
    working_dir: /EloqKVCassandra
  eloqkv2:
  eloqkv3:
  logserver1:
    image: eloqdata/eloqkv_full:latest
    container_name: logserver1
    networks:
      eloqnet:
        ipv4_address: 172.20.0.11
    ports:
      - '9001:9001'
    command: sh -c "export LD_LIBRARY_PATH=/LogService/lib:${LD_LIBRARY_PATH};/LogService/bin/launch_sv -conf=172.20.0.11:9001,172.20.0.12:9002,172.20.0.13:9003 -start_log_group_id=0 -node_id=0 -storage_path=/wal_eloqkv1"
    working_dir: /LogServer
    tty: true
    stdin_open: true
  logserver2:
  logserver3:

Write latency plummeted, and TourBuddy handled the chatter like a champ—all without breaking the bank.

The Cherry on Top—More Features to Explore​

Now, my EloqKV cluster was a lean, mean, fully elastic machine. I could kill a compute node, a log node, or even a storage node, and TourBuddy kept humming along, thanks to EloqKV’s high availability magic. No single point of failure could stop my agent from sharing travel tales.

EloqKV isn’t done surprising me. It’s packed with goodies like cross-shard LUA scripting, Multi/Exec commands, and explicit transactions with Begin/Commit/Rollback APIs. Dive in and explore!

Wrapping Up the Tour​

Building TourBuddy with EloqKV was like taking a trusty companion on a journey. From a simple cache to a decoupled, elastic database, EloqKV adapted to every twist and turn, keeping my agentic AI app fast, durable, and cost-effective. Whether you’re caching hot data, persisting history, or scaling for the masses, EloqKV’s decoupled architecture has your back. So, grab the YAML snippets, fire up Docker, and take your own agent on a tour—you’ll be amazed at where it can go!

LLM-powered Artificial Intelligence (AI) applications are driving transformative changes across industries, from healthcare to finance and beyond. We are rapidly entering the Agentic Application Age, an era where autonomous, AI-driven agents not only assist but actively make decisions, manage tasks, and optimize outcomes independently.

However, the backbone of these applications—the data infrastructure—faces immense challenges in scalability, consistency, and performance. In this post, we explore the critical limitations of current solutions and introduce EloqData’s innovative approach specifically designed to address these challenges. We also share our vision for an AI-native database, purpose-built to empower the Agentic Application Age, paving the way for smarter, more autonomous, and responsive AI applications in the future.

Why We Need New Data Infra in the AI Age​

AI applications are inherently data-intensive. To build such an application, developers must design a system capable of storing, processing, and retrieving diverse data types to fuel machine learning models, ensure real-time decision-making, and support continuous improvement. A typical AI application integrates several components:

• Data Ingestion and Preprocessing: Collect raw data from various sources and transform it into formats suitable for analysis.
• Model Training and Fine-Tuning: Train AI models on structured and unstructured data to improve performance on specific tasks.
• Inference and Deployment: Use trained models to make predictions or automate workflows.
• Feedback Loops: Capture user interactions and outcomes to retrain models for improved accuracy and adaptability.

AI applications deal with diverse data types, each with specific storage and retrieval requirements:

• Structured Data: Typically stored in relational databases for tasks requiring tabular representations (e.g., user profiles, transaction histories).
• Unstructured Data: Includes text, images, and videos, stored in document databases or object storage systems.
• Embeddings: Vector representations of unstructured data for similarity search, often stored in vector databases.
• Graphs: Relationships between entities (e.g., social networks, knowledge graphs), stored in graph databases.
• Metadata: Information about data objects, stored in SQL or document databases for indexing and retrieval.

Challenges​

With platforms like LangChain and LlamaIndex, developers can seamlessly integrate multiple databases into AI workflows, enabling efficient storage and retrieval of multi-modal data. However, this approach often causes several challenges.

The first challenge is management complexity. AI applications typically need multiple databases to manage different data types. Each database comes from a different vendor, requires specialized skill sets, and has unique backup and failure recovery procedures. Backup strategies and failure recovery operations vary significantly between database systems, complicating overall infrastructure management. The additional effort required to maintain consistent backup routines, implement disaster recovery protocols, and ensure high availability across diverse database environments significantly increases operational complexity.

The second challenge is related to flexibility and agility. Modern databases are often designed to handle a specific type of workload under a specific range of performance envelopes. For example, Redis is designed to address latency issues, by sacrificing persistency, while PostgreSQL guarantees durability by trading-off scalability. Unfortunately, the landscape of AI applications is still rapidly changing. There is simply no “standard architecture” to handle typical AI application needs. Choosing a simple data architecture might involve a total revamp of the entire data pipeline as the application evolves.

The third challenge, which might be much more fundamental, is the difficulty of ensuring data consistency across multiple databases. AI applications often require combining data from diverse sources, which introduces transactional challenges. Databases traditionally provides ACID transactional semantics to ease the complexity of application development. This valuable property is lost when a query needs to be carried out by multiple databases. Without the ACID property, application developers are left to deal with the data consistency issues in application code, which is often error prone.

Consider building a simple Retrieval-Augmented Generation (RAG) application to handle, say, financial reports of the public companies. RAG requires a document to be chunked and vectors of each chunk to be added to a vector database, while the document itself stored in a document DB. If the document is added while the insertion of the vectors failed, we may not be able to retrieve the document. On the other hand, if the vectors are added while the document failed to be inserted, we may get “broken links”. After a while, the developers may feel that knowledge graph of the companies might be a helpful source of information, and GraphRAG might increase the recall quality. Therefore, we may need to add a graph database to the pipeline. Soon, real-time news and social media might be interesting data sources, so we need to quickly ingest these feeds. Shall we use a streaming platform? As one can imagine, the data pipeline quickly grows out of hand. How to guarantee consistent user experience when data is constantly being added to multiple complicated and independent database systems? How to make joint queries when the databases are constantly out of sync?

How EloqData can Help​

Some argue that eventual consistency is sufficient for AI applications. While this is true in certain scenarios, we believe a one-stop solution with full ACID transactions and lower costs is not only desirable but essential.

Why Eventual Consistency Falls Short

• Operational Complexity: Managing retries, failures, and inconsistencies in an eventual consistency model adds significant development and maintenance burden.
• Scalability Risks: As AI applications grow, maintaining consistency across multiple databases becomes increasingly challenging.

The Promise of a Unified Solution​

EloqData is redefining data infrastructure for AI applications with a one-stop solution ConvergedDB, that offers high performance, scalability, and full ACID transactions. This breakthrough eliminates the need for complex middleware like Kafka and allows developers to focus on building applications rather than managing infrastructure. At the heart of this solution is our innovative Data Substrate architecture.

Looking into the Future​

While current RAG implementations may not demand strong ACID guarantees, the future of AI applications points toward more complex, collaborative agent architectures. Imagine a network of AI agents collaborating on tasks ranging from financial transactions to regulatory compliance. These scenarios will undoubtedly require strong ACID properties to ensure consistency and reliability.

Future-Proof Your Infrastructure with EloqData​

By choosing EloqData from day one, organizations can:

• Eliminate Scalability Concerns: EloqData’s architecture ensures seamless scalability, whether your business grows vertically or horizontally.
• Ensure Consistency: As multi-agent architectures become the norm, EloqData’s cross-model transactional guarantees will prove invaluable.
• Maintain High Performance: With no performance degradation, EloqData delivers a robust solution for the most demanding AI applications.

The future of AI applications requires data infrastructure that is scalable, consistent, and performant. EloqData’s one-stop multi-model database ConvergedDB, powered by its Data Substrate architecture, delivers on all three fronts, enabling developers to focus on innovation rather than infrastructure.

ConvergedDB is launching soon! Your feedback is greatly appreciated—stay tuned!

Had we launched our product a decade ago, using C++ would have been an obvious and unremarkable choice. However, it's 2024, and the landscape has changed. Today, languages like Rust, Zig, and other type-safe options like Golang are considered modern and trendy for systems programming. So, when we chose C++, a language that some might view as outdated or less "cool", or even bug-prone and "unsafe", it’s natural for people to wonder why.

In this article, we’d like to share the thought process behind our decision to choose C++ over some of the newer, more fashionable languages, the historical lessons we drew inspiration from, and the upcoming progress we expect in the future.

Choosing a Programming Language Is Important​

Selecting the right programming language is crucial for any software project, but it becomes even more significant for complex systems software such as databases. The choice of language influences various aspects, including performance, ease of development, and maintainability. In a domain where efficiency and reliability are paramount, the programming language serves as the foundation upon which the entire system is built.

For databases, the implications of this choice are profound. A database must be capable of handling vast amounts of data while providing fast query responses and ensuring data integrity. These requirements necessitate a language that not only excels in performance but also allows for scalable and efficient development practices. Additionally, databases often undergo continuous development and enhancement over decades, making maintainability a critical factor. A well-chosen language can simplify the process of updating and expanding the software's features over time, ensuring that it remains relevant and effective in an ever-evolving technological landscape.

Consider the Hadoop big data stack, which is predominantly built on the Java Virtual Machine (JVM). While Java and JVM ecosystems have been one of the most popular programming language families and were lauded for their portability and rich features, in retrospect, this choice may not have been without controversy. The performance and memory overhead of the JVM, particularly issues related to garbage collection, has caused numerous challenges for developers. Indeed, RedPanda and ScyllaDB are notable examples of rewriting mature, widely-used Java-based frameworks—Kafka and Cassandra, respectively—in C++ from scratch to avoid the JVM penalties.

Another important consideration is the popularity of the programming language and the availability of developers familiar with it. For instance, Spark and Kafka are developed using Scala, while Couchbase and Rabbitmq are in Erlang. Although these languages offer robust features and capabilities, they are not as widely adopted as other programming languages. This relative lack of popularity can create challenges when it comes to larger-scale developer engagement and finding experienced programmers. Toolchain support is generally not on par with more popular programming languages. A less common language may result in increased difficulty in recruiting talent, slowing down development processes and limiting community support for troubleshooting and innovation.

By the late 2010s, Rust emerged as one of the leading programming languages for developing database software. Newer projects such as TiDB, RisingWave, DataFusion, and NeonDB are prominent examples that leverage Rust's capabilities to build efficient and high-quality databases. Notably, RisingWave even published a blog post detailing their decision to discard ten months of work in C++ to rewrite their entire codebase in Rust. Given that EloqData began its journey around 2021, when Rust was already well-established as a robust programming language with excellent features for building safe and performant databases, one might wonder why we opted for C++ instead.

Building a Database from Scratch in C++ in 2024​

When we began our project, we were keenly aware that Rust was a highly competitive language for building the foundations of our database. Our eventual decision to choose C++ was based on three main factors.

The first strength of C/C++ lies in its database ecosystem support. Most existing and popular databases are developed in C/C++, providing a wealth of resources and innovations we could leverage. Our Data Substrate technology aims to create a unified, modular architecture that can capitalize on these existing resources while avoiding the need to reinvent the wheel. Although Rust offers good interoperability with C/C++, its memory management model and certain safety restrictions can complicate integration with many established projects.

Another advantage of C++ is its extensive support for foundational libraries. Since most operating systems and lower-level drivers are written in C or C++, bindings for these languages are often the native and best-supported APIs. Performance-focused libraries for IO and networking, such as DPDK, RDMA and liburing, as well as memory management tools such as mimalloc, are developed with C/C++ and provide native support. In contrast, other languages typically require additional layers to effectively utilize these libraries. We anticipate that this trend will continue, with newer hardware and OS abstractions favoring C/C++ support first and foremost.

The third advantage of C++ is its longevity and mature toolchain. Infrastructure software often requires continuous updates and improvements over several decades. For instance, Oracle Database is over 45 years old, while MySQL and PostgreSQL have been around for around 30 years. Even relatively newer systems like Cassandra, MongoDB, and Redis are over 15 years old. To develop a reliable infrastructure solution, we need to be prepared to maintain the codebase for potentially half a century. A lot can change in the tech world over such a long period—consider that 20 years ago, Perl was a highly popular language, and Delphi was more widely used than Python.

When building long-lasting software, it’s crucial to consider the long-term survivability of the programming language, such as continued improvements on the compilers, up-to-date library developments, and modern IDE, debugger and profiler support. In this respect, C++ is a much safer bet. Its extensive history, active development community, and proven resilience over time give us confidence that it will continue to be relevant and well-supported for decades to come.

Of course, C++ comes with its share of legacies that can present challenges compared to many modern languages. To maximize productivity on C++ projects requires a certain level of discipline. While we won’t elaborate on the myriad best practices we’ve implemented to mitigate some of C++'s shortcomings — since these are well documented and widely discussed elsewhere — we acknowledge that effective use of the language demands a strong commitment to coding standards and testing methodologies. In particular, the most harsh arguments against using C++, i.e. memory unsafeness, can be significantly mitigated when developing with a certain modern subset of the C++ language.

Going Forward​

At EloqData, we strongly adhere to a modular design philosophy, as we are committed to building a lasting system that will support decades of continued improvements. We recognize that effective API interface design is crucial for enhancing software development productivity and maintainability. This principle is not only reflected in the overall architecture of Data Substrate, which accommodates various query and storage engines, but is also embedded throughout our software development process. We anticipate that innovations will continue to emerge—such as improved memory allocators, more efficient RPC libraries, and optimized hash-table implementations—and we aim to leverage these innovations as they become available in the future.

It is relatively straightforward for us to experiment with other programming languages in our projects when appropriate. We are eager to replace certain modules with components implemented in type-safe languages like Rust where it makes sense. Rust is an exceptional language with a strong following in the systems community, and we aim to utilize it more in many of our upcoming projects.

In contrast to many startup companies that emphasize rapid iteration, quick feedback loops, and fast prototyping, EloqData has taken a different approach. We place a stronger emphasis on doing things right from the onset to avoid future technical debts. While this focus may slow us down a bit, we believe it is a necessary investment. However, it’s worth noting that avoiding future debts is futile if the product and technology lack a viable future to begin with. Ultimately, whether we made the right choice will take time to determine. Regardless, we take pride in the decisions we’ve made and look forward to seeing how our efforts can help our customers tackle their most challenging data problems.

In this blog, we evaluate small scale clusters to show the behavior of EloqKV accross servers. Scalability in a larger scale cluster with different number of servers will be evaluated at a later blog. All benchmarks were conducted on AWS (region: us-east-1) EC2 instances, with Ubuntu 22.04. In all tests, we use EloqKV version 0.7.4.

Transaction in EloqKV​

Fifteen years ago, the esteemed database researcher and Turing Award Winner Mike Stonebraker famously wrote an article in Communications of the ACM declaring, "No ACID Equals No Interest" for enterprise users. Unfortunately, due to the high costs associated with distributed transactions, many distributed databases avoid full transaction support in favor of better performance. For example, while Redis supports limited transaction operations in single-node mode, it does not support transactions across servers in a cluster.

Thanks to our Data Substrate architecture, EloqKV is a fully ACID-compliant database. In addition to offering durability, which was discussed in a previous blog post, EloqKV's transaction capabilities support the Redis WATCH, MULTI, DISCARD, and EXEC commands even in a cluster.

In this blog, we focus on benchmarking the MULTI and EXEC commands for PUT/GET operations—specifically, performing a series of read and write operations atomically across a cluster of servers. We believe this workload provides valuable insights into the costs associated with distributed transactions. Although EloqKV also supports WATCH, DISCARD, and Lua scripting, creating standard representative test cases for these features is more challenging.

In EloqKV, by default, the ACI (Atomicity, Consistency, Isolation) part of ACID is always enabled. No configuration changes are required to enable MULTI and related commands in a cluster. EloqKV supports different levels of isolation, with the default being Repeatable Reads, which is the isolation level used in the experiments discussed in this blog.

In Repeatable Reads isolation level, reads and writes are about the same complexity. For read requests, each key must be read and then validated during the transaction commit phase to ensure that no modification happened in between reading and commiting. For write requests, a write lock must be acquired for each key and then released at the commit phase. Both require extra round-trips to accomplish and thus more expensive than non-transactional operations. In EloqKV, a single-key operation is executed as a transaction and will not incur additional overhead, regardless of the isolation level.

Experiments​

In the first experiment, we compare EloqKV and Redis in batch mode across different workloads. We focus on two batch modes:

1. Pipeline: In this mode, the client sends multiple commands to the server without waiting for responses to previous commands. The server processes these commands sequentially and returns all the responses at once. This batching approach significantly reduces network communication overhead, especially when executing many commands. Notice that each command in the pipeline is executed independently, with potentially other commands executed in between. However, we do enforce that the commands for any given key is executed in the order they appear in pipeline.

2. MULTI / EXEC: This mode ensures that a group of commands is executed as a single atomic operation, meaning either all commands are executed or none are. Please note that the Redis MULTI/EXEC command without WATCH normally cannot fail because Redis execute commands in a single thread on a single server, whereas EloqKV, being multi-threaded, can fail a transaction and roll back due to concurrent transaction conflicts.

Redis does not support Multi Exec in cluster mode if keys in a single batch do not fall on the same shard. To work around this, users must use hashtags to ensure keys involved in a transaction are located on the same shard. This can be cumbersome and often cause load imbalance. Moreover, for Redis, Pipeline support in cluster mode is client dependent. It is not a feature supported on all Redis clients. EloqKV, on the other hand, does not have these limitations. Transactions and Pipelines work on a cluster of nodes just as on a single node. Though EloqKV does support hashtags to colocate keys in order to reduce network overhead.

In the following experiments, EloqKV operates in pure memory mode, with persistent storage and write-ahead-log (WAL) disabled.

Hardware and Software Specification​

Server Machine:

Experiment:​

We developed a new benchmarking tool, eloq_benchmark, specifically to test the transaction performance of Redis and EloqKV, as memtier_benchmark does not support Multi Exec. You can download eloq_benchmark here

We run eloq_benchmark with the following configuration:

eloq_benchmark --h $server_ip --p $server_port --numKeys=$keynum --numConnections=$conn --getRatio=$ratio --opType=$optype --batchSize=$batchsize --numTestOps=$testops

• --numKeys: Number of entries, which is set to 1000000.

• --numConnections: Number of concurrent connnections, which is set to 256 for single-node and 768 for three-node cluster.

• --numTestOps: Number of test operations, which is set to 5000000.

• --getRatio: Set it to 0 for write-only workload, 0.5 for mixed workload and 1 for read-only workload.

• --opType: Set batch mode, set it to pipeline for pipeline mode, set it to tx for MutilExec atomic mode.

• --batchSize: Number of Put/Get operations on random keys in a batch, which we set to 6.

Results​

Below are the performance results of batch mode of Redis and EloqKV among various workload. Note that the number of operations of PUT/GET in a batch is fixed at 6.

X-axis: Represents the different workload types (read/write/mixed) used in the benchmark, simulating a range of real-world scenarios.

Y-axis: Throughput in Thousand OPS (Operations Per Second) for the batches. This number should be multiplied by 6 (batch size) to obtain the total KV operations.

On a single node, EloqKV significantly outperforms Redis in both pipeline and Multi Exec modes. With a fixed batch size of 6 keys, EloqKV achieves a throughput exceeding 200 million KV operations per second (KPS) in both modes on a single server. Multi Exec is slower than Pipeline due to additional book-keeping needed for atomic operations.

The throughput of a three-node EloqKV cluster is lower than that of a single-node EloqKV. In Pipeline mode, this is because of the additional network round trips. For Multi Exec, additional operations are needed for lock acquisition and releasing. Even so, the performance is quite respectable.

Evaluate the Impact of Batch Size​

Transaction size affects the efficiency of distributed transactions. In this experiment, we test eloq_benchmark with batch sizes ranging from 1 to 6.

Result​

Below are the performance results of EloqKV Multi Exec command with different batch size among various workload.

X-axis: Represents the different workload types (read/write/mixed) used in the benchmark, simulating a range of real-world scenarios.

Left Y-axis: Throughput in Thousand OPS (Operations Per Second), shown as the bars.

Right Y-axis: Percentage of Transaction Retries, shown as the dashed lines.

As expected, EloqKV’s transaction throughput decreases as the batch size increases. This is because larger batch sizes introduce additional work. Notice that the total KV operations carried out in the cluster must be multiplied by the batch size. Currently, EloqKV does not perform "query optimization" within a transaction. To guarantee transactional semantics, the operations in a batch are executed sequentially. In the future, we may optimize this by allowing some operations within a batch to be executed in parallel.

In our workload, we selected several random keys from a range to perform PUT/GET operations. As mentioned earlier, the key range was set to 1,000,000 in all experiments. We observed that transaction retries increase with larger batch sizes in both mixed and write-only workloads. This is due to the higher likelihood of transaction conflicts as the batch size grows. Reducing the level of concurrency or expanding the key range can help mitigate these conflicts. Additionally, the more concurrent writes, the more likely conflicts will occur. For read-only workloads, no transaction conflicts arise, so no transaction retries occurred.

All benchmarks were conducted on AWS (region: us-east-1) EC2 instances running Ubuntu 22.04. Workloads were generated using the memtier-benchmark tool. In all tests, we use EloqKV version 0.7.4.

Understanding Durability​

Durability is crucial for data stores, especially in applications where data loss during server failures is unacceptable. Most persistent data stores achieve durability through Write-ahead Logging (WAL), ensuring that data is written to disk before responding to users. Additionally, many stores periodically checkpoint the data into a more compact form, allowing the log to be truncated.

However, many key-value (KV) caches prioritize performance over durability. For example, Redis uses an Append Only File (AOF) to achieve some level of durability by fsyncing data to a log file, either periodically or after each write command. Redis's single threaded architecture significantly increases latencies of other operations if a write operation blocks the thread, therefore, AOF sync writing for every write command is rarely deployed in practice. Consequently, DragonflyDB forgoes AOF altogether due to lack of demand, instead offering only periodic checkpointing similar to Redis's RDB.

EloqKV is a fully ACID-compliant database, providing full data durability through WAL. Leveraging our decoupled Data Substrate architecture, the WAL for an EloqKV server can either be embedded within the same process or run as a separate LogService. The log can be replicated across multiple machines or Availability Zones, can scale using multiple disk devices, and can utilize tiered storage to archive older data on more cost-effective storage.

However, we recognize that durability may not always be necessary for all applications. Currently, EloqKV, persistency can be turned on and off through a configuration. In a future release, EloqKV durability can be enabled on a per-database basis so that durable and non-durable workload can co-exists on a single EloqKV instance. When durability is disabled, EloqKV avoids the overhead associated with durability, delivering uncompromised performance as demonstrated in a previous blog post. In this blog, we evaluate EloqKV with durability enabled.

Comparing with Kvrocks​

In the first experiment, we compare EloqKV with Apache Kvrocks, a Redis-compatible NoSQL database that supports persistence. We evaluate the performance of EloqKV and Kvrocks under write-intensive and mixed workloads. To ensure data durability, we enable fsync Write-Ahead Logging (WAL) for both databases. For EloqKV, both the transaction service and log service are deployed on the same node (c7gi.8xlarge). To fully utilize available disk IO, we start two LogService processes to write WAL logs in EloqKV.

Hardware and Software Specification​

Server Machine:

For EloqKV, we enable persistent storage and turn on WAL (Write-Ahead Logging).

# set it to on to turn on persistent storage
enable_data_store=on
# set it to on to turn on WAL
enable_wal=on

For Kvrocks, we mainly changed two configuration options.

# If yes, the write will be flushed from the operating system
# buffer cache before the write is considered complete.
# If this flag is enabled, writes will be slower.
# If this flag is disabled, and the machine crashes, some recent
# writes may be lost.  Note that if it is just the process that
# crashes (i.e., the machine does not reboot), no writes will be
# lost even if sync==false.
#
# Default: no
# rocksdb.write_options.sync no
rocksdb.write_options.sync yes

# The number of worker's threads, increase or decrease would affect the performance.
# workers 8
workers 24

Disk performance plays a critical role in write-intensive workloads. Therefore, we conducted benchmarks using both local SSDs and Elastic Block Store (EBS). Local SSDs offer low latency and high IOPS, making them ideal for high-performance needs. However, in a cloud setup data on local SSD may get lost if the virtual machine (VM) is stopped. On the other hand, EBS provides high availability, allowing the volume to be attached to a new VM if the original VM fails. Moreover, EBS is elastic, allowing precise control over number of volumes and their sizes. In our case, a 50GB EBS gp3 volume is plenty for our WAL needs. Such a volume only cost $4 per month while providing 3000 IOPS and 125 MB/s throughput. Given the distinct advantages and limitations of local SSDs and EBS, we conducted our experiments with both.

We run memtier_benchmark with the following configuration:

memtier_benchmark -t $thread_num -c $client_num -s $server_ip -p $server_port --distinct-client-seed --ratio=$ratio --key-prefix="kv_" --key-minimum=1 --key-maximum=5000000 --random-data --data-size=128 --hide-histogram --test-time=300

• -t: Number of threads for parallel execution, which we set to 80.

• -c: Number of clients per thread. We set it to 5, 10, 20, 40 to evaluate different concurrency levels. This resulted in total concurrency values of 400, 800, 1600, and 3200, calculated as thread_num × client_num.

• --ratio: Set:Get ratio is set to 1:0 for write-only workload, and 1:10 for mixed workload.

Results​

Below are the results of the write-only workload.

X-axis: Represents the varying concurrencies (thread_num × client_num), simulating different levels of concurrent database access.

Left Y-axis: Throughput in Thousand OPS (Operations Per Second).

Right Y-axis: Average latency in milli seconds (ms).

EloqKV significantly outperforms Kvrocks on both EBS and local SSD. On EBS, EloqKV achieves a write throughput that is 10 times higher than Kvrocks, while on local SSD, it is 2-4 times faster. This performance improvement is due to EloqKV's architecture, which decouples transaction and log services, allowing multiple log workers to write Write-Ahead Logs (WAL) and perform fsync operations in parallel, thereby enhancing overall throughput. Additionally, EloqKV maintains significantly lower latency compared to Kvrocks, even under high concurrency.

Below are the results of the mixed workload.

Results show that EloqKV outperforms Kvrocks on mixed workloads as well. EloqKV maintains a read latency of less than 1 ms even under a heavy mixed workload with nearly 900K OPS. In contrast, Kvrocks on EBS exhibits significantly higher latencies, with both read and write latencies exceeding 10 ms even at relatively low concurrency, and rising to over 50 ms as concurrency increases. Even on local SSDs, Kvrocks' read latency remains much higher than that of EloqKV. This demonstrates that EloqKV can sustain low read latency even when the cluster is under a heavy write workload.

Experiment II: Scaling Disks of WAL​

EloqKV's decoupled WAL log service can deploy multiple log workers writing WAL logs to multiple disks. Since WAL logs can be truncated once a checkpoint is completed, the required disk size is often quite small. Kvrocks does not support writing redo logs across multiple disks, so this experiment is conducted with EloqKV only.

Server Machine:

We benchmarked EloqKV with different numbers of WAL disks and varying thread counts using the following command:

memtier_benchmark -t $thread_num -c $client_num -s $server_ip -p $server_port --distinct-client-seed --ratio=1:0 --key-prefix="kv_" --key-minimum=1 --key-maximum=5000000 --random-data --data-size=128 --hide-histogram --test-time=300

• -t: Number of threads for parallel execution, which we have set to a fixed value of 80.
• -c: Number of clients per thread. We configured it to 40, 60 and 80, this resulted in total concurrency values of 3200, 4800 and 6400, calculated as thread_num × client_num.

In this experiment, we have higher concurrency compared with previous experiments due to increased latency caused by seperating LogService from TxService.

Results​

The following graph shows how disk count impacts the performance of EloqKV.

X-axis: Represents the varying thread numbers employed during the benchmark, simulating different levels of concurrent database access.

Left Y-axis: The throughput in Thousand Operations Per Second (KOPS)

Right Y-axis: The average Latency in ms.

We can observe that as the number of disks increases, the throughput scales near linearly when the disk count is 1, 2, and 4, with a corresponding decrease in latency. Adding even more disks continues to boost throughput, but at a slower rate. For 6 and 8 disks, the throughput levels off and remains nearly the same even under high concurrency.

Experiment III: Scaling Up TxServer​

As observed previously, throughput does not increase further when the number of disks exceeds six. This indicates that logging is no longer the bottleneck; to achieve even higher throughput, scaling up the CPU in TxServer could be the next step. Obviously, scaling-out could be another option, but we will leave that to another blog.

Server Machine:

Result​

The following graph shows how the number of CPU cores affects the performance of EloqKV as we have many disks providing logging IOPs.

Adding more disks beyond 8 on a 32-vcore CPU does not significantly increase throughput. By scaling up the CPU of TxServer from 32 to 48 vcores, we can achieve a notable increase in throughput and a decrease in latency. Under heavy concurrency, latency decreases significantly from 10ms to under 8ms when more CPU cores are added.

Analysis and Conclusion​

In this blog, we evaluate EloqKV and show its performance when data durability is strongly enforced. On a plain low end server, EloqKV can easily sustain over 200,000 writes per second with acceptable latency. While this is lower than the pure in-memory cache performance highlighted in our previous blog, it is still quite suitable for many real-world applications. Notice that this performance number is not much different from what a single-process Redis server can achieve on the same hardware in pure memory mode.

Additionally, we showcase EloqKV's architectural advantage by scaling the LogService to enhance write throughput while maintaining resources used by the TxService. This capability is made possible by our Data Substrate architecture. Considering a scenario where, despite high volume of updates, the total data volume can easily fit on a single server's memory (an example is high-frequency trading). EloqKV's full scalability is crucial to support such applications without wasting valuable resources.

In this blog, we evaluate small scale clusters to show the behavior of EloqKV accross servers. Scalability in a larger scale cluster with different number of servers will be evaluated at a later blog. All benchmarks were conducted on AWS (region: us-east-1) EC2 instances running Ubuntu 22.04. Workloads were generated using the memtier-benchmark tool.

KV Store Clustering​

For most real-world applications that require a key-value (KV) cache, even single-threaded Redis is often sufficiently fast. In fact, the limiting factor is usually memory capacity. As a result, KV caches are commonly deployed in cluster mode.

Most KV caches support horizontal scaling and operate in cluster mode by partitioning data into slots and distributing them across shards. In a horizontally scaled cluster, performance and capacity can generally scale almost linearly, though load imbalances and failure cases can introduce challenges. To achieve this scalability, developers must understand the concepts of slots, shards, and tags. And the so called cluster-aware clients need to know the cluster topology to direct requests to the correct server.

Clustering for KV cache is fraught with pitfalls, not the least is because nodes in a cluster may fail. External tools such as Sentinel, Twemproxy, Dragonfly Cloud services are often required to monitor cluster health and handle failovers. How these tools interact with clients are often not well specified. Additionally, a KV cache cluster behaves differently than a single node server. For instance, the "MULTI / EXEC" commands do not work in a clustered environment.

The fundmental issue is that many KV caches were initially designed as single-node servers, with clustering added later as a bolt-on feature. For example, Redis was released on May, 2009, while Sentinel support arrived in Redis 2.8 in December 2013. Redis Cluster became a stable feature only with Redis 3.0 in April 2015. The same can be said for many other KV stores.

Why EloqKV Clustering is Different​

EloqKV eliminates these issues by being designed as a fully distributed transactional database from the start. While it can function as a single-node server and use various clustering tools for horizontal scalability, it is also capable of operating as a cluster without exposing cluster details to clients. Clients can interact with any node in an EloqKV cluster without worrying about which server holds the key, how data is sharded, if there’s a failure in the cluster, or whether the cluster is reconfiging to dynamically increase or reduce capacity. This simplifies the development process and reduces the need for cluster-aware clients.

Obviously, making this process transparent to the client and shielding cluster details has cost. In particular, a node redirecting requests will incur an extra network round trip. Therefore, we do allow enabling a flag so that servers follow the Redis cluster protocol and do not redirect requests when data requested is not local. In this case, smart Redis clients will operate as expected. Unlike most KV cache clusters, EloqKV clusters are strongly consistent. For example, if a node is dropped from the cluster due to a network partition, it is aware of the situation and can refuse to serve external requests until it rejoins.

We compare the performance of a single-node EloqKV instance with that of an EloqKV cluster to evaluate the cost of transparent redirection. In this assessment, EloqKV operates in pure memory mode, with persistent storage and transactional features disabled.

Hardware and Software Specification​

Server Machine:

Experiment:​

We conducted performance benchmarks on a single-node EloqKV database under varying read-write ratios, utilizing a single memtier-benchmark client. The command used for this benchmarking is as follows:

memtier_benchmark -t 32 -c 16 -s $server_ip -p $server_port --distinct-client-seed --ratio=$ratio --key-prefix="kv_" --key-minimum=1 --key-maximum=5000000 --random-data --data-size=128 --hide-histogram --test-time=300

To assess the performance of a three-node EloqKV cluster, we utilized three memtier-benchmark clients in both regular and smart client modes. In regular client mode, users interact with the EloqKV cluster as if it were a single node without needing to consider where the keys are stored. The same memtier-benchmark command used for a single-node setup is applied to each regular client, with each client connecting to a different EloqKV node in the cluster.

For smart client mode, we ran memtier-benchmark in cluster mode with varying read-write ratios, using the following configuration:

memtier_benchmark -t 32 -c 16 --cluster-mode -s $server_ip1 -p $server_port1 -s $server_ip2 -p $server_port2 -s $server_ip3 -p $server_port3 --distinct-client-seed --ratio=$ratio --key-prefix="kv_" --key-minimum=1 --key-maximum=5000000 --random-data --data-size=128 --hide-histogram --test-time=300

Results​

Below are the results of comparing the throughput of a single-node EloqKV instance with that of a three-node EloqKV cluster across various workloads. For the three-node cluster, we conducted benchmarks using both a regular client and a smart client. When using the regular client, EloqKV automatically redirects requests to other nodes if the requested key is not stored locally. In contrast, with the smart client, all requests are sent directly to the node that holds the key, based on the cached cluster configuration within the smart client.

X-axis: Represents the different workload types (read/write/mixed) used in the benchmark, simulating a range of real-world scenarios.

Y-axis: Measures the OPS (Operations Per Second).

When using the regular client, the three-node EloqKV cluster has slightly lower throughput than the single-node instance due to the added network round trips and scheduling overhead caused by redirection. Despite this, the cluster still achieves over one million operations per second (OPS). Importantly, this requires no changes to application code, allowing developers to easily scale and overcome memory capacity limits without modifying their code or relying on smart clients.

Notice that a single Redis server can only support around 200K OPS (see our previous blog for a detailed analysis), thus we believe that EloqKV in cluster mode is already fast enough to support most application scenarios without relying on specially designed smart clients. Still, EloqKV does support smart clients. As expected, with a smart client, the three-node EloqKV cluster supports near three times the throughput of a single-node instance across various workloads. This highlights EloqKV's compatibility with smart clients.