CedarDB and the agent database bottleneck

CedarDB put the agent data layer on stage

On October 13, 2026, TechCrunch published its 2026 Startup Battlefield Top 20 list, covering companies presenting at Disrupt across categories from organs to orbit. CedarDB was included with a pitch around a database layer for AI agents. The significance for infrastructure operators is not that another database vendor entered a startup competition. It is that the database is now being discussed as part of the agent runtime, not just as a system of record sitting behind an application.

That change matters for anyone operating a quarter rack to a few racks of GPUs and CPUs. Agent workloads are not the same as single prompt inference, offline batch training, or a stable web application. An agent may retrieve context, call tools, write intermediate state, check permissions, update a task queue, generate embeddings, perform vector or hybrid search, and then call the model again. The GPU is only one component in a loop that crosses databases, object stores, queues, policy engines, and network paths.

If those supporting systems are underprovisioned or poorly scheduled, a GPU cluster can look busy from the outside while delivering low productive utilization. GPUs wait for feature retrieval, database commits, object reads, rate limited tool calls, or consistency sensitive state. Users see latency. Operators see allocated GPU hours that do not turn into billable work. Finance sees a power bill and depreciation curve that do not match the revenue plan.

For hyperscalers, this is a regional capacity planning problem spread across thousands of servers. For an operator in the GCC, MENA, or any market building sovereign AI capacity on owned hardware, it is more acute. There may be only 8 to 64 GPUs in the first estate. A single oversubscribed storage network, a missing quota, or an unmetered database tenant can affect the whole business case.

The bottleneck moves from FLOPS to loops

Agent systems create repeated loops. A user request may require 5 to 20 model calls, multiple retrieval operations, and several writes to durable state. Some calls need low latency. Others need transactional correctness. Some context can be cached. Some cannot, because tenant isolation, auditability, or policy rules require fresh reads.

This shifts the bottleneck from raw accelerator throughput to loop throughput. A GPU can generate tokens quickly, but an agent can still be slow if every turn waits on a saturated database connection pool or a storage pool with poor small object performance. The operator problem is no longer just how many accelerators are installed. It is whether the rest of the rack can keep those accelerators fed without violating tenant boundaries.

The practical failure modes are familiar:

1. GPU pods request large slices of capacity, then block on retrieval or writes.
2. Vector search and transactional state share the same storage pool without class of service.
3. VM tenants run noisy database workloads that affect Kubernetes tenants.
4. A team reserves premium GPUs for agent inference, but their embedding jobs run whenever they want and consume CPU, memory, and storage IOPS.
5. Metering captures GPU allocation, but not the database and storage work that made the GPU useful.
6. Operators cannot explain why utilization dropped, because telemetry is separated across Kubernetes, VMs, Ceph, and the database layer.

CedarDB's Battlefield appearance is a useful marker because it reinforces that agent infrastructure includes a purpose built data path. Whether an operator uses CedarDB, PostgreSQL, MySQL, a vector database, an analytical engine, or a mixed design, the capacity question is the same: how many concurrent agent loops can the estate sustain per GPU, per tenant, per watt, and per storage pool.

Why small and mid size GPU estates feel it first

A hyperscaler can absorb imbalances with very large resource pools. A small operator cannot. If a few racks contain 24 GPUs, several CPU nodes, and a Ceph cluster, every architectural decision has a visible opportunity cost.

Consider an estate with 16 accelerators, 4 CPU heavy nodes, and 3 storage nodes. If an agent service reserves all 16 GPUs for peak traffic but spends half its runtime waiting on retrieval and state writes, the operator has effectively bought an expensive distributed lock manager. The GPUs are present, powered, and depreciating, but the useful work is constrained elsewhere.

The bottleneck may not be obvious during a demo. A single agent session runs well. Ten sessions run well. Then 200 concurrent sessions arrive, each creating tool calls, embedding lookups, database updates, and audit events. Latency grows first. Then retry traffic appears. Then the scheduler adds more pods, which increases pressure on the same database and storage systems. The GPUs may show moderate utilization, but the queue length and end to end response time degrade.

This is why GPU orchestration cannot be separated from CPU, storage, and tenancy. A platform for owned hardware needs to provision bare metal, place Kubernetes and VM tenants on the same fleet, partition GPUs where appropriate, and meter the non GPU resources that determine whether the GPU allocation is productive.

Clastiq is built for that operating model. It combines bare metal provisioning through MAAS and Juju, Kubernetes and VM tenancy, GPU partitioning through mechanisms such as MIG and time slicing where hardware supports them, Ceph storage, per tenant metering and chargeback, quota and policy engineering, air gap and sovereign deployment patterns, turnkey installation, and human led local support. The point is not to hide the hardware. The point is to make the whole estate schedulable and accountable.

Co-design GPU scheduling with data capacity

The first operator move is to treat database and storage capacity as first class scheduling constraints. A GPU job for agent inference should not be admitted only because a GPU slice is free. It also needs CPU, memory, network, database connections, storage IOPS, and policy allowances.

In a Kubernetes environment, this can be expressed through namespaces, quotas, priority classes, node labels, storage classes, and admission policy. In a mixed Kubernetes and VM estate, the same principle applies at the tenancy layer. The operator needs to know which tenants are allowed to run database heavy workloads, which storage pools they can consume, which GPU partitions they can reserve, and which traffic patterns are acceptable.

A simple mapping might look like this:

This table is intentionally plain. It reflects the daily work of keeping a few racks stable. Operators do not need a separate platform for every row. They need a control plane that exposes enough of the physical estate to make sane placement decisions.

For example, an agent tenant may need 4 GPU slices, a low latency database VM, and a Ceph pool with a defined replication and performance profile. Another tenant may need batch embedding for 6 hours overnight. A third may need CPU only retrieval testing. If those tenants are not separated by quota and policy, one team can consume shared storage or database capacity and make another team's GPUs look underutilized.

A worked example for utilization and chargeback

Assume an operator runs 16 GPUs in a compact estate. The all in IT load for the GPU servers, CPU nodes, storage nodes, and networking is 15 kW at the rack level. The operator charges internal or external tenants 2.80 dollars per reserved GPU hour. This rate is illustrative. It should be replaced with the operator's own capital cost, support cost, power cost, facility cost, and margin target.

At 45 percent average sellable utilization, annual reserved GPU hours are:

16 GPUs x 24 hours x 365 days x 0.45 = 63,072 GPU hours

Annual GPU hour revenue at 2.80 dollars is:

63,072 x 2.80 = 176,601.60 dollars

Since the estate consumes 0.015 MW of IT power, the annualized revenue per MW of IT load is:

176,601.60 / 0.015 = 11.77 million dollars per MW per year

Now suppose the operator fixes the agent loop bottleneck. The fix is not a faster GPU. It is better placement of the retrieval tier, storage class separation for agent state, tenant quotas on database connections and storage IOPS, and GPU partitioning so smaller agent sessions do not reserve whole devices. Average sellable utilization rises to 70 percent without adding accelerators.

16 x 24 x 365 x 0.70 = 98,112 GPU hours

98,112 x 2.80 = 274,713.60 dollars

274,713.60 / 0.015 = 18.31 million dollars per MW per year

The difference is 98,112 minus 63,072, or 35,040 additional sellable GPU hours per year. At 2.80 dollars per hour, that is 98,112 dollars of additional annual revenue from the same GPU count and the same 15 kW IT load.

This is the arithmetic that matters to a small or mid size operator. Utilization is not a dashboard vanity metric. It determines whether the estate can pay for power, support, spares, financing, and future capacity. Agent workloads make that arithmetic more sensitive because idle time can be hidden inside cross service waits.

Chargeback should reflect this. A tenant that reserves GPU slices and drives heavy database write amplification should see both charges. A tenant that uses CPU retrieval and storage but no GPU should not be invisible. A platform such as Clastiq can expose per tenant metering across GPU, CPU, memory, storage, and policy defined allocations, so the operator can move from argument to accounting.

Partitioning is useful, but only with admission control

MIG and time slicing can improve utilization for inference and agent workloads, especially when a tenant does not need a full accelerator. The risk is fragmentation. If every tenant can request arbitrary GPU partitions without matching CPU, memory, and data path capacity, the platform simply creates more ways to wait.

Partitioning should be tied to workload classes. A latency sensitive agent endpoint might receive smaller but protected GPU slices, a fixed CPU request, and access to a low latency retrieval tier. Batch embedding might receive larger windows during off peak hours, with lower priority and a separate storage class. Development notebooks might use time sliced capacity with strict quotas and automatic expiration.

The exact mechanism will differ by hardware. NVIDIA, AMD, and mixed CPU and GPU estates expose different partitioning and telemetry options. A hardware agnostic operator should define the policy intent first: isolation, utilization, fairness, and recoverability. The implementation can then follow the installed hardware.

A simple quota setup for a tenant namespace could start like this:

kubectl create namespace agent-team-a
kubectl label namespace agent-team-a tenant=team-a workload=agent
kubectl apply -n agent-team-a -f quota-agent-team-a.yaml
kubectl get resourcequota -n agent-team-a

The quota file should not stop at GPU count. It should include CPU, memory, pods, persistent volume claims, storage class limits, and any device plugin resources available in that cluster. For production, it should be paired with admission policy, priority classes, network policy, and metering labels. The important practice is to make the tenant's data path visible at the same time as its GPU path.

Storage design for agent state

Ceph is often a practical fit for small and mid size estates because it gives the operator a unified storage substrate across block, object, and file use cases. It is not magic. Agent workloads can punish storage designs that were sized for simple model checkpoints or VM images.

Operators should separate at least four storage patterns: model artifacts, vector or search indexes, transactional agent state, and logs or audit data. Model artifacts are large and relatively stable. Indexes may be large, memory hungry, and read intensive. Agent state can be small but latency sensitive. Audit streams can be append heavy and retention driven.

Putting all of this into one undifferentiated pool makes troubleshooting hard. A better pattern is to define storage classes and pools with explicit expectations. Replication, erasure coding, device class, placement group sizing, and network design should be chosen based on the workload. For a few racks, the operator may not have infinite separation, but even basic class of service is better than a single shared bucket of contention.

This is also where sovereignty and air gap requirements matter. In GCC and MENA deployments, data residency, regulator expectations, and customer contracts often require local control over logs, prompts, embeddings, and derived data. Agent systems can create sensitive intermediate state that is easy to overlook. The database and storage layer must be designed for tenant isolation, retention, backup, and deletion from the start.

Clastiq's role in this model is to align the storage layer with the tenancy model. Ceph capacity is not just raw terabytes. It is a metered, policy controlled resource attached to tenants and workloads. When storage is included in chargeback, teams have an incentive to clean up indexes, right size logs, and schedule batch jobs rather than treating the storage cluster as free.

Mixed Kubernetes and VM tenancy is normal

Agent infrastructure rarely lives entirely in Kubernetes. Some database systems run well as managed Kubernetes workloads. Others are operated as VMs for clearer lifecycle control, storage attachment, or support boundaries. Some tenants want notebooks. Others want API endpoints. Some require isolated VM environments for data handling or license reasons.

For an owner operated GPU estate, forcing everything into one runtime is usually counterproductive. The better model is to run Kubernetes and VM tenancy on the same physical fleet, with clear boundaries and common metering. Bare metal provisioning through MAAS and Juju helps standardize the base layer. Above that, the operator can present Kubernetes clusters, VM pools, and storage services according to tenant needs.

The hard part is not creating a VM or a cluster. It is preventing resource accounting gaps. If a VM based database supports a Kubernetes based agent service, both sides need tenant tags and cost allocation. Otherwise the GPU namespace gets charged while the database VM becomes shared overhead. That hides the real unit economics of the workload.

Operators should also watch network paths. Agent loops can create heavy east west traffic between inference pods, retrieval services, databases, and object stores. A few oversubscribed uplinks can reduce GPU utilization as effectively as a failed storage disk. Placement policy should keep chatty components close where possible, while maintaining tenant isolation and failure domain discipline.

Policy is an infrastructure control, not paperwork

Agent workloads make policy operational. A tenant may be allowed to use a certain model, but not a certain dataset. A workload may be allowed to retain agent memory for 30 days, but not indefinitely. A development namespace may be allowed time sliced GPUs, but not full GPU reservations. A batch embedding job may be allowed to run only outside business hours.

These rules affect scheduling and cost. If policy is handled manually, the operator will either block useful work or allow unbounded consumption. Quota and policy engineering turns those rules into repeatable controls. It also makes audits easier, because the operator can show how workloads were admitted, metered, and isolated.

For sovereign and air gap environments, this is especially important. Local operators may need to prove where data was stored, which tenants accessed which resources, and how capacity was allocated. The database layer for agents increases the amount of state under governance. Prompts, retrieved passages, tool outputs, embeddings, and action logs can all become regulated or contractually sensitive data.

What operators should take from CedarDB's pitch

The useful lesson is not that every operator must adopt a specific database. The lesson is that the agent data layer is now part of the critical path. Treating it as an application team detail will strand infrastructure investment.

Before buying more GPUs, an operator should ask several questions. How many concurrent agent loops can the current database tier sustain? What is the p95 latency for retrieval under realistic multi tenant load? Are GPU reservations correlated with database wait time? Do storage pools have tenant level metering? Can the operator charge for CPU and storage usage that supports GPU workloads? Can a noisy VM tenant affect Kubernetes inference services? Are audit logs and intermediate agent state covered by retention and sovereignty rules?

If the answers are unclear, the next improvement may be orchestration, not hardware. More GPUs can make the financial problem larger if the estate cannot feed them. Better scheduling, partitioning, storage class design, and chargeback can raise utilization without increasing power draw.

For operators in the GCC and MENA, the opportunity is significant. Demand for local AI capacity is growing, while sovereignty and data residency requirements limit the use of remote hyperscaler regions for some workloads. A quarter rack or a few racks can be commercially meaningful if utilization, chargeback, and governance are designed together. Revenue per megawatt is won through disciplined operations, not only through accelerator procurement.

What to do this week

1. Profile one agent workload end to end. Measure GPU active time, database wait time, retrieval latency, storage IOPS, and queue depth for the same request path.
2. Add tenant labels to every related component. Include Kubernetes namespaces, VM names, storage volumes, database instances, buckets, and monitoring dashboards.
3. Create or review quotas that include GPU, CPU, memory, persistent storage, database connections, and storage class limits. Do not meter GPUs alone.
4. Separate storage classes for model artifacts, retrieval indexes, transactional agent state, and audit logs. Start simple, but make contention visible.
5. Test a noisy neighbor scenario. Run an embedding or database heavy job beside an inference tenant and confirm that policy, priority, and metering behave as expected.
6. Recalculate revenue per MW using actual sellable utilization, not installed GPU count. Use the result to decide whether the next spend should be GPUs, storage, networking, or orchestration.

Clastiq runs this operating model on the operator's own hardware, from a quarter rack to a few racks. request a demo

Sources

TechCrunch, The 2026 Startup Battlefield Top 20