On Product HuntAgent ObservabilitySee how we help you debug AI agents, and share your feedback.

What I wish I (and my agents) had known about ClickHouse #6: move to Altinity?

Originally published on LinkedIn

The Denali Park Road heading east through spruce forest toward the snowy Alaska Range
Heading east on the Denali Park Road, September. Photo: NPS / Emily Mesner, via Wikimedia Commons.
On this pageWhy we looked at Altinity againWhat we triedWhat we measuredThe one-pagerThe decisionAfterthoughtsWhat's next?

Earlier this month, EnterpriseDB (EDB) acquired Altinity. The news reminded me that we did a proof of concept (PoC) with Altinity in 2025. I figured I would share the story and the one-pager I wrote for the PoC. I hope it is useful if you are in a similar dilemma, choosing between ClickHouse Cloud and Altinity (or a similar vendor) for a project, or debating whether it is worth moving existing workloads off ClickHouse Cloud.

When we started the logs project, we briefly looked at Altinity, Tinybird and Aiven, and chose ClickHouse Cloud. We had to move fast and had no time to do a proper evaluation, so we picked the safest option. There were also not many third-party comparisons and data points we could use at the time.

Why we looked at Altinity again

In the late spring of 2025, we had been serving our logs product from ClickHouse Cloud for about five months. We had already picked the obvious low-hanging fruits. To speed up some slow queries in our customers' workloads, only two options were left:

  • Increase the cluster size and the max_threads count.
  • Wait for a few fixes that were already in an open-source ClickHouse release, but not yet in a ClickHouse Cloud release.

We also wanted to improve our costs. By my calculation, ClickHouse Cloud charged roughly $110 per vCPU per month. If we ran the infrastructure ourselves, we would pay roughly $30 per vCPU per month at the on-demand EC2 price. With reserved instance prices or the startup credits that cloud providers offer, the price per vCPU would be even lower.

What we tried

We tried two of Altinity's deployment models: their SaaS service, and bring your own Kubernetes (BYOK), where Altinity manages ClickHouse inside your own Kubernetes cluster. BYOK was the main setup we evaluated. With BYOK, ClickHouse runs on our own infrastructure, in our own AWS account, so we pick the instance types and disks and pay AWS for them directly. It would also give us full flexibility on which release to run, and even which fixes to apply. We double-wrote a few log tables to both ClickHouse Cloud and Altinity: our public playground logs and our own internal logs.

What we measured

We were also looking for lower costs, so we tested a smaller Altinity cluster: each node had half the vCPUs and half the memory of a ClickHouse Cloud node. With the same vCPUs and memory, Altinity would have cost about 5% more than ClickHouse Cloud, mostly because of the 2 TB EBS disk on each node. With half the vCPUs, it cost about 30% less.

We ran the same queries on both, with the query cache and the filesystem cache turned off. Altinity ran ClickHouse 25.3, and our ClickHouse Cloud service ran 24.12. On ClickHouse Cloud, we ran each query with two max_threads settings, 8 and 16. On Altinity, we used 8.

Query time on ClickHouse Cloud at 8 and 16 threads and on Altinity with local and S3 disk, for three log queries. Query 1: 2.6, 1.6, 2.1 and 3.4 s. Query 2: 5.9, 5.3, 2.7 and 3.3 s. Query 3: 6.3, 2.8 and 3.1 s, not run on S3

With half the vCPUs and half the threads, Altinity on local disk was comparable to ClickHouse Cloud: slower on query 1, and faster on queries 2 and 3. On S3 disk, Altinity was slower on query 1 and still faster on query 2. On ClickHouse Cloud, more threads sped up queries 1 and 3 a lot but barely helped query 2, and more threads means larger, more expensive clusters.

In short, for similar query speed, Altinity's list price was about 30% lower than ClickHouse Cloud, but only while the data stayed on local disks. The quote assumed 2 TB of local disk per node. Logs with long retention need more disk, unless the old data moves to S3, where Altinity lost much of its speed advantage.

The one-pager

Below is the one-pager I wrote for the PoC, verbatim. I only removed a few pieces of sensitive information.

The PoC one-pager listing the pros of moving to Altinity BYOK (control over the ClickHouse version, performance, cost, support, long-term goals, easy to migrate off) and the cons (stability, availability, operating disks, upgrades and scaling, and the one-time data migration)

Reading it again, the pros were mostly about knobs and control, and the cons were mostly about the work and risk of running it.

The decision

We stopped the double-writes and did not move, for three reasons:

  • Fear of the data migration tax. Altinity was faster, but to get that speed we first had to move every log table, materialized view and dbt model. That was a large project, and customers would see no benefit until it was finished.
  • We did not want disks as the durable storage for our logs. Some of our team members had operational nightmares managing EBS volumes in large-scale production environments. And with data on S3 disks, Altinity lost much of its speed advantage in our tests.
  • A change in priorities. Other work became more urgent, and the move kept slipping.

Afterthoughts

Right after we made the decision, a ClickHouse Cloud release caused an incident for us. Our service was upgraded to a new version, and the next morning some log queries for one customer got stuck. Log search was partly unavailable for that customer for several hours. Ingestion was not affected. The cause was a new feature in the release, and ClickHouse fixed it by reverting that feature (details on our status page).

After that, we asked ClickHouse to put our services in the last batches of the regular release channel. ClickHouse Cloud also has a slow release channel, which gets each release about two weeks after the regular one, but it is only available on the Enterprise tier. Our Scale plan was not eligible, and the ClickHouse support team did a one-time override for us as a courtesy. The one-pager listed newer releases as a pro and exposure to new bugs as a con. This incident was that con, on ClickHouse Cloud, where we did not have the flexibility to choose when to upgrade.

The biggest lessons were for ourselves, because we often find ourselves in Altinity's shoes. Most of our new customers have to migrate from another observability vendor, such as from Datadog to Oodle and from Elastic to Oodle, and helping them past the fear of a migration has always been challenging.

During the PoC, the Altinity team mentioned that a data loader tool was available for data migration, but we did not receive more information about it. Without it, the migration looked like a large project, and that was enough to stop us.

Customers sometimes have a stronger than usual fear of migrations and new workflows because of past experience. In our case, the team had an architectural dislike of managing disks at large scale. When a customer evaluates us, the migration and the operational load are part of the decision, not just the product.

So we have worked on making a migration to Oodle easier to understand before a customer commits to it. We added detailed migration instructions, case studies from customers who have moved, and live video walkthroughs of a migration. There are more to do, and we will keep making migrations to Oodle smaller and safer.

What's next?

In the next episode, I will come back to the JSON column type. Episode 5 was about a wide JSON column; the next one covers other ways JSON made our log searches slow.

Written byTao WanCo-founder at Oodle. Previously at Snowflake, Roku and Amazon.

See your own telemetry in Oodle

Start debugging in minutes.

Start freeBook a demo

Get new posts by email

Engineering write-ups from the team.