Changing the engine mid-flight: A startup data pivot

The short version

Eighteen months into building Kaleida, a bootstrapped startup helping leaders turn performance data into real career conversations, the data model that had carried us through validation stopped flexing. Basic customer requests were getting harder, not easier. Every screen was running expensive Dynamo scans instead of queries. Half the team wanted to swap DynamoDB for SQL and get back to shipping.

Instead we paused and took the time to understand the actual problem. We ran Event Storming and found that a large part of it was a shared-language and modelling problem rather than solely a tech problem. We brought in an outside expert, defined our domain models, ran a formal R&D experiment for our tech problems, and then rebuilt on the same technology with a completely different data model: DynamoDB single-table design. We executed it as a dual-write migration with verification, alarmed fallback reads, and per-object cutovers, while continuing to ship features and onboard customers. It immediately unblocked five prospective customers who had previously been unable to onboard.

The signal

We started with one dynamo table per object. A User table, an Assessment table, a UserContribution table. There were no diagrams and no modelling discussions. The models lived as domain classes in the code, and that was a deliberate choice rather than an oversight. We were prioritising learning and delivery over modelling for a flexibility we did not yet know we needed. It was genuinely fine, because we were a small team iterating collaboratively and everyone could see the models and contribute to them.

There was an earlier fork, about a year in. We already knew Assessments was wrong. We had built it on the assumption that people work through an assessment from beginning to end, and they don't. They go backwards, forwards, stop-start, and jump around. We adapted the UI to cope, but the model still wanted a start date, a finish date, and one assessment closed before the next opened.

Restructuring away from it felt like a big step, because Assessments was the wrapper around all skill information. A core concept, not a peripheral object we could delete. We knew its whole existence was wrong. We did not yet know what right looked like. So we patched. I am an advocate for the last responsible moment, and I am certain that bias influenced the call.

What broke it was success. More customers, more prospects, more features. And then our founding customer restructured their organisation and needed more than one team lead per team, people in more than one team, and more role levels. Every customer was different and special, and the feature requests were piling up from every side.

Assessments weren't really assessments. The way we displayed teams didn't match our customers' actual teams. Our permission sets didn't quite line up. Small patches on top of each other were turning into problems.

Our simple and elegant system was starting to groan and creak. But that is what software does. It grows up, and we were just reaching our time-to-grow-up phase. This is a signal, not a disaster or a failure.

Resisting "just swap the tech"

The team squarely blamed the database, and they weren't wrong. It was the biggest pain point, and SQL was familiar and sitting right there in our existing skill set.

Any time you find yourself using the word "just", there is a chance that you are doing some mental laziness.

Swapping to SQL was a legitimate option. It was not the only option. If you are replacing your whole database, you can afford to spend a little time doing a proper evaluation. No matter how rushed you are, the choices you make here have long reaching effects.

So we did three things before committing to anything.

1. Model what you have

Not to redesign. To understand. We drew out the existing data structures, put every outstanding feature request on a post-it against them, sorted each one as possible, easy, or hard, then clustered them to separate the universal from the per-customer. The questions we were actually asking: were the objects wrong, were the indexes wrong, or were the queries wrong?

Two rules for this step: don't fix it, and don't judge it. This is information gathering, not a performance review of past decisions. Plenty of teams leap straight to redesign, and that is not wrong, but you overlook things when you skip understanding what you already have.

2. Model what it should be

The technical deep dive surfaced something we hadn't expected. Part of the team thought it was purely a tech problem. Others thought Dynamo was still right and we were simply using it wrong. Underneath both positions was the same thing: we were collectively misusing the language. Different terms meant different things to different people, especially amongst the founding team.

We ran Event Storming with a dedicated facilitator, across a few sessions over a few days. Buy-in was easy for a slightly funny reason: everyone already believed other people were misunderstanding them, and wanted a session where they would finally be understood.

We ran it with everyone in the company, including the people who were not engineers. That was not a nice-to-have. The words we were misusing were sales words and product words as much as technical ones, so a room with only engineers in it would have agreed on a language that was still wrong.

Tech choices are much easier and obvious when you understand and agree on the shape of the problem.

It gave us aligned terminology, agreement on correct system behaviour, explicit boundaries, and new domain models built from the whole team's understanding. It also gave us something I didn't anticipate: newer team members could start to trust us when we told them something.

This became the point where we stopped patching and committed to a rebuild. We now knew how far away our existing structures and models were from what we needed to grow into.

3. Ask, then experiment

Dynamo had given us things I really valued. No servers, a friendly free tier, easy IaC, little schema management, automatic scaling, and cheap. It felt like a great tool we hadn't managed to make work for us. So was it a skill issue, or a tech issue?

When you are stuck, remember you never have to be alone. I went to my network and found an expert who looked at what we had built and what we wanted, and said: I have an idea for you. The idea was DynamoDB single-table design. Our reaction was whoa, incredible, and also scary and new. It felt right, and we were not confident we could implement it ourselves.

When you lack confidence but are inspired, use an experimentation method. We chose a formal R&D space, modelling both existing and future data to verify we both could and should take this approach. Several iterations, all lightweight. It's not an experiment if you end up building a second product alongside your real one. We did "enough to learn". That is what buys you guidance and confidence about the path forward.

Here is what that actually looked like. Seven things we needed the new model to handle, five short iterations, and how each one landed.

Auditing worked from the first pass. Relationships, the thing that had us stuck in the first place, did not come good until the fifth. Experiments turn your gut feels and intentions into data-validated decisions.

The new design

One table, two global secondary indexes. Keys became composite strings of the type and the ID, concatenated with sub-types, using # as the break. The GSIs invert the access pattern: instead of querying for the individual, query for the team and get all the user or skill data in one go.

Previously we had very targeted queries plus scans for anything large, across several tables. Now there is one table and two projections, and every access pattern is a query. This is the access-pattern table from our internal documentation, as shown in the talk.

That table lived in our internal documentation, not just in the code. When you bring in new tech you need to make it accessible to everyone, including the people doing client support who are not in the codebase. Make it easy while they learn the new patterns.

Doing it without stopping

We had a live product, customers onboarding, features shipping, and a company ramping up rather than down. A tools-down, all-hands rebuild was never on the table. Four guiding principles held the plan together, and they held the whole way through.

• Continue business as usual. Customers kept onboarding and features kept shipping throughout.
• All new work migrates forward. Anything new was built against the new architecture, so we were never making more work for the people doing the rebuild.
• A small dedicated team for the pivot. In a company of four, that meant one person focused on it for the first two months.
• Rollback and verification at every step. Quality was part of the brand, so customers should never notice.

That last one is a great non-negotiable when you are playing with tech that is new to you, because it mitigates a lot of the human mistake risk that comes with new tech.

The approach: simple, but not easy

1. Dual writes. Every place that wrote to the database now wrote to both structures, with migration scripts ETLing existing data across. New things went into new tables deliberately, because if you get it wrong you blow it away, fix it, and rerun it. We wrote a new data access layer at the same time, for clean code and a clean teardown later.
2. Move the reads, with a fallback. Not a straight replace. The old reads stayed in place, and any read failure on the new database raised an alarm and fell back to the old one. Belts and braces. It bought us the confidence to test in production. Cutover was per data object rather than horizontal: all frameworks, then all users, then all teams, then all skills data.
3. Delete the old writes. I was tempted to skip this, on the "it's fine, we'll come back and clean this up" reasoning. Our resident expert had their sensible hat on and said no. A half completed migration is insanity. Double the maintenance, and the bane of your future self.

"Sounds like you've done nothing for months"

Very few groups are as time-poor as a bootstrapping startup, so here are the real numbers rather than a vague reassurance.

The thinking and experimentation phase took weeks, not months. Whole team sessions were a few hours or half days here and there, alongside business as usual. Other pieces were the focused work of one person for a week or so.

The migration itself ran about four months from first dual write to the old structures being deleted. For scale: the whole company was four people, the product and engineering team was three, and two of those were dedicated engineers. One person was on the rewrite full time for the first two months. The final month was a dedicated push. The first three ran as background work against a live product.

Honestly it could have been under three months if we had allowed ourselves an outage. Roughly a quarter of the elapsed time bought the guarantee that customers never noticed.

Time spent experimenting and testing tech is not time spent progressing product or delighting customers. That is genuinely risky, and workload balance has to be managed carefully. But if you want to be alive in five years as a company, you don't YOLO at this point.

There is never a good time to pivot on data or tech. There are plenty of bad times, but if it's not a terrible time, then it's a good enough time.

What it bought us

• Five in-pipeline prospective customers onboarded immediately, having previously been unable to onboard due to configuration constraints
• Feature shipping resumed with the whole team
• Customers never noticed. Zero perceived disruption, by design
• A shared domain language across the team, which outlasted the migration itself
• We celebrated, deliberately, before moving on. This is not optional

The plane didn't crash.

What I'd tell you

The path, in nine steps

1. 1. Kept it simple while we were learning
2. 2. When it stopped working, dove deeply into why
3. 3. Modelled past and future
4. 4. Experimented to boost confidence
5. 5. Built a plan to go forward
6. 6. Moved slowly and safely along the plan
7. 7. Got it over the line
8. 8. Finished the clean up
9. 9. Celebrated the win