When we started building an AI harness at Refrens, I had the same question many founders were asking: if AI can write so much of the code, how much more should our engineering team be able to deliver? The promise of three-times productivity was hard to ignore. I wanted to understand what that promise would look like inside a company with an existing product, an established codebase, and engineers who already had their own ways of working.

I also wanted an answer we could measure. A task finishing unusually quickly makes a good demonstration. What interested me was whether the team could consistently get more useful work done, and what we would have to change to make that happen.

TL;DR

• Completed engineering work increased by 43%, measured in story points. Turnaround time, including testing and peer review, was about 16% shorter for a comparable mix of tasks.
• Engineers spent less time coding and could work in parallel. Developers reported roughly 50% less hands-on coding time per task, allowing them to move other tasks forward while the harness worked.
• Junior developers became more independent and ready for complex work. They could resolve about 80% of their day-to-day doubts with AI before asking a senior developer for help, giving us more confidence to trust them with larger assignments.
• The harness is only as good as the context you give it. Access to the relevant repositories and clear guidance on which code patterns to follow improved its output.
• The productivity gains have been worth the cost. We moved from $20 Claude plans to spending $100 per engineer per month, with some spending $200.
• Adoption took time. In the beginning, we faced resistance from engineers who were hesitant to change familiar workflows. We started with one engineer in the second half of June, gradually brought more on board, and had everyone using the harness as their primary way to write code by the end of August.

1. Before the Harness: How We Worked

1.1. We Were Already Using AI, But Everyone Was Figuring It Out Individually

Before the harness, our engineers were using Claude for individual coding tasks. Someone would bring it a problem, provide some context, and work through the response. Each developer had their own approach, and the results depended heavily on what they knew to ask and what information they supplied.

Much of the knowledge needed to produce a useful solution stayed with the engineer. They knew which repository mattered, which implementation was current, and which apparently reasonable change might cause problems elsewhere.

That worked well enough for individual tasks. Making it useful across the team required a more consistent starting point.

We wanted the AI to have access to the relevant code and clearer guidance about how our system worked. That is what we mean by a harness: the setup around the coding tool that gives it the context, access, instructions, and tools it needs to write code, run tests, and verify its changes within our codebase.

To judge whether that setup was helping, we first needed to be clear about what we were measuring.

1.2. How We Measure Engineering Work

We use story points to estimate engineering tasks. The estimate reflects the work involved: how much needs to change, how difficult the problem appears, and how much uncertainty there is about the solution.

A small, familiar fix might receive 1 or 2 points. A task involving several parts of the system might receive 8 or 13. We use a Fibonacci-style scale: 1, 2, 3, 5, 8, 13, and 21.

The gaps grow with the size of the task because larger assignments are harder to estimate closely. There is usually less uncertainty around a small fix than around a change that touches several parts of the product. The scale helps us account for that without pretending we can predict the effort precisely.

Before the harness, we generally expected an engineer to complete about 25 points in a two-week sprint. Actual delivery was consistently closer to 20. That gave us a starting point for comparison, although adding up story points only tells part of the story.

1.3. Why We Also Look at the Sum of Squared Story Points

Consider thirteen 1-point tasks and one 13-point task. Both add up to 13 points, but they can demand very different things from an engineer. Someone comfortable completing the smaller tasks may struggle with the larger assignment because it requires a different level of investigation or technical judgment.

A move from 2 points to 3 points also cannot be understood as adding one fixed unit of complexity. The estimates are relative, and difficulty can grow disproportionately as tasks become larger.

To make that difference more visible, we also look at the sum of each completed task’s squared points. Thirteen 1-point tasks contribute thirteen to this measure. One 13-point task contributes 169.

Squaring gives larger tasks more weight. It helps us examine the mix of work being completed alongside the total story points.

We treat this as an internal proxy. A 13-point task is not objectively 169 times more complex than a 1-point task. Both measures depend on our estimates, but together they give us more information than either one alone.

We also track turnaround time, from the day a developer starts working on a task until its peer review is complete. This lets us look at output and how quickly work moves through the process.

2. Building the Harness: Learning as We Went

2.1. We Brought in Someone Who Could Question Our Existing Habits

We started working on the harness in the second half of June, with one engineer and an external consultant (Ritesh from Vertexcover Labs).

The consultant brought experience building harnesses and a clearer idea of what a useful outcome could look like. He also came without years of attachment to our existing workflows.

Every developer develops preferences around the part of the system they know. Those preferences are often sensible, but we wanted to avoid building the entire harness around one person’s habits. An outsider could ask questions across repositories and challenge assumptions that the rest of us had stopped noticing.

For the proof of concept, we chose one mid-level engineer. Some of our senior engineers were skeptical about changing workflows they had spent years developing. We wanted to establish whether the harness could become useful in everyday work before asking everyone to adopt it.

2.2. The First Week Made Manual Coding Look Easier

Day one: ‘I could’ve coded this myself by now.’

Our first engineer was learning a new workflow while dealing with inconsistent output. When the harness chose a poor approach, he had to understand what had gone wrong, correct it, and still complete the original task.

His frustration was understandable. On some tasks, doing the work himself would have been faster.

As we investigated, we repeatedly found gaps in context. Sometimes the harness lacked access to a relevant part of the code. In other cases, it could see both an older and a newer implementation and followed the older one. This was especially noticeable in frontend work.

That was an uncomfortable but useful discovery. Our engineers knew which approach to use because they had worked through the history of the product. We had expected the harness to make the same judgment without giving it that history.

Early output needed considerable correction. As we clarified the preferred approaches and standardized our inputs, it improved.

We gradually got better at asking a useful question when something went wrong: had the harness been given enough information to do the task properly? That gave us specific things to fix.

“Wait… did I just train my replacement?”

2.3. Once the First Engineer Was Comfortable, We Expanded

Once the first engineer became comfortable with the harness, we began seeing improvements in how quickly he could complete tasks. 

In July, we brought a couple more engineers into the workflow, with the first engineer helping them get started. We continued adding one or more engineers at a time. By the end of August, every engineer was using the harness as their primary way to write code.

It took time for the changes to show results. In mid-August, the team was still getting used to the new workflow, and we were still improving the harness. Our output dipped during this period, but by September, we began to see clearer gains in productivity.

3. After the Rollout: What Changed

3.1. Junior Engineers Had Somewhere Else to Take Their Questions

For less-experienced engineers, some of the clearest benefits came from well-defined bugs and smaller improvements. They could take a task from Asana, work through it with the harness, and bring a proposed change into the development process with more of the initial investigation already done.

It also helped with bugs that had been difficult to reproduce. With access to the codebase, AI could sometimes identify a likely cause and propose a fix before we had reproduced the issue ourselves. The change still needed validation, but the investigation could move forward.

Junior developers became more independent, as they could resolve about 80% (their estimate) of their day-to-day doubts with AI before asking a senior developer for help.

A question that previously interrupted progress could often be investigated immediately. When they did need a senior engineer, they could explain what they had tried and where they were stuck.

Senior engineers still played an essential role. They spotted missing assumptions, assessed broader consequences, and corrected output. Their experience also helped improve the harness for everyone who used it afterward.

3.2. Larger Tasks Could Stay Whole

Earlier, a 13-point assignment was often broken into smaller tasks to make it easier to understand and manage. With the harness, developers could provide the broader objective and retain the surrounding context, allowing larger assignments to stay together. Alongside the shift toward giving some junior developers larger tasks, this changed how we could plan and distribute the work.

3.3. Less Hands-On Time, More Work in Parallel

Developers reported spending roughly 50% less hands-on coding time per task. The harness could inspect the codebase, draft an implementation, run commands, and handle repetitive steps while the developer focused on direction, review, and decisions.

Earlier, engineers usually finished one task before moving to the next. With the harness, they could move another task forward while AI worked on the first, returning when it needed their input.

They could also access the harness from their phones and continue work when away from their computers.

But less hands-on time does not mean a task finishes in half the time. Testing, revisions, and reviews still need to happen. To understand the difference, we compared turnaround times for tasks completed with and without the harness.

How Turnaround Time Changed

We measure turnaround time from the day a developer starts working on a task until its peer review is complete. This includes coding, testing, revisions, and time spent waiting for reviews.

For a fair comparison, we looked at tasks with the same number of story points, then used the same mix of task sizes to calculate the overall average.

The benefit varied by task size:

• Small tasks: For a straightforward 1-point task, coding manually could be quicker than setting up the harness. We had only two harness tasks in this category, so there is too little data to draw a firm conclusion.
• Medium tasks: 3-point tasks showed the biggest improvement, with average turnaround time about 30% lower. 2- and 5-point tasks also finished sooner.
• Larger tasks: 8-point tasks had about 9% lower turnaround time, while 13-point tasks took roughly the same time. In our experience, larger assignments still needed substantial review, which limited how much faster they could finish.

Across a comparable mix of tasks, average turnaround time was approximately 16% lower with the harness, at 10.62 days compared with 12.61 days.

3.4. What Else Changed in Our Engineering Output

Our before-and-after comparison showed a 43% increase in story points completed. 

Story points completed. Source: Refrens’s Asana chart.

We also saw a 50% increase in the sum of squared story points. Average task complexity stayed broadly stable across the team. We therefore should not interpret the point-squared increase as evidence that everyone was suddenly doing much harder work.

We were completing more work overall, and the weighted total increased too. Keeping larger tasks together can also increase the squared-points total, so we read this measure alongside story points and changes in how we structure tasks.

Sum of squared story points completed. Source: Refrens’s Asana chart.

Within that team average, however, something had changed. Developers who had historically received smaller tasks began taking on larger assignments during the rollout.

That suggests AI helped us become more comfortable trusting a wider group of developers with complex work. As a founder, I find that worth paying attention to. An average can remain stable while the team gains more people capable of moving a larger assignment forward.

The comparison reflects a period in which several things were changing together. Engineers were learning, the harness was improving, and we were clarifying conventions in the codebase. Team availability and the work entering each sprint also varied.

I see the numbers as useful evidence from our experience, with those conditions attached. They help us understand the change without pretending we have isolated every cause.

3.5. Before and After the Harness

Looking across the experiment, these were the changes we saw in spending, output, and the way developers worked.

3.6. How Much Do We Spend on AI per Engineer?

We started with a $20 Claude plan for the first engineer. As usage grew, we moved to $100 per engineer per month, with some engineers now spending $200 a month. Our total Claude subscription spend for engineers is roughly $1400-$1500 a month. For us, the improvement in productivity has been worth the cost.

3.7. The Harness Has Become Part of the Engineering Work

Three engineers are actively maintaining the harness today. As they identify missing context or a recurring mistake, they improve the setup so the next person benefits from what they learned.

I expect more engineers to contribute. Each time someone adds what they know about the product, the next engineer has less to figure out alone.

4. What Matters to Us

We started this project chasing 3× productivity. We haven’t reached it yet. But something else has begun to change: instead of getting scared that AI will take their jobs, our developers are learning to tame it. They are learning when to trust it, when to challenge it, and how to turn its abilities into their own productivity.

For our junior engineers, the harness has brought a sense of liberation. An unfamiliar part of the codebase no longer means stopping and waiting for help. They can explore, try an idea, and work through a problem before turning to a senior colleague. Each problem they solve makes the next challenge feel more within reach.

We will keep chasing 3× productivity. But as a founder, seeing people move from fear to confidence, and from hesitation to taking ownership, already matters more to me than reaching that number.