RSS Feeds

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s issue of The Pulse. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Interesting information from Anthropic about why the company moved off Git and built its own custom source control system. From their engineering blog (emphasis mine):

“At Anthropic, coding is no longer the bottleneck to building systems or doing research. Claude is able to autonomously iterate on research loops, micro-optimize complex programs, and build new services from the ground up at a speed and scale we haven’t seen before. The bottleneck, now, is all of the developer infrastructure built for human scale: source control, builds, and CI.

In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day. We don’t expect this rate of growth to slow down anytime soon; the latent demand for merging code is practically infinite, and every bottleneck we remove immediately leads to a further increase in the amount of code we produce.”

That’s a 12x increase in PRs in a four-month period! Interestingly, just a year ago, I shared how I had gathered Google was preparing for a similar tenfold increase across its broader infrastructure. From the deepdive published in July 2025:

“Google is preparing for 10x more code to be shipped. A former Google Site Reliability Engineer (SRE) told me: ‘What I’m hearing from SRE friends is that they are preparing for 10x the lines of code making their way into production.’

If any company has data on the likely impact of AI tools, it’s Google. 10x as much code generated will likely also mean 10x more code reviews, deployments, feature flags and source control footprint.”

Back to Anthropic, a few things about the company are different from many other tech businesses:

• Use of a monorepo. Like most large tech companies, Anthropic has a massive monorepo, where the working copy itself is 25GB in size.
• 25 million git checkouts per day, just for CI! All ~3,000 staff extensively use parallel Claude agents – typically 10-15 per engineer – so a lot of pull requests are created; somewhere in the region of 30,000 per day. All these PRs trigger a massive number of checkouts, mostly from the CI system.
• Nearly all clients are always online. Git was built on the assumption that clients go offline at various times, and that when they come back up, they need to catch up with branches and merge their changes into the repo. However, when clients are always connected to the corporate internet – like at Anthropic – this assumption no longer holds. Design decisions to accommodate offline clients mean inefficient checkouts and more data transferred than necessary.
• Hard to horizontally scale git. Scaling git is done by running git mirrors: replicating git repositories across several mirror nodes. A primary git server accepts the writes, pushes out to all mirrors, and clients fetch from various mirrors. Anthropic ran hundreds of git mirrors, but found that once their repo outgrew the mirrors’ disk capacity, they could no longer scale the repo.

So, Anthropic created a new source control system, which is called ‘Scion’ internally. It’s closed source, and Anthropic has shared the high-level design principles and data structures used, but no code details. The problem with having your own source control system (and moving off Git and GitHub) is that GitHub’s code review system is lost! So, Anthropic also had to build a new code review and code management system, which they call ‘Antique.’

I reached out to the team building Scion and Antique to learn more about how Scion works. Here’s what they told me:

Technical details

The design of Scion is pretty simple: stand up “server” nodes (that contain the whole repository), store changesets inside of Google Spanner (Google’s distributed, highly available database) and blobs (large files) in the highly durable Google Cloud Storage:

I asked Nick G – one of the authors of the original blog posts – about additional details and here’s what he told me:

• Rust: the language that Scion’s server and client are written in.
• Go + React: Antique, Anthropic’s new code review and code management platform has its backend in Go and frontend written in React. It works nicely with Scion.
• More resilient replication strategy: by default, Google Spanner offers multi-zone replication with three replicas. Anthropic has increased the number of replicas (they didn’t tell me the exact number, but an educated guess could be five or more replicas), and switched to multi-region replication. All these changes make sense, given how critical code is and that it’s sensible to reduce the likelihood of losing any data – even at the cost of higher write latencies.
• Backups: on top of replication, Anthropic keeps full backups of the database and soft deletes on the blob storage where file contents live. Spanner’s point-in-time recovery also lets Anthropic restore from a bad write. Smart!

Scaling is an interesting topic. I asked Nick if the system autoscales or scales manually. He told me:

“Every component can autoscale, but so far it hasn’t needed to, even as our merge volume grew more than 10x this year. The data is content-addressed and never changes once it’s stored, so the caching tiers do most of the work.”

A team of seven engineers builds and maintains all of this:

• Four worked on the core Scion server and client
• Three were split between building code search, merge queues, etc.

100% of Anthropic’s monorepo has moved off Git. This was no small feat to pull off! Nick gave me more details about the migration:

“The migration was not simple, but Claude was instrumental in getting it done. It’s also ‘self-healing’ in a way: if we get all the important use cases migrated and have something like 99% coverage, we can just force the cutover and Claude can help us fix anything that actually mattered.

100% of our monorepo is off of git. A few smaller repos that were never in the monorepo are migrating now, and they’re much simpler.

All of our community resources continue to live on GitHub: skills, plugins, Claude Code’s public repo, etc. And last week we launched OSS Scanner, which gives open-source projects free security audits using our strongest models (inspired by Google’s OSS-Fuzz).”

Big Tech moved off Git long ago

My take is that Anthropic has “Big Tech problems” much earlier than most of Big Tech had to tackle them. Anthropic employs around 3,500 staff, but their coding output – thanks to each person using around 10 agents all the time – is closer to that of a tech company with 35,000-70,000 staff! That’s a Meta or Google-sized company by the number of engineers.

Google and Meta have long since moved off Git because it did not scale for them.

Google built its own version control that took inspiration from Perforce. Git itself was created by Linus Torvalds in 2005, seven years after Google was founded. So git was not an option for Google to use in the early years: the company chose Perforce instead. It was around 2011-2012 that Google had ran into scaling issues with Perforce and built its own source control system called ‘Piper.’ It’s also a centralized version control model and still works well enough for the search giant. It’s no surprise to know that Piper was inspired by Perforce, and Google built Piper to be more performant for their use cases than Perforce was. We covered more on Google’s internal stack in the deepdive Inside Google’s Engineering Culture.

Meta found that git and other systems could not scale as needed, and so built ‘Sapling.’ Meta initially moved over to Mercurial, and then custom-built and open sourced Sapling for itself. As the company shared at the time:

“Sapling began 10 years ago [around 2012] as an effort to make Meta’s monorepo scale in the face of ever-increasing engineering growth. Publicly available source control systems were not, and still are not, capable of handling repositories of this size. Instead of moving away from the monorepo and sacrificing engineering velocity, we decided to build the source control system we needed.

Along the way we realized there were also large opportunities to increase developer velocity by improving the source control experience. By investing in new UX, we’ve made it possible for even new engineers to understand their repository and do things that were previously only possible for power users.”

Sapling itself is more of a client-server architecture. From the documentation:

“Sapling started from Mercurial as a distributed source control system, it then transitioned to a client-server architecture to solve the challenges. The server can utilize distributed storage and pre-build various kinds of indexes to provide more efficient operations.

While Sapling is less “distributed”, we tried to make the difference transparent to the user. For example, our lazy commit graph implementation does not require extra commands to “deepen” the graph. The user sees the full history.

That said, we do drop support of pulling from a lazy repo. Commit cloud covers these use-cases.”

Did Anthropic secretly rip off Sapling’s and Mononoke server

I asked Anthropic why it doesn’t open source Scion. Nick G’s reply:

“Mostly bandwidth! Even a no-support code drop takes a lot of work to do well. We have a small team building and running Scion. While the principles we used to build it are durable, the underlying code changes very frequently. For example, we’re still removing git-compatibility layers we only needed for the migration. We’d rather put that time into sharing what we learned, and as the post says, into working with GitHub, GitLab and the OSS community on scaling git.

To emphasize that point: most teams don’t need to leave git! But we think there are things git could benefit from, mainly change sets, derived data, and the checkout protocol. Our goal was to inspire conversation and collaboration around those ideas.”

The explanation, on one end, makes sense. On the other, Meta themselves also emphasized how most teams don’t need to leave git, and still open sourced Sapling for anyone to use, or take inspiration from.

One key component of Sapling is Monoke: the also open source server meant to scale up accepting thousands of commits every hour, across millions of files, written in Rust.

Could have Anthropic avoided open sourcing Scion, because it might have shown how much they copied from existing open source projects? The creator of Monoke wrote a post accusing of the Anthropic team of ripping off Monoke — and the Bonsai data model — writing:

“This [Scion] was a ripoff. My post yesterday was trying to be as charitable as possible given what I knew, and in particular that the similarity to Mononoke was mostly unintentional and the lack of credit was the result of not doing due diligence. However, after publishing the post I privately received confirmation that Scion intentionally and blatantly ripped off Mononoke, and in particular that there was overlap in personnel. There is no chance that this was an unintentional homage. This is something I had a hunch about but was not comfortable saying yesterday, given what I knew back then.

The resemblance is much deeper than just the notion of derived data. Some people pointed out that derived data is just an index, which is a well-known concept. I want to clarify that derived data was merely the most striking resemblance. It goes much deeper than that, and extends to the entire data model, as well as a number of the structures built around it.”

Anthropic did not give credit to Monoke in their original post, and only added a small attribution after the complaint.

To be clear, the point of open source is that you can do what you want with the software without much limitations (save for what the license contains): but it’s poor form that it took a public complaint for Anthropic to admit that the clever data structure they presented in their blog post heavily derived for an existing and open source data strucutre.

Will central VCs stay a niche, or expland beyond Big Tech?

Big Tech has preferred centralized version control systems (VCS), and I wonder how niche of a trend central VCSs will remain. As we’ve seen since well before AI, some of the largest tech companies dropped git because they did not like the performance degradation from offline support they no longer need.

My question is that even with agents, how many companies will see git “break” similarly to how it has at Anthropic? When this happens, how many will find git’s mirroring functionality inadequate for their own needs?

There’s a new batch of vendors positioning themselves as git-compatible version control solutions built for AI scale: Entire, founded by GitHub’s last CEO, Thomas Dohmke (a current podcast season sponsor), Cursor Origin by SpaceX, and Code Storage by The Pierre Company. At the same time, GitHub itself seems to be investing a lot more in making its platform more reliable and scalable.

My hunch is that very few companies will go through the trouble of building yet another source control system. In this way, Anthropic reminds me a bit of Google, which has been known to build custom internal systems which are not publicly available, to address the company’s internal needs. This can be disorienting for Google engineers after they leave the company and find they’ve become out of touch with industry-standard tools.

Read the full issue of The Pulse this is from, or check out this week’s The Pulse. This week’s issue covers:

1. Do we need to rethink software security because of AI? Half a dozen of high-profile security incidents and breaches on a single day: and many of them are connected to AI (or sped up because of it). Attackers are gaining unfair advantages with AI, and defenders will have to react.
2. Is Anthropic’s new version control system “a rip off”? The creator of Mononoke (an open source, high throughput version control server) accused Anthropic for ripping off their design, and not giving credit, presenting their work like it was novel. Not a good look on the AI lab.
3. Industry Pulse. Saudi Arabia made a “problematic” offer to pay alleged OpenAI spy’s bail, Shopify builds tooling for self-healing software, Foursquare’s sale disappoints, Claude Code gets a major speed boost with gzip, and more.
4. Predictions on the future of software development. Some bold predictions from software engineer and Amp cofounder Thorsten Ball, like how open source doesn’t make sense in its current form anymore.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s issue of The Pulse. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

When I visited Ramp’s headquarters last month in New York, I talked with the four-person development team behind ‘Glass,’ which is Ramp’s “internal Lovable.” This name refers to Lovable.dev, the leading AI vibe-coding tool.

Ramp’s ‘Glass’ is basically a vibe-coding platform for non-engineers to create complex applications which run internally. They can share the applications they make with others to use and modify. This is what the landing page looks like:

Users can connect various internal systems as “connectors” that can then be used as data sources for their apps:

The team showed me a few popular apps, including a custom CRM built by someone in marketing, which had 50+ fields and filters on-screen. It seemed overwhelming to me, but the creator reportedly finds it extremely powerful to use.

Since then, I’ve learned that many other mid-sized-and-above companies are building similar “internal Lovables” that don’t usually see the light of day in the outside world.

As of today, Ramp has launched Glass as an external product, describing it as "An AI workspace where knowledge compounds." It's interesting to see Ramp productionizing a formerly internal-only tool!

Stripe’s “internal Lovable”, Harbor

Stripe shared details last month about its own vibe-coding tool (my words) that they refer to more elegantly as an “AI-assisted prototyping tool” called ‘Harbor’. It was built by a single engineer in around six weeks. Here’s an example of a prototype built with Harbor:

Why build your own internal “prototyping tool?”

Lovable, Replit, Retool, and other vendors specializing in prototyping and building internal tools already exist, so what’s the point in bothering to build your own? Here are the main reasons I’ve gathered:

• A tiny investment. Stripe had a single engineer build Harbor v1, and Ramp had two engineers put it together in a matter of weeks. For companies with hundreds of engineers or more, this is a very reasonable time/effort investment.
• Expose internal data to be used. Because the tool produces internal-facing prototypes/tools, it’s a lot safer to expose internal data sources to it. Doing the same with third-party tools like Lovable or Replit would be risky.
• Use the existing design library. Both Ramp and Stripe paid special attention to ensuring that the apps generated do follow internal design guidelines. Third-party apps are not great at this, out of the box.
• Onboard most non-engineers, when done right. Ramp’s Glass project is extremely widespread, to the point that onboarding to Glass and creating a simple tool is now part of new hire onboarding. Pretty much every non-engineer uses Glass to build their own tools.
• Don’t wait on vendors to build what’s needed. Why spend time and effort on tailoring a vendor solution which is straightforward enough to build? It would probably be a comparable effort to select the right vendor to go with, in terms of compliance reviews and price negotiations.
• It’s educational and helps engineers to upskill. Engineers building these tools get better at building and maintaining AI infra at a time when recruiting engineers who are experts at AI infra is hard enough. Internal Lovables are a way to train more devs in-house by investing a few weeks in putting a solution together.

Among tech companies that have built their own coding agent harness that runs in their cloud, I expect many to build internal vibe-coding tools for non-engineers like Stripe and Ramp have done. After all, it’s not that complicated, gets much higher adoption than vendor products, and helps build out more AI infra expertise in-house.

Read the full issue of The Pulse this is from, or check out this week’s The Pulse. This week’s issue covers:

1. OpenAI software engineer accused of spying. A software engineer working on OpenAI’s Agents SDK has been arrested after allegedly planting a secret audio recording device in a meeting room at the AI company. The FBI and US Department of Justice are involved. The details read like the plot for an espionage movie.
2. Anthropic builds its own source control system. Git had too many performance problems at 30,000 PRs per day, so Anthropic built its own. I talked with the team, including asking why they didn’t open source its new system, ‘Scion.’
3. New trend: AI-coding rounds in Big Tech? Meta and Google are adding an additional “AI-assisted” coding round to the interview loop. From a hiring manager’s perspective this sounds sensible, and I bet other places will follow – or have already done.
4. Industry Pulse. An observation that college students are not always so “AI-native”, Netflix lets go 6% of staff, a16z disbands recruitment team, Satya Nadella warns against trusting AI models (and frontier labs), and Meta’s new state-of-the-art ‘Muse Wonder’ model.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s issue of The Pulse. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Firebase – built by Google – has had a nasty outage this week with shockingly poor incident management at odds with how Google itself usually deals with high-severity incidents.

The outage started on Tuesday (29 Sep) at 5:41pm (PDT), when iOS apps using the Firebase SDK started to crash upon first opening; every iOS app that uses the Firebase SDK with analytics enabled was affected in this way. Developers of affected apps opened a GitHub ticket, in the absence of much else to do. On the ticket, the message “it’s crashing for me too!” was oft-repeated.

6:51pm (PDT): acknowledgement. An hour and ten minutes after the crashes started, an engineer on the Firebase team acknowledged that they were aware of the outage.

It’s unclear if the Firebase team was alerted via this ticket with 100+ comments by devs, or if Google’s own monitoring tool showed the issue. I asked Google/Firebase two days ago and haven’t had a response.

Not having anything better to do than wait for Google to resolve the issue, the memes began:

Others attempted to help the Firebase team by pinpointing the potential issue. Indeed, before a Google engineer acknowledged the incident, an external developer found the root cause at 6:37pm PDT; it was a zero-length entry that was crashing the SDK:

Given the flags are shipped by the backend, the offending change was a backend one, and the easiest resolution would be to roll it back, which the community practically begged Google to do:

Here’s a neat summary of the incident from another dev:

7:24pm (PDT): rollback starting. An hour-and-a-half into the incident, the Firebase team started rolling back the offending backend change:

8:16pm (PDT): rollback complete. And the rollback completed ~50 minutes later:

Software engineer, Nick Cooke, on the Firebase team posted a summary with more accurate timestamps:

What we can deduce from this:

• TTD (time to detect): one hour? The Firebase team never shared how long it took them to detect that practically all iOS apps using Firebase had started to crash. On the GitHub ticket, they acknowledged the incident 70 minutes after it started. Update: in the postmortem, later published by the team, they wrote how the team was alerted 20 minutes after the rollout, via crash alerts and GitHub issues. Good question why it took another 50 minutes to acknowledge the issue, though?.
• TTM (time to mitigate): 2-6 hours. It took two hours and eleven minutes to roll out the fix, but due to caching (apps that had cached the incorrect server response served this cache for additional four hours, and so kept crashing for up to six hours.)

Incident management basics

The Firebase team itself closed the outage with a short report effectively saying that there had been an outage, but they’d resolved it now, so thanks for your patience and have a nice day.

This handling of a high-impact incident is absolutely not typical of Google, the company that coined the term ‘Site Reliability Engineer’ and wrote the SRE book.

For one, Firebase never bothered updating its status page. Oddly enough, the official Firebase status page showed all systems green – despite the acknowledgement of the outage. Indeed, during it and afterward, they didn’t update the status page to indicate the lengthy outage:

But status pages exist for good reasons, including:

1. To communicate with customers during and after an outage
2. Offer transparency on the stability of the service

It’s worth asking: if an outage that takes down most (or all?) iOS apps using Firebase doesn’t warrant an update to the status page, then what does!

Google published a postmortem four days later, answering questions on how the outage happened. On Friday, 2 October, Google published a postmortem on the Firebase blog. It was a configuration change that crashed so many iOS apps. From the postmortem:

“On September 28, 2026, a routine configuration cleanup unexpectedly caused a large number of iOS applications using the Google Analytics for Firebase (GA4F) SDK to crash.

2026‑09‑28 17:38 (PST): A stale, legacy configuration flag was cleaned up.
2026‑09‑28 17:41 (PST): The malformed configuration payload begins rolling out globally to production servers. Outage begins: Clients fetching the new payload start crashing on launch.

The SDK missed validating that a flag’s name was not nil, ultimately causing the crash. Backend data anomalies should not cause app-side crashes.”

In the postmortem, Google noted that engineers were alerted to the outage through both GitHub reports coming from external developers, as well as their internal monitoring. It took another hour to pinpoint the cause being a legacy configuration flag cleanup.

Firebase says they have no way to update their status page for client-side outages. In the postmortem, Google explained that there is no place to indicate client-side outages on their dashboard (emphasis mine):

“Throughout the outage, both the Firebase and Google Ads status dashboards remained green. Because these dashboards rely primarily on server-side health metrics, they did not register client-side SDK crashes.

Commitment: Moving forward, we are actively working to: integrate SDK-related outage information into our status dashboards, streamline the manual update process, and improve GA4F status representation within the Firebase dashboard.”

It’s good to see Google not dropping the ball fully, and recognizing that both their dashboards and their incident management process need improvement.

It’s fair to ask though: why did only iOS crash, and not Android? Firebase’s Android SDK seems to be hardened more than iOS, as the feature flag removal did not crash Android devices.

Especially that now, with AI, it’s easier than ever to compare iOS and Android implementations to ensure they are identical – and it’s what Shopify has been doing during their native rewrite – could it have been a missed opportunity for Google to audit the differences between the iOS and Android SDKs? To me, not having an action item here feels like a missed opportunity.

Still, this is a good reminder to anyone and everyone shipping iOS and Android apps: aim to harden them, and when possible, run tests with malformed payloads, then fix crashes those payloads cause.

Déjà vu: the 2020 Facebook SDK crash

The last time there was a similar crash was in 2020, with Facebook. That May, apps such as Spotify, TikTok, Pinterest, and others also started to suddenly crash due to the Facebook SDK crashing all apps using it. Back then too, devs followed along on a GitHub ticket and they also found that bug: a value that should have been a dictionary but was a boolean:

Then as now, there was banter by devs being made to wait for a fix:

And requests to not move fast and break things any more:

Making light of the situation:

And also anticipating the resolution:

In the end, Facebook reverted the backend change, but shared even less than the bare minimum details from Google this time. This is all we know about that 2020 outage that was arguably more wide-ranging than the Firebase one:

I wonder if some people think that public-facing incident management is no longer important or valuable, even for developer-facing products. I’m not shocked that Facebook/Meta never bothered to communicate much about their outage because dev tools are not part of the DNA there.

But with Firebase, I am surprised that more than a week later, the postmortem is still not visible on the Firebase status page.

And maybe this is Google “shipping their org chart” playing out, live. The outage technically was caused by Google Analytics (who made the feature flag change), but is the responsibility of the Firebase SDK (whose iOS SDK was not hardened enough to deal with this new payload). The outage itself was buried inside a Google Ads dashboard (!!) which suggests that whatever team is seen responsible for the outage is inside the Google Ads organization.

In the end, despite the Firebase team committing to “improving status dashboard latency and coverage,” last week, those teams are in no hurry to carry out this work. AI agents might be making lots of work more efficient, but following up on action items seems to move at the same snail pace at Google, as it did pre-AI!

Read the full issue of The Pulse this is from, or check out this week’s The Pulse. This week’s issue covers:

1. New trend: building internal vibe-coding platforms at mid-sized companies. Ramp and Stripe built platforms for non-engineers to build internal websites and tools with, and both are taking off in those workplaces. I expect more companies to do the same.
2. Do us engineers really enjoy hard problems? Or do we actually like pattern-matching with backend problems? A provocative post by Cloudflare engineer, Sunil Pai, suggests there are other motives.
3. New open models launch in the EU and US. Kolibri, Mistral Large 4, and Beam by Reflection could challenge China’s dominance in open weight models.
4. Industry Pulse. Why Figma doesn’t let any agent use its MCP server; Google Cloud adds Swift support on the server side, Anthropic’s two-week sprint to speed up Claude Code, Coinbase dumps React Native shortly after Shopify announces doing so, Claude Opus 5.5 formats the C: drive, and more.

Before we start: if you happen to be in San Francisco on Thursday, 5 November, join me on the System Update with The Pragmatic Engineer event. This is an evening with OpenAI, Linear and DoorDash and myself, organized by Sentry. We get into what AI-augmented automations they’re running in prod, and how it’s going, in an off-the record (that is: not recorded!) and raw conversation. Seats are limited, and you can RSVP here.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from the last week’s issue of The Pulse. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

The creator of Ruby on Rails, David Heinemeier Hansson, caused quite a stir last week with comments in his Rails World keynote, when he revealed that coding by hand is dead at his company, 37signals.

This is a big deal because 37signals created Ruby on Rails, and they are known for their software craft there, especially when it comes to code quality. It’s also a business that’s 27 years old and is profitable. Despite that pedigree, DHH caused a stir among the dev community, saying:

“At 37signals, a couple of weeks ago, we made the decision that it clearly means we’re done writing code by hand. We have gone pencils down on the idea that we were gonna write code by hand, as a normal course of business creating things.

Writing code by hand at 37signals is now an exceptional state. It is like seeing a bug in Sentry: something here went wrong; why was the agent not able to produce what we wanted? Okay, maybe for a little while, we’ll still get the old pencil out and dot it down for them, but then we fix the machine, we fix the factory, we get things going again. This is a recognition of what’s already happening.”

DHH compared the maturation of AI tools into being highly capable at coding with the impact upon the craft of painting of the arrival of the camera:

“On November 24th, 2025, we got the “Kodak Brownie” of our era. We got Opus 4.5. AI technology, accessible in a harness that many people could afford to use and experience for the first time what it’s like to create software in pairing with a new form of intelligence. This was the tipping point for me. There was everything before November 24th, and then there was everything after. This is going to be the date that history books going forward will mark as the inflection point for the age of agents.”

He shared how 37signals has embraced a future where coding by hand is almost entirely absent:

• Embracing native mobile apps instead of web: famously, 37signals is bearish on native iOS and Android apps and has built web versions instead. With AI, they are betting on native apps being much easier to be built with a small team and are already building new ones.
• Moving backend services to Rust, not Ruby: This is due to performance reasons and because agents write good enough Rust. That’s remarkable to hear from the creator of Ruby on Rails!
• Ruby on Rails remains for web apps: 37signals is not leaving RoR behind, but only because Ruby on Rails’ convention-over-configuration design makes it easy for agents to work with it.

DHH closed by revealing that he no longer even thinks of himself as a professional programmer (emphasis mine):

“I have retired from being a professional programmer. I think it was somewhere around 4 to 5 months ago, maybe March. I spent a quarter of a damn century chiseling code by hand and loving every moment of it. This is not something to look back upon with regret; this is something to look back upon with joy and accept that it is over.

Writing code by hand is no longer an economically productive enterprise for the vast majority of programmers working at the vast majority of companies. On the other side of that is a new career as a professional maker of things, steering intelligence that was only available in science fiction up until a few moments ago.

One of the things we’re gonna have to revisit is everything we think we know about software architecture. The main tool that we’ve used for a very long time is abstractions. Abstractions don’t make quite the same sense in the age of agents. The reason we did abstractions was in part not to repeat ourselves; well, now the price of repetition has gone to near zero.”

It’s worth noting DHH’s keynote chose a spicy topic for a conference attended by engineers who are personally and professionally invested in the craft of building software!

Decline of coding by hand is long predicted

In the first issue in The Pragmatic Engineer this year, on 6 January, I wrote:

“When AI writes almost all code, what happens to software engineering? No longer a hypothetical question, this is a mega-trend set to hit the tech industry. (...)

The bad news is that change will probably be rapid. It’s barely been a year since the idea of Claude Code was born in Boris Cherny’s head, and already similar tools like OpenCode, Codex, Factory, Amp, Cursor, and more capable agents are changing how software is written. Change has always been part of working in tech, but I cannot recall it being this fast, or happening across the whole industry at once!”

I concluded that this change was on its way, based on my own experience of building software with Opus-4.6 and GPT-5.2, and from talking with experienced engineers who had resisted “AI hype” for good reason, but who had come to see that AI can now generate code that’s “good enough” in many cases.

Back then, I made a few predictions about what will happen when AI agents are producing most of the code for engineers:

• Sloppier code
• Weak software engineering practices hurting sooner
• “Coders” who are not software engineers see less demand
• Tougher work-life balance for engineers
• Junior engineers pushed to become seniors, fast
• Computer science education increasingly required for new hires
• A massive explosion in code and software, for which someone must be accountable

So far, it’s a messy transition and we engineers are responsible and accountable for a lot more code that we didn’t write, but which is in production anyway.

Non-engineers also getting into agents

At the end of January, I shared a deepdive that was pretty close to home for me: my brother’s 30-person, 15-engineer startup, Craft Docs, made its own sharp pivot to AI by building their own AI harness for non-engineers – called Craft Agents – two weeks before Claude Cowork was released, and months before ChatGPT Work launched.

Craft resisted the temptation to use AI when it did not feel productive, but with the model releases of November 2025, they found LLMs are not only useful for coding, but also for non-engineering work like customer support. In the deepdive, I went into more detail about non-engineering use cases (which engineers enabled) like:

• Automatic triaging of bug reports with agents
• Data enrichments added to all workflows
• Customer support “skills” like processing feature requests
• The marketing team building websites without devs
• HR automating tedious work
• Finance automating personal workflows

Craft Docs seemed early to a trend that has become more widespread, by having both their own engineering and non-engineering folks onboard to an AI harness. Now, there are signs other companies are doing the same: at OpenAI, non-engineering units like finance, recruitment, and legal moved over to Codex in June 2026:

In some ways, it could be comforting to know that it’s not only software engineering where the tools and workflows are quickly changing: every other function in tech is experiencing the same!

It’s messy right now

Just last weekend, a rant by an anonymous engineer in Big Tech hit a nerve with many people in the industry. An engineer with the username voxium posted (emphasis mine):

“The state of engineering right now is horrible. It has been half a month since I started a new role at a big company.

Nobody knows anything here. The specs, code, tests, PRDs, tickets, resolution of those tickets, reports, etc., everything is made by Claude Code. Nobody on my team likes this.

They are being forced to ship as much as they can. I have heard multiple times from higher management that pushing code is not a bottleneck, so why are we slow?

People are working 12 to 13 hours a day just to press enter. Nobody is reading anything. Humans in corporate are doing nothing on their own.

Everyone, literally everyone, from an L1 to an L7 engineer here is doing the same thing. Talk to Claude.

There is no sense of victory. Nobody is resolving bugs. In reality, nobody is thinking anymore. Everything is done by LLMs. It is so soul-sucking.

I would not mind it, to be honest, if we were at least given the time to check out the code and see what is going where. But no, the goal is to just ship. No matter what happens.”

This post rings true because it is happening at many places where there’s more AI usage, engineers do “outsource” thinking to LLMs, and end up not caring about anything else except shipping something to production.

Quality in decline

Since the beginning of the year, the quality of software has been degrading pretty much everywhere, much of it caused by over-reliance on AI, or perhaps more accurately, the outsourcing of thinking and decision making to AI. In July, I moved my video podcast off of Spotify after a series of unexplainable outages, and Spotify’s engineering team seemed to take no real pride or accountability in fixing the root causes of the issue.

Only this week, Uber shipped a new feature to production in the Uber Eats app – a new way to select extras with your food order – with seemingly no QA testing:

Inside this new “add-ons selector” in Uber Eats, I noticed three bugs at once:

• “Choose up to 999”: no engineer, designer, or PM bothered to check what happens when a restaurant does not fill out a number on how many toppings to add, or adds a ridiculously large number. The most toppings that my screen allowed to be selected was six and not 999, so I could not even choose the option of 999 buns for my burger.
• Sloppy overflow. A rule of thumb, during my time at Uber, was that text will never overflow, even when localized. Basilcummaynaise (basil mayo in Dutch) broke this rule but still shipped.
• Functional bugs in the selector. I originally tried to order from my favorite Mexican place: a bowl with no rice or bulgur as the base. There’s the option to select “rice”, “bulgur” or “nothing” as the base, but selecting “nothing” counts as an extra side, and the app doesn’t allow the ordering of a bowl with no base.

I’ve used the Uber Eats app for years, and this was the first time I saw such a sloppy feature release. I assume that devs and PMs building it have all “checked out”, stopped doing proper QA, and assume that the agent will take care of all of it. There’s no other way to explain three bugs shipped to all customers but seemingly noticed by nobody until I posted about it. To the Uber Eats team’s credit, they reached out and are looking into fixing all three issues.

Software engineering to be more important than ever

I’m personally past the shock and grief stages of agents taking over the activity of coding. At first, I assumed this change would reduce the amount of work for engineers. But, counter-intuitively, that actually seems to be growing:

• We need to understand the characteristics of LLMs better. LLMs feel familiar as they can produce text in a way only humans could do before. But they are less reliable, still prone to hallucination, suffer from capability gaslighting, and many other problems. They can also be expensive and slow.
• New systems need to be engineered. Agentic “software factories” can now produce code, based on the input provided. But how is this code validated? How much can detecting defects or various issues be automated? This is a brand new area, and we need to build new types of systems, often based on old ideas. One such example is OpenAI’s software factory, the other one is Ramp’s Inspect internal coding harness.
• Nondeterministic LLMs can generate deterministic code. An area I feel is under-explored and under-appreciated is the use of LLMs to substitute LLMs usage in agentic “software factories” with deterministic code they generate. For example: instead of running AI code review that is expensive and slow on all PR requests, could AI generate linters that catch the majority of common issues? If this is possible, complex lint rules would run faster, be more reliable and cheaper to execute than LLM calls.

New categories of systems and products will be built by engineers who “get” LLMs and AI engineering. We are seeing the majority of venture funding pour into AI companies because AI creates new business models, new revenue streams, and disrupts “traditional” software. For example, who would have thought that companies would spend tens of thousands of dollars, per engineer, on AI coding tools? Or that the category of AI inference providers would become as massive as it already is from barely existing two years ago?

This technological change will re-jig parts of the tech industry: the winners will surely win big, and teams and companies choosing inaction could be out-executed and displaced by nimble competitors. And in many ways, this is great news for us software engineers who keep up with the technology. Companies are now investing in innovation and are willing to pay top-of-market for software engineers who can help them build AI products or become AI-native.

Read the full issue of The Pulse this is from, or check out this week’s The Pulse. This week’s issue covers:

1. Firebase: global outage & poor handling by Google. The Firebase iOS SDK crashed after a backend change, crashing all apps which used Firebase analytics for 2-6 hours. Google did not update the status page or offer any postmortem, which is a head-scratcher from a company known for standout incident management practices.
2. OpenAI’s platform play from AWS playbook? OpenAI is becoming a platform where it’s possible to allocate ChatGPT spend on open models and AI offerings from among 16 partners, not just OpenAI models. It’s fair to ask if Anthropic will consider a similar platform play.
3. More data on companies moving to open models. Vercel’s AI gateway shows 60% of model spend goes to open weight models, and OpenRouter also shows open models are being more used than closed ones.
4. Why CTOs and VPEs are quitting en masse: another take. What if it’s not “founder mode”, but about people who love building software, feeling like they can do it solo (or with a small team) with AI tools?

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from a past issue of The Pulse. Full subscribers received the article below fourteen days ago. If you’ve been forwarded this email, you can subscribe here.

I was at dinner with a bunch of CTOs and Head of Infrastructure folks recently, and from the conversation it was clear that many companies are struggling to source CPUs in the current climate, and are coming to terms with the end of juicy discounts from cloud providers for machines in the new era of surging demand fueled by AI.

The ‘memory crisis’ afflicting sectors like video gaming is well established and has been extensively covered in terms of shortages of GPUs, but now it seems like things are just as hard for businesses in need of CPUs from cloud providers.

In a sign of how things are changing, the disappearance of CPU spot pricing was mentioned at the table. Customers used to be able to pay up to 90% less than the standard price for CPUs, as cloud providers slashed CPU prices for machines that were lying dormant and unused. But that’s no longer the case. It seems that spot pricing has vanished because there’s no longer any lack of demand for CPUs – quite the opposite.

I was surprised, but a lot of people chimed in; apparently, it’s now nearly impossible to get CPUs on spot instances without long-running connections with cloud providers. Also, reserving specific CPUs now needs to be done months in advance, and cloud providers will even turn down certain reservations because they don’t have enough CPUs or the right type of CPUs.

Even big players struggle to reserve CPUs

I have asked turbopuffer CEO Simon Eskildsen about their experience of CPU availability in the cloud, since turbopuffer, as a product, runs on CPUs, not GPUs. They operate in AWS, GCP, and Azure, so I asked how easy it is to get CPUs these days. Simon’s response:

“Getting CPUs is not easy anymore. As Reinforcement Learning (RL) is becoming a large amount of the workloads: RL needs a lot of CPUs. So the labs are sucking up a lot of CPUs. During RL, they need to teach the models how to do things, like searching, and then they need the model to run software, which then takes CPUs to run.

Then, outside of RL, agents need to do all kinds of very general purpose things on a CPU. So as the demand curve is shifting to general purpose agents, CPU demand is also going up.

Even the big companies are fighting each other for the right to get the CPU allocations. I would assume that it gets a lot worse before it gets better on the CPU side.”

I was able to confirm what Simon said about larger companies struggling; a VP of Engineering at a large inference provider told me they are at the limit on how much GPU and CPU capacity they can buy from their cloud providers. They have cash to spend and want to rent more capacity, and are willing to accept the longest leases. Despite that, cloud providers tell them no more is available!

AI hogging CPUs

Katelyn Lesse, Head of Platform Engineering for Claude Platform, has written about the reasons for the massive CPU demand increase:

“In the past few years, AI-fueled demand has skyrocketed, and these few companies suddenly needed multiple years and tens of billions of dollars to actually add enough capacity. We ended up with 3 separate bottlenecks in factory capacity that AI is exacerbating. At TSMC, GPUs are competing with CPUs (and with Apple, Qualcomm, and Broadcom) for production lines. And at SK Hynix, Samsung, and Micron, HBM [High Bandwidth Memory] is competing with regular DRAM for wafers.

What we’ve ended up with is CPUs getting squeezed from both sides. AMD doesn’t own fabs [semiconductor fabrication plants], so its CPUs need to come out of TSMC’s constrained allocation. Intel does own fabs, but it’s been working through yield problems and is now pulling some of its capacity from PC chips in order to make more server chips. And CPUs need DRAM which has gotten more expensive because memory production has shifted toward HBM. Analysts are expecting CPU supply to add more comfortable headroom before memory does, but their expectation is that it’s still going to be multiple quarters away.”

AI-fueled demand does increase CPU load, as shown in this graph from Uber, displaying the growth in agent requests over the past six months:

Increasingly, “agent requests” not only generate code which is inference-heavy – and therefore needs GPUs – but they also run tools that compile the code, run tests, run linters, and all of this is CPU-heavy. At companies like Uber, Ramp, and others, AI agents no longer run on the dev’s local machine, but on a dedicated instance in the cloud. So, the companies reserve more CPUs on their respective cloud providers for agentic workloads. We recently covered how Ramp built and runs its cloud agent, Inspect.

Basically, the problem is:

• AI applications use more and more CPUs, thanks to agents running a lot more software. AI data centers used to have a ratio of 1 CPU to 8 GPUs. Now the ratio is more 1:4, and it could shrink to 1:1.
• Companies that can manufacture more CPUs are busy on other hardware. TSMC is busy producing GPUs, which might be more profitable than CPUs. Meanwhile, CPUs also need DRAM, but DRAM manufacturers (SK Hynix, Samsung, and Micron) are instead producing high-bandwidth memory (HBM) because it’s more profitable. This is why memory prices are spiking; even Big Tech is unable to buy RAM, as previously covered.

To secure CPUs, it’s necessary to do capacity planning up to 12 months in advance. Katelyn says that server orders are being fulfilled in ~six months, instead of 1-2 weeks’ time as previously, and that prices are up by between 10-20%. So, it’s probably time for capacity planning. Katelyn:

“Most of us have never capacity-planned CPUs. We planned databases, we maybe planned accelerators if we needed them, and we autoscaled on-demand into CPU capacity as much as our budgets allowed us to. But general purpose compute is now something many teams will need to commit to ahead of time, which means you should probably start to forecast and plan around it. If you’re operating at scale, there are some things to spend your energy on.”

Using existing CPUs more efficiently is something to do, as of now. The CPU capacity shortage won’t go away, and any new CPU allocations requested could take months to turn up. So, what can we do if new capacity lags? One option is utilizing current resources more efficiently!

This is a great time to review and to establish now which services are CPU-intensive, and whether or not they need to be. Also check on services which are utilizing little CPU: can they run on fewer nodes, so that some CPU capacity can be allocated to services that need it more?

The best time to secure more CPU capacity is most certainly right now. I’m hearing rumors that certain cloud regions no longer accept new tenants because all CPU capacity is leased, or negotiations elsewhere are difficult. I’m also hearing that customers are already paying today to reserve capacity that will only come online in data centers from December. This seems predatory by providers, but demand is so high that this is how they likely prioritize new capacity allocation – while earning much higher profits than usual.

If your company has dynamic workloads, and you’ve used spot instances in the past, now could be a good time to allocate fixed capacity – even if it’s more expensive. If you expect meaningful growth, doing so now might mean having options at some cloud providers or in some regions.

It seems like this issue has spread everywhere as a corollary of widespread AI adoption. There’s a GPU shortage, memory shortage, and now a growing CPU shortage as well. Back at the end of last year, there was even an hard drive shortage. The only compute primitive not in short supply seems to be networking!

Read the full issue of The Pulse this is from, or check out this week’s The Pulse. This week’s issue covers:

1. Writing code by hand: is it over? In his Rails World keynote, David Heinemeier Hansson (DHH) declared the end for writing code by hand for professional work – at 37Signals at least. Is this change now unstoppable?
2. Amazon and Meta struggle to hire and keep engineers. Both Big Tech companies are scrambling to hire engineers who they previously laid off or enforced job reassignment upon. It seems experienced engineers remain in demand after all.
3. Opus 5.5 released and it’s good. Anthropic has released its new model that’s 40% the cost of using Fable 5.1 and has superior coding capability.
4. Code reviews to vanish sooner rather than later? Marc Brooker, Distinguished Engineer at AWS, believes that humans will have no role in routinely reviewing code by hand, and explains why this is all but inevitable.

Update: three weeks after publishing this article, Bloomberg covered the same trend in their article OpenAI, Anthropic costs push more startups to build off cheaper open models, covering the exact same phenomenon we discuss below. If you would like to stay weeks ahead of mainstream media for tech industry trends, subscribe to The Pragmatic Engineer and get trends like this straight into your inbox

Subscribe to The Pragmatic Engineer

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of five topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

In May, I covered an emerging trend of companies wanting to cut back their AI spending, starting with engineering departments. Different approaches were being tried:

• Experimentation with running cheaper, open models on inference providers
• More investment in model routing to route simpler requests to cheaper models
• Knowledge-sharing sessions on how to use AI models cost-effectively
• Setting per-developer monthly AI usage limits

A few months later, it seems that several companies have managed to achieve this, according to sources I’ve spoken with.

Uber: AI costs down 50%

Uber managed to blow through its annual AI budget in the first three months of this year, and it wasn’t a surprise to hear, in May, Uber’s COO say that it was getting harder to justify spending on tools like Claude Code without seeing benefits from the leading models. It wasn’t long until engineering teams at the ridesharing giant set to work on how to optimize AI spend, and their efforts weren’t in vain.

Uber cut the cost per AI request by 34%, and the cost per AI session by 52%:

Of course, Uber keeps using more AI tokens and starting more AI sessions, but thanks to optimizations the cost has been flat since March, despite significantly more usage:

How did they pull it off at Uber? It was via a combination of different approaches:

• Open weight models, run using inference: run open weight models on inference services, which are significantly cheaper than frontier ones.
• Optimized model selection: benchmark all available frontier and open models, to build an accurate picture of their present capabilities
• Ongoing benchmarking: run benchmarks every week based on real work, and update them
• Cheaper subagent models: subagents do smaller tasks not requiring the most expensive models
• Reduce model effort: Uber found that defaulting to Medium effort gives the best cost-to-output ratio with advanced models
• Optimize requests: trigger automatic compaction above 400K tokens, even for models with 1M context windows
• Cache prompts: cache prompts to save money when using Uber’s own harness, Minions
• … and more: Uber wrote an engineering blog post detailing the dozens of optimizations taken to reduce token cost without noticeable change in the quality of code generated by agents

From the outside, the single biggest win seems to be Uber’s transition to using open models for certain tasks. Open models cost 2-20x less, compared to frontier ones:

Pinterest makes 90%+ cost savings by dropping frontier models

Interesting details from Pinterest’s earnings call last month reveal how much the social media platform saves by running open models. Here’s what CEO William Ready said (emphasis mine):

“Our approach to model deployment includes our own compact fit-for-purpose models built for Pinterest-specific use cases and suitable open source models post-trained in our own environment within our secure cloud infrastructure. When we leverage open source models, such as with Pinterest Assistant, we are seeing superior performance for our use cases when compared to closed third-party models because we are able to post-train open models on our highly unique data.

With open models, we are achieving cost per transaction at less than 8% of the cost of comparable closed proprietary models. This gives us substantial headroom to deepen and extend these capabilities over time in a way that is differentiated, highly effective, and cost efficient.”

Basically, what used to cost Pinterest $100 to run on a closed, frontier model, they now spend $8 on by using open models on owned or rented inference!

AT&T: 56% savings by swapping Claude for open models

With 100,000 employees, AT&T is a big spender on AI. The telco giant cut its AI bill by 56% while measuring a 2% decrease in the quality of AI’s output, after they moved workloads over to open models. From The Information:

“Austin said he’s found that open source models are “just as good or better” than older models sold by the likes of Anthropic and OpenAI. For instance, AT&T’s software developers still rely on cutting-edge models for complex tasks like generating code, but can use cheaper open source models for less intense tasks like generating summaries of previously submitted code, he said.

After the company began using router provider LiteLLM, the costs of some advanced AI tasks such as coding fell by as much as 56% while the quality of the AI’s performance fell just 2%, Austin said.”

Anthropic overpriced compared to the rest of the market?

Only a few months ago, Anthropic was the preferred model (Claude) and harness (Claude Code) among engineers. But Anthropic’s models are becoming steeply more expensive at a time when open weight models – and also OpenAI – are getting much cheaper. Meanwhile, Opus 5 is 100x more expensive (!!) than models like GPT-5.6 Luna xhigh and DeepSeek. That may be simply too much to ignore for some tech companies:

Seeing this data, I’m not surprised that more tech companies are looking to run open weight providers on inference providers, due to the significant savings available from a model that’s similarly capable as one from Anthropic.

What worked for Stripe, Coinbase, Uber & Ramp

The engineering team at Databricks interviewed engineers at Stripe, Coinbase, Uber, and Ramp, and collected how different approaches helped save costs for them. The summary:

To answer the question posed in the header of this report, it’s apparent that using open models is indeed the approach offering the biggest savings, followed by smart model routing. Spending controls and context optimization also bear down on costs, but they don’t come close to the first two techniques in results.

A week after publishing this article, Ara Krahzian at Ramp has confirmed that AI spend in August, has, indeed, declined at the top 1% of businesses by 10%, based on Ramp data:

I’d wager those companies are not spending fewer tokens, but they are optimizing cost, in ways outlined above.

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. New trend of CPU shortages: after a GPU shortage and memory shortage driven by AI companies, we’re now experienceding a CPU shortage, thanks to AI agents using a lot more CPU with tool usage. If you will need more compute in the future: secure it now, while you can (even if it’s expensive to do so).
2. Growth dream ends for more COVID-era unicorns: Miro sold itself to Bending Spoons for $1.35B, after it was valued at $17B in 2022. Airtable saw a similar valuation cut last month, and it seems a batch of now-overvalued, VC-funded companies are desperate to sell.
3. Industry Pulse: Overtime at Google to get Borg working on SpaceX’s data centers; SpaceX cuts Claude Code tokens by 90%; OpenAI launches Astra; Meta unveils Muse (and Mark Zuckerberg pushed production code in this release); – to which Mark Zuckerberg made personal contributions; OpenAI’s agents go rogue, again.
4. Do engineers lose touch when AI handles incidents? In the aviation industry, pilots are exposed to emergency situations every six months, to keep their critical problem solving skills sharp. In the tech industry, we might need something similar, especially if AI would take on handling of the simpler incidents.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Two months ago, I asked why Meta appeared intent on destroying its engineering organization, at a time when the social media giant was reporting record revenue and profits. The question was raised after the company did two unexpected things:

1. Laid off 10% of staff. Executed large layoffs in May, with circa 10% of engineers shown the door.
2. Moved 20-30% of engineers to AI training. At around the same time, infra and product teams lost a further 20-30% of their engineers, who were reassigned to data labeling work for AI training.

The outcome of that period was low morale and a string of embarrassing outages, including a “zero auth password reset” outage on Instagram, where anyone’s account – including that of former US president Barack Obama – could be taken over just by asking the AI bot to replace Obama’s email with a different one.

Now, thanks to reporting by Reuters, new details have emerged about a plan for much larger layoffs, which eventually did not go ahead. The news report is pretty damning, and I want to get into what the planned AI job cuts reveal about Meta at this point in its history, what Zuckerberg might have been thinking, and what it could mean for other tech companies.

Making Meta “AI-native:” Project Organization Transformation

The plan was formed in January of this year. As per Reuters:

“In January, Meta CEO Mark Zuckerberg and his top lieutenants gathered for their annual leadership retreat at his Hawaii compound. There they hatched a radical plan to reimagine work at the social-media giant in the age of artificial intelligence.

Code-named Project OT – short for Organization Transformation – the plan envisioned an “AI native” future for the owner of Facebook and Instagram. AI would take over much of the daily work performed by thousands of human employees. Virtual workers would be overseen inside Meta by smaller, “talent-dense” cadres of human staffers, according to one internal planning document reviewed by Reuters and three people familiar with the project.”

The idea was that many existing teams could be reduced by 60% in their size through layoffs and reallocation of workers to other parts of the business. Underpinning this was the assumption that AI would enable these smaller teams to operate as well as before. HR at the social media giant projected that the project would involve a bigger layoff than happened in 2022-2023, when 25% of staff were let go. The new plan was to do one layoff+restructuring in May, and another in November.

I suspect a 30-40% company-wide layoff was planned.

But at the last minute, something changed. From Reuters (emphasis mine):

“But on the night of May 19, just hours before the first layoff wave, Zuckerberg blinked. Meta laid off 10% of its employees the next day, but it called off planning for the November cuts, according to one internal document reviewed by Reuters.

By then, Meta employees were in open revolt, convinced that the company’s AI transformation initiatives were partly aimed at replacing them.”

The Reuters report shows those employees were right: Meta’s AI initiatives were indeed aimed at laying off as many of them as possible, without changing overall productivity!

Even though these 60% cuts did not happen, some teams had 30-40% cuts and struggled to cope with their workloads. It also didn’t help that I talked with teams whose key engineers got reassigned to AI labeling: those were devs with critical domain knowledge that was lost after they left.

Why did Meta want 60% smaller teams?

You must assume that a company like Meta acts rationally overall, and on that basis it’s worth figuring out what the rationale might be in the case of ‘Project OT’. The article offers a hint: executives at the company had been captivated by “AI-native” businesses in Asia, Reuters claimed:

“Meta executives, including Chief Data Officer Alex Schultz and Head of Product Naomi Gleit, visited Asia last year and admired how startups there had built their organizational charts around AI, according to three people familiar with the trips. Meta executives also commissioned their own research into how AI startups were organized and set up pilot projects to determine what being “AI native” would mean at the company, according to one source familiar with the research and internal documents describing the pilots.”

And indeed, in February Meta experimented with “AI-native pods” as reported in The Pulse at the time. The presentation obtained by Reuters shows leadership intended to achieve 60% reductions in team size with small, 3-5 person, “AI-native” teams doing the work of what had been between 10 and 20 people:

And Meta is probably on the money that engineering teams are becoming a lot smaller at startups – and “AI-native companies” are also getting smaller, by size. But those are companies that are growing slower, without ever having done mass layoffs. Meanwhile, Meta seems to have wanted to become smaller not organically, and over time, but with a brutal layoff and sudden reassignments, in the span of a year, ignoring the impact such a sudden change would have on the company, teams, and employees.

Downsides of tiny teams

In theory, a smaller team could work better with less communication overhead and quicker decision-making, so there are cases where the upsides of a small team outweigh the downsides. For example, if there’s a small team of very senior folks with sound judgment skills, outstanding domain knowledge, and who don’t care about growing professionally anymore. However, such expected gains would come with several real costs, mostly associated with losing so much experience and skill:

• Domain knowledge: A lot of personal domain knowledge is suddenly gone.
• Redundancy: what if someone is on vacation, another is sick, and the other has an urgent appointment that cannot be moved? In a 10-20 person team, it would mean business pretty much as usual. But in a 3-5 person team, you’re down to two people doing everything!
• Capacity for oncall: a healthy oncall schedule needs 6+ engineers if every alert is to be taken seriously by an engineer whose main focus is oncall and systems stability.
• Lack of “slack time”: Innovation often comes from having time to focus on other work, instead of putting out fires. A bigger team naturally creates more “slack time” that can be used for other things, like university recruiting events, writing engineering blog posts, working with other teams on building things together, etc. In contrast, one that’s stretched thin with a maximum of 5 people or less gets almost zero slack time.
• Professional growth: engineers pair with more devs and get more feedback on larger teams. There’s more discussion and generally more opportunities to learn.
• Judgment: For honing one’s skills, there’s more experience and mentoring to be gained in larger teams than on small, “AI-native” ones where engineers spend the most time with AI. How good is AI’s judgment, anyway?

Is Zuckerberg’s worst fear being out-executed by a startup?

With Meta’s business posting record revenue and profits, and facing zero pressure to radically change how the social media giant operates, it’s worth asking why the social media giant was in a rush to get to 60% smaller teams. It would be a significant challenge for the 75,000-strong company, unlike for some small startups with under 100 people.

My hunch is that Mark Zuckerberg is paranoid about a startup which executes better and that could “destroy” Meta at some point. After all, this is exactly what Facebook did, back in the day. In 2008, Myspace was the king of social networks and Facebook was only a small player – yet three years later, Myspace’s usership had collapsed. The Huffington Post analyzed the collapse (emphasis mine) at the time:

“Just over three years ago, in the spring of 2008, Myspace was top dog. That April, the upstart Facebook grabbed the lead and never looked back. In those three years, Myspace has lost over forty million unique visitors per month, lost both co-founders, laid off the vast majority of its staff and more generally, has diminished to a cluttered afterthought of the power it once was.

In an interview with Businessweek, former founder Chris DeWolfe blamed Myspace’s overenthusiasm and underexecution on the product side for many of the site’s problems.

“We tried to create every feature in the world and said, ‘okay, we can do it, why should we let a third party do it?’” said DeWolfe. “We should have picked five to ten key features that we totally focused on and let other people innovate on everything else.”

Instead, Myspace unleashed a slew of products that were buggy and dysfunctional and confusing and alienating to users, and which couldn’t keep pace with Facebook’s own progress.

“[Myspace failed] to execute the product development,” former Facebook president, Sean Parker, said in a recent interview. “They weren’t successful in iterating and evolving the product enough, it was basically this junk heap of bad design that persisted for many, many years. There was a period of time where, if they had just copied Facebook rapidly, I think they would have been Facebook. The network effects, the scale effects were enormous. There was so much power there.”

Ironically, Myspace’s desperate attempts to recoup its former success came in the form of imitating Facebook, a site it’d once tried to set itself apart from. It adopted the news feed Facebook had popularized, and neatened up the site itself in a way that also suggested it was taking visual cues from Zuckerberg’s page. In November 2010, the site integrated with Facebook Connect, calling it “Mashup with Facebook.”

Myspace had twice as many employees as Facebook (around 800 at the time), and grew faster than Facebook in 2003-2007. But Facebook out-executed Myspace by being more nimble and more focused. Zuck’s business has seemed to try and be like a startup in its nimbleness of execution since then, not wanting to give anyone the chance to disrupt it like it did to Myspace.

If so, is Zuckerberg being paranoid about a similar threat to Meta today? Myspace was far from being the decades-old company that Meta has become! It was only founded six months before Facebook and got more traction in its early years, but fumbled execution as it grew. In contrast, Meta is today the tenth largest publicly traded company by market capitalization in the world, with a $1.4T valuation.

Then again, maybe there’s cause for Zuckerberg to be paranoid: Anthropic, only five years old, with one twentieth of the workforce Meta has, and might be going public at a close to $2T valuation as soon as October. Anthropic is not a direct competitor to Meta – it’s not a social media company – but Zuckerberg clearly sees AI companies as a form of competition to Meta’s business model. After all, every minute a person spends chatting with an AI chatbot like ChatGPT, they’re not spending it on Instagram, Facebook or WhatsApp.

Maybe this is one reason for the forced reallocation of 20-30% of software engineers to do data labeling and other training tasks on Meta’s AI model. To Meta’s credit, Muse Spark is a pretty capable model, and while it is behind the likes of GPT-5.6 and Opus 5, it’s already ahead of Google’s AI models – no small feat!

Most valuable assets: people or GPUs?

Let’s consider how Zuckerberg might respond if he perceives these things:

• Smaller teams execute better with AI
• These smaller teams can out-execute Meta: like Anthropic has done with AI model development
• There’s a danger that Anthropic and OpenAI could do with Meta, like Facebook did with Myspace

One approach would be to lay off 20-40% of the workforce, but there are consequences:

• Workers reject being treated like “cattle”. Meta’s “Project Organization Transformation” assumed that productivity would go up if teams greatly shrank and used AI tools. But would this happen? When people realize 60% of their colleagues were reassigned or let go because of AI, they might look further ahead: will another 60% be laid off at some point for the same reasons? Work could start to resemble the “Hunger Games”, where people have job security only until the next model release.
• Engineering is officially a cost center, not a profit center. We previously covered how most tech companies treat engineering as a “profit center” that generates revenue, and is therefore worth investing in. At such companies, engineers are treated well; not just financially, but in how leadership treats them as a key part of the business. At Meta, software engineering became a cost center pretty much overnight!
• Mission, what mission? People often join a company and stay motivated over time due to a mission they personally believe in. What if next year’s mission is to lay off as many people as possible, or to survive future culls? That doesn’t seem like a very inspiring mission.

What makes Meta worth its $1.4T valuation, anyway? Meta generates $228B annual revenue, and $68B profit (net income). The company is valued 6x its annual revenue and 20x its annual profit because investors bet its revenue and profits will continue to rise. But how does this happen? It’s via advertising, innovation, and launching new products.

How do you promote and enable the innovation which creates the products of tomorrow? “AI-pilled” folks might look to the technology as it gets more capable. But AI-native companies which can innovate will achieve results faster, putting Meta behind the likes of Anthropic, OpenAI, and SpaceX.

Or you develop a smaller workforce full of entrepreneurs and innovators, who will invent these new approaches and products. Basically, the best employees need to be motivated to stick around longer term.

That’s the problem with large layoffs; they prompt precisely the best employees to quit to join competitors, or launch their own businesses. This happened with Meta’s previous layoffs, as covered two weeks ago in ‘Meta’s self-inflicted resignation wave’. When leadership declares the ‘bottom’ 20-40% of the workforce is redundant, then very few people feel safe, and key members of Meta’s engineering organization will get offers from AI labs and Big Tech rivals. This is the “resignation wave” in action, all started by May’s layoffs and forced reassignments.

Knowingly or not, Meta creates an internal “mercenary” culture, where more of the people who stay are in it for the money and little else. Everyone knows they could well be laid off at any time the AI becomes good enough to replace them. People cannot control whether they end up on a list of positions to be cut, so it’s sensible to just make as much money as possible while awaiting the seemingly inevitable. That sounds like a pretty miserable place to work.

It could also lead to a situation where the workforce becomes more populated by those with no better options, who are not in demand from other companies.

Do social impacts matter to Meta?

A final element of the planned drastic job cuts, as revealed in the Reuters report, that I want to touch on is the potential wider, external impacts. Honestly, I’m surprised that none of Meta’s leadership seems to have considered this angle.

By executing massive layoffs for the sake of AI, Meta could have invited more regulation of the emerging AI sector. Meta is one of the largest tech employers in the US, and the CEOs of Anthropic and OpenAI are on record for predicting mass unemployment, and calling on governmental intervention should it happen. In fairness, other tech companies have also held major AI-related layoffs; Block let go 40% of its workforce, about 4,000 people in February.

But Meta is not just another tech company: it’s the world’s leading social media company, and just lost a major US lawsuit alleging that its platforms harm children and faces an $18B fine. As a result, it has committed to make its platforms less addictive. If the planned cuts had happened at Meta and all those staff became unemployed, how would it have dealt with complying with the court’s ruling to make its products safer for children? Could AI be relied on to deliver this with much less human input?

In light of the recent legal defeat, it wouldn’t be a good look for Zuckerberg’s company to blatantly put profits ahead of people by dismissing a load more workers. The combined effect of the two events would create terrible optics. In response, the government could decide that Anthropic’s and OpenAI’s CEOs were right about the threat to society of mass job losses and roll out things like:

1. Stricter employee protection, specifically around AI-related layoffs.
2. Higher taxation on profits of companies that “replace” staff with AI, and channel the revenue to deal with widespread whitecollar unemployment
3. Start taxing AI at source whenever tokens are sold, and use the surplus to counter the social ills of unemployment

Or the government could do nothing and leave it to the market to deal with unemployed whitecollar workers by creating enough new companies to employ highly-skilled software engineers, PMs, designers and other folks.

Overall, the canceled plan to cut thousands more jobs in this climate just adds to the feeling that there are no adults running Meta. Since Sheryl Sandberg quit in 2022 as Chief Operating Officer and Zuckerberg’s “right hand”, Meta has acted irrationally, irresponsibly, and unpredictably:

• 2022-2023: laying off 25% of staff
• 2024-2025: immediately rehired even more people so that 2025’s headcount returned to 2022 levels, raising the question of what the point of layoffs even was
• 2026: on track to overtake Google as #1 in advertising revenue this year
• May 2026: conducted sudden layoffs and forced reassignments of engineers, while aiming to replace as many devs with AI as possible for no obvious reasons like external pressure or competitive threats.

I’ve long had a generally positive view of Meta’s engineering culture, but with the company’s leadership seemingly worshiping AI and holding their colleagues in disdain, you have to wonder which software engineer would choose to work at Meta if other options are available. There are many tech companies that value their human software engineers, understand that great teams make for great companies, and that AI is a tool and not a replacement for human energy, motivation, and thoughtfulness. But Meta is clearly not among them.

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. New trend: tech companies moving to open models. Uber, Pinterest, Stripe, Coinbase, Ramp, and AT&T are making large savings on their AI bills by dropping proprietary models and using smart model routing.
2. Automatic software maintenance experiments by Linear and Anthropic. Both startups are experimenting with how far they can push AI agents to automatically fix bugs and remove tech debt. It’s working better than anyone might’ve expected in the recent past, but not producing code that can be merged without review.
3. Frontier AI lab wars: OpenAI pulls models from SpaceX / Cursor. With SpaceX now a frontier model and rival to OpenAI and Anthropic, OpenAI has pulled its GPT models from Cursor. This isn’t an option for Anthropic which is dependent on the SpaceX compute they rent to serve Claude.
4. HR tech startup’s one-dev-per-project approach. A full-remote HR startup with 70 engineers has a single engineer run each project, and says the approach works well. Will this approach be adopted elsewhere, especially at other full-remote startups?
5. Industry Pulse. Meta moved over to Slack for better agent interoperability, layoffs at Uber and PagerDuty, Anthropic upsets users by calling a rate limit decrease an “increase”, token usage explodes on OpenRouter, AI drives surging demand for Apple’s Mac Mini & Mac Studio, and more.

The pace of change in software engineering is only accelerating, especially since January of this year. This is all to do with the industry-wide adoption of LLMs, AI tooling, and AI infrastructure. At The Pragmatic Engineer, we always want to capture as much of what’s going on as possible at startups, Big Tech, and other businesses, in order to share with subscribers. That’s why we’re launching an essay competition with a chance to win up to $10,000, this week!

Send us up to 10,000 words on how you see things at your startup or tech company, or at a business you’re connected with. The best entries will be published in the Pragmatic Engineer and win prizes of $2,500, plus one grand prize of $10,000 for the best essay we read.

The Pragmatic Engineer reports many developments in tech in our deepdives, including how building software is changing at Anthropic, why Ramp built its own in-house coding agent, how Uber uses AI for development, how Codex is built and what it could mean for the future of software engineering, and much more. Almost every company we talk with currently says that processes which worked fine for years are suddenly pretty outdated.

This pace of change makes it hard to cover the full breadth and diversity of tech in one deepdive per week – especially right now, when engineers are building and using new and better tools in every part of the software development lifecycle. 

So, we’re launching a writing challenge to encourage software builders to share how things on the ground are changing. We’ll share these cutting-edge reports with readers in future editions of the Pragmatic Engineer.

So, tell us what’s new, different, better, and worse in your part of the tech industry since AI entered your workflow! 

What we’re looking for

• An essay written for software engineers: 3,000 – 10,000 words on one of these topics:
  • How software engineering is changing at your company, institution, or open source project
  • The engineering culture at your company, institution, or Open source project in 2026
  • How you build software in your company, institution, or open source project
  • Software engineers turning into AI engineers at your company, institution, or open source project. Basically, how the work is changing
• First-hand details: Winning essays will include the authors’ personal observations and opinions.
• Include details about the team and company: Including details about the team and company in the spotlight is important and enriches the essay by painting a more complete picture for readers; what engineering practices are changing, what’s still the same, what about tooling, and processes? Is there any “traditional wisdom” that no longer holds true – or is still true today? It’s not necessary for writers to work at a company/team being written about.
• Software engineers and “builders” only: We want to hear from folks who build software; meaning that you have built from scratch, pushed to prod, or been paged when your service (or someone else’s) caused an outage. Basically, if you do this currently, or did it in the past, then your essay is eligible for entry. You don’t have to be a builder right now if you’re still close to those who build software.
• Individual work. This competition is specifically for individuals, so articles written collectively by teams are not eligible. Of course, outside of this competition, The Pragmatic Engineer always welcomes articles from teams that want to share interesting developments!
• No AI writing allowed. Writing is thinking, and we are interested in your own thoughts and observations, not a sharp, one-sentence observation turned into three paragraphs of blob text by an LLM. Using AI as a research tool is allowed if you’re confident about the accuracy of its output, but all the words must be written by a person.
• Submitted essays must not be available to read elsewhere before 20 October. The essay has to be purpose-written for this competition and cannot have been published online or in print prior to 20 October 2026 in order to be eligible. After this date, a submitted essay can be published on any company blog, personal blog, or other medium. In fact, I recommend publishing it – one goal of this challenge is to increase general awareness of how the industry is changing, covered from different viewpoints.

Prizes

The competition has the following prizes:

• Grand prize: $10,000 and publication in The Pragmatic Engineer (following editing), and a Pragmatic Engineer surprise package. For winning entries, we reserve the right to make edits for legal, editorial, style, grammar, and punctuation reasons.
• Runner-up prize(s): $2,500 + sharing an excerpt in The Pragmatic Engineer + a The Pragmatic Engineer surprise package. Focus of the runner up category includes viewpoints from:
  • Bootstrapped (non-VC-funded) companies
  • More “traditional” companies that not digital-first, or which even predate digitalization
  • AI-native startups founded in 2023 or later which have been using AI tools since day one
  • Academia, government institutions, and open source projects

For quality deepdives: We plan to publish a roundup of essays in the future on topics which contain interesting learnings, and that clearly show great effort on the part of authors. These pieces will receive a The Pragmatic Engineer surprise package.

Timeline

• End of day, 4 October, PST: deadline for all submissions.
• 20 October: decision on the prizes.
• 3 November: expected publication of the Grand Prize winner

FAQ

Why is AI writing banned from this contest? I want to use AI to write my essay like I do for writing code! 

It’s a rule that you cannot include writing in your essay which was generated by AI. As everyone who reads this newsletter likely knows; there is no shortage of ways to detect AI-written text these days! The Pragmatic Newsletter is fully written by humans, and the same applies here. The reason for this hardline stance is that we believe in the value of human thought and experience, as told by people with first-hand knowledge. It’s also about the audience: The “audience” for AI-written code is usually a machine, whereas the writing in Pragmatic Engineer is for humans, by humans. AI produces a statistical average in its output which creates uniform, dull text that’s an inferior reading experience to human writing – at least today it is.

What about AI spellcheckers like Grammarly or similar ones?

These are fine to use, to eliminate those pestering typos. Just make sure they are your words!

Can the entry be anonymous?

No, because we only want to receive essays that are stories and observations that are yours to tell. And so we are only looking for entries that make the company/team/project/institution name clear, as well as your own name.

Of course, writing it with names mentioned could well mean getting permission to talk about your company/team/organization in public, and it also requires more skill to highlight not just the good changes, but the tricky, or not-so-good changes. This is part of the challenge.

Getting in front of The Pragmatic Engineer readers (and to enter this specific challenge) is not possible as anonymous.

Can I use images as well?

Of course! We love images and illustrations.

Can I publish an essay with the note that it's pending my company approving publishing it?

You can, but we treat it as unreviewable until we receive notice that it is approved for us to review. Basically, if the approval comes too late compared to the prize decisions to be announced on 20 Oct, the piece will not be eligible for prizes.

Can I publish my essay online after writing it?

In order to be eligible for consideration and prizes, an essay cannot be available to read anywhere online or in print before or after it is submitted to this contest. After 20 October 2026, no publication restrictions apply.

Can my essay be changed after I send it?

For spelling, punctuation, grammar, legal, and editorial reasons, the Pragmatic Engineer reserves the right to edit or amend all writing published on this platform.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

OpenAI put an impressive-sounding case study about how they helped Asana save $5.9M with a single migration. From OpenAI (emphasis mine.)

“Asana cleared 5 years of engineering work in 2 weeks with Codex. Using OpenAI Codex, Asana replaced an outdated testing system in two weeks for about $12K.

For this project, Codex helped Asana’s engineers remove Enzyme, an outdated testing system that had made the company’s code harder to upgrade. Model and infrastructure costs came to about $12,000, compared with Asana’s roughly $6 million estimate for the previous staffing plan.

After 1.5 weeks of engineering effort spread across two calendar weeks, Enzyme was fully removed. Model and infrastructure costs totaled about $12K. For comparison: the previous plan was expected to take at least five years and estimated to cost roughly $6M. The experience changed which long-running software projects the company believes are practical to take on.”

For context, Asana migrated from Enzyme to React Testing Library, which indeed would not be a simple migration. However, their estimate of four engineers (each on circa $300K/year, according to OpenAI’s arithmetic), spending five years on the project, had me like:

But thinking about this for longer raised the question: what does a migration from Enzyme to React Testing Library even look like?

Enzyme to React Testing Library migration

Let’s take a simple test and see how it looks in various testing libraries.

For our test, we want to verify that a button increments a counter. Here’s our button in React:

Now, the test to verify this, in Enzyme:

And let’s rewrite this test in React Testing Library:

The two tests do the same, but they have completely different syntax! Let’s see just how different they are, with a side-by-side comparison:

The main reason for this difference is that the two frameworks use a fundamentally different approach to testing. Whereas Enzyme is oriented towards component testing (notice how the test operates on a component instance), the React Testing Library operates on the rendered Document Object Model (a data structure representing the HTML shipped to the client) so the test sees the whole rendered page, not just the component its written for. That’s why the testing approach will differ radically between the two frameworks, especially when testing complex user journeys. You can learn more here on the tradeoffs between the two approaches from the React Testing Library author.

It took Airbnb 6 weeks to migrate 3,500 tests with AI

Last year, Airbnb revealed how they migrated their Enzyme test suite of 3,500 component test files within six weeks with LLMs. The estimate of doing this by hand was 1.5 engineering years. Airbnb did the LLM-aided migration in a multi-phase process:

Airbnb’s team had to build loops to keep retrying migrations; once they did, 75% of files were migrated in just four hours, and the migrations were straightforward. They then built a more sophisticated refactor pipeline for the remaining 25% of tests; after building the pipeline, the new loop migrated most of the remaining tests (97%) in total, after running over 4 days. The remaining 3% was done with LLM input, with engineers finishing it in a week.

This was in March 2025, when the frontier coding model was Claude 3.7 Sonnet. Today, models are a lot more capable, such as the likes of GPT-5.6 Sol and Claude Fable 5.

AI makes impractical migrations doable

On the basis that it took Airbnb six weeks, I find it credible that it took Asana two weeks to migrate what is probably a similarly complex test infrastructure from Enzyme to RTL, a year later.

The time and $6M cost as quoted by OpenAI feels inflated. I assume the numbers were based on an estimate that a fulltime engineer could do a maximum of X tests migrated per day, where X was between 5 and 10. Then, calculate the number of engineering years this takes (perhaps 20 engineering years), and multiply by the cost of an engineer. You estimate a project like this when it’s an undesirable project you really don’t want to do as an engineer!

So, looking at it from this point of view: does it matter if the estimate was 1.5 years (Airbnb) or half a decade (Asana, hypothetically)? Or if the estimated cost was $1M or $5M? It would still be an impractical migration and a foolish endeavor: too long and distracting, at least, in the “traditional” way!

Pre-AI, years-long migrations were rare. In 2021, Sentry took 1.5 years to convert their frontend codebase from JavaScript to TypeScript. That was a migration of 1,100 files and 95,000 lines of code! Around 10 engineers worked on the migration, so if we assume a $300K-per-engineer cost, that’s a $2–4M cost for circa 95,000 lines of code.

Indeed, Asana’s reported $12,000 migration cost could actually cost even less if there was ruthless focus on cost optimization. What about using a model that’s 10x cheaper than OpenAI, like an open model running on inference providers? After all, why use the most expensive model, especially if a company already owns GPUs, running models, making inference practically free (except for power costs, that is.) For a first run, $12,000 would be affordable for a company paying $300K for engineers. But for subsequent runs, I’d wager it’s worth spending time optimizing the cost, and saving $10K per migration (or more!)

A few more details from inside Asana

I managed to catch up with Dan Ubilla at Asana, who leads the Developer Productivity group at the company. He helpfully clarified a few things about the Enzyme migration post on the Asana site:

• Asana started the Enzyme migration in 2024. The team had 4,000+ Enzyme files to migrate, and they took a first stab at the migration with LLMs, at the time. Beyond migrating existing tests, much of this phase focused on getting the codebase into a state where future tests could be written more idiomatically. This included improving mocking, data set up, and coverage instrumentation. They finished migrating about 25% of the files to RTL, choosing the tests that had the highest return on investment: meaning easiest to migrate, or most frequently updated ones.
• The remainder of the migration became low priority. Asana prioritized migrations as critical (“must do soon”), important (“important to do soon”), and opportunistic (“nice to have”). The company has a good number of critical and important migrations, and the remainder of the Enzyme migration sat as one of the many opportunistic.
• Five years was about when the migration was projected to finish. The effort was not about “a team working for five years, nonstop on the project” – not at all! It was about the timeframe that the opportunistic Enzyme migration would have been completed, given its priority, and done, realistically. Given this migration was going on for two years, the five-year estimate was a rational one.
• The team wanted to prove that LLMs drastically speed up migrations, and demonstrated it with an opportunistic , low-priority one. The Enzyme migration was not all that important to do quickly, and would have sat around for years. So it was the perfect candidate to prove that with LLMs, long migrations can be sped up!

And finally, one addition from my end:

The $6M cost was a back-of-the envelope estimation. I asked Dan how this estimate of the migration costing $6M came together. Dan confirmed that the estimation was done the same way as most of us do these estimations:

• Estimate how long it takes an engineer to manually migrate a single Enzyme file (including rewrite + validation). Be generous with this estimation
• Multiply it by the outstanding files to migrate (the 3,000+ ones)
• Add in the time to remove any traces of the framework, monitoring, static analysis tools
• Multiply by the hourly rate of a typical engineer
• … and the number came out to $6M

This estimation does not take into account that migrating even ten files by hand takes less than ten times more than it takes to migrate one file (you become more efficient), nor does it assume LLM usage. It’s a baseline to get a sense of how the work would have been done, pre-AI, if distributed across all Asan engineers to pick up, and migrate few files, whenever they have time. My take is that this number is probably an overestimation, but the point is less about the number, and more to convey that this is a lot of work!

Internally, Asana’s team and leadership are now convinced about the usefulness of LLMs for migrations, and they’ll be using them for other, long-running, otherwise soul-sucking migrations.

Expect long-avoided migrations to finally happen

A few months ago, Uber shared that they executed a massive JUnit migration in four months with two engineers and AI: moving 600,000 unit tests spanning 15 million lines of code (!!) by moving from the unsupported JUnit 4 to JUnit 5. During the migration, 1.25M lines of code were modified. This type of migration used to be impractical; with AI tools, it took eight engineering months of effort, plus AI costs.

And there’s the Bun migration (530,000 lines of code from Zig to Rust in two weeks for a $165K API cost) as another example of drastically faster migrations with AI.

The best part about rapid, AI-assisted migrations is not needing to worry about supporting “old” libraries. One of the main reasons library migrations have been such a pain was the need to keep supporting the old library or technology during the migration. Shortening this window could well be worth the additional cost, at least until we figure out how to drastically reduce the time and cost of these migrations, as well.

The shared characteristic of all of the above migrations is that engineers needed to plan for it, design verification loops, and be involved throughout. I always dreaded migrations, so I see it as good news that we have a new tool to use for “grunt work” like this.

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. Did Meta really decide to reduce team sizes by 60% because of AI? An in-depth report by Reuters details how Meta’s leadership decided to slash team sizes by 60%, hatching plans in January to execute the social media giant’s largest-ever layoffs. But Mark Zuckerberg changed his mind at the last second, and now the company is stuck with all-time low morale, a wave of resignations, and its prized culture turning mercenary. Analysis.
2. More thoughts on Ramp’s in-house AI infra. Building AI tools in-house is akin to an “internal bootcamp” for AI engineering, which probably justifies the exercise at most tech companies.
3. Industry Pulse. GitHub’s load increase is speeding up, Stripe acquires OpenRouter, Ramp launches Ramp Router in public, a career “double boomerang”, App Store revenue drops for the first time ever at Apple – and what is a bug?

This email is a response to the 50+ podcast pitches that arrived in my inbox just last month, and keep arriving in growing numbers, from PR agencies and marketing teams at various companies. It's also a message to tech founders building stuff for developers: focus on building something that devs love, or find epic, and not hiring marketing/PR folks who then mass-email podcast hosts with AI-generated podcast pitches.

I host The Pragmatic Engineer Podcast. Thanks to so many of you tuning in, the podcast is widely listened to – it's one of the top-ranked software engineering podcasts, and episodes typically reach 500K+ people. If you're listening to the podcast, or reading the newsletter: thank you!

The podcast only has guests who, as as a software engineer, I'm excited to talk to and learn from. These folks typically fall into one of these categories:

• Built/builds epic pieces of technology. Examples include Chris Lattner (built LLVM, Swift, Mojo), Anders Heljsberg (created C#, TypeScript, TurboPascal), Andrey Breslav (created Kotlin), Greg KH (longtime Linux maintainer), Kat Cosgrove (longtime Kubernetes release lead), Mitchell Hashimoto (Terraform, Ghostty).
• Works at a company I find very interesting and relevant. Examples include Boris Cherny at Anthropic (created Claude Code), Elizabeth Stone (CTO at Netflix: a company whose engineering culture I've been fascinated with), or Mai-Lan Tomsen Bukovec (leads S3 at AWS)
• Is a deep domain expert - typically with a decade+ of experience. For example, Nicole Forsgren (developer productivity, DORA, SPACE), Laura Tacho (DX), Alice Ryhl (Rust), Robert Erez (CI/CD)
• Deep thinkers with a decade+ of experience. For example, Kelsey Hightower, Grady Booch, or Charity Majors.

As the podcast has become more popular, I am getting overloaded with pitches from PR people, Comms people, and Marketing people, though. The typical pitch is usually AI-written, and goes something like this:

From <CommsPerson@Startup.com>

Hi Gergely,

I have a potential guest and topic that feels unusually aligned with The Pragmatic Engineer’s audience.

X is the co-founder and CEO of Startup, which now serves ## million developers and ## thousand companies. The platform processes {large number of business metrics} every day, giving CEO a large-scale view into how engineering ability and team expectations are changing as AI-generated code becomes standard.

The useful conversation would not be a Startup product story. It would examine questions engineering leaders are actively dealing with:

- AI-generated topic #1 assumed to be a fit for the podcast
- AI-generated topic #2
- AI-generated topic #3

CEO has built Startup from its earliest days in Location all the way to global enterprise scale. He/she can bring both operating experience and concrete data instead of generic AI predictions.

Would this be worth exploring for the podcast?

Best,

MarketingPerson
Startup

I get 10-20 of these pitches, per week, in my inbox. Until now, I've been ignoring them, but then MarketingPerson sends two more follow-ups. I now have an AI agent monitoring my inbox, and I decided to at least send an AI reply to explain why I'm not sending a response to these pitches.

If you look at how my podcast works, it should be obvious to why such guest requests are not a good fit - and why having founders wanting to talk about their own startup would be a bore of a topic, and, frankly, a waste of time for 99% of listeners – possibly more.

But to spell it out to the marketing folks:

A) Send out AI emails: expect nothing back. Don't expect a response to an AI-written email, or a templated email. I get it, you think it's a numbers game: send a tweaked email out to 100 podcasts, and hope that at least one replies. In reality, by playing the same game as everyone else does, you are assisting to the noise, and end up drowning yourself in it as well.

And this goes not only to you, as a marketing person, but also to a founder/CEO who thinks they can put in zero effort in A Thing and get great results: you're kidding yourself. To those founders who have set themselves the goal of being invited onto a specific podcast: I'd challenge you, ask a former guest, how they got invited? I can already tell you that it sure wasn't an AI-generated email sent out by a marketing person they hired!

B) My podcast doesn't take self-pitches. The only type of pitch I listen to is a software engineer telling me "wow it could be great to have this and this person on the podcast, because of this thing they built/thing they did." In these cases, I check out that thing they built or thing they did. When a lot of people tell me the same thing, I might listen more.

But a CEO recommending themselves as a podcast guest? Or their marketing or PR team recommending the same? Ignore.

But to leave these marketing folks with some advice:

If your CEO/founder has so much to say about a topic: say it and write it! Ironically, every time I look up what the founder has written about the topic they (or their marketing team) are pitching, there's nothing! Nada.

If the pitch is sincere, then write down your thoughts! Post it on your company blog or personal blog.

But if you're just looking for "distribution", "free publicity", mass emailing podcasts is not how to go about it. Everyone is doing it, and it's not working for anyone.

The only founders you see on podcasts who are worth listening to? They built some epic stuff, are often hands-on, and they did not mass-email dozens or hundreds of podcasts with an AI-generated, generic agenda to talk about.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Two months ago, I covered how Meta seemingly deliberately started destroying its once-standout engineering organization. The company did 10% layoffs at a time when revenue and profits hit an all-time high, and reassigned about 20-30% of software engineers to data labeling with basically no notice.

The result was that a good chunk of software engineers at Meta – even those not reassigned – were starting to interview elsewhere:

I said – and still maintain – that the layoffs were an unforced, self-inflicted error on Meta’s part. Coupled with forced reassignment, they struck disaster. And I’ve now gathered new details that confirm that Meta is bleeding top engineering and product management talent:

Some layoffs reversed, in the 11th hour

In the UK, mass layoffs require a notification to those potentially affected, before the cuts can happen. A few weeks after this notification, a good chunk of folks were “un-notified,” I confirmed. But those on notice had already started to look for jobs, obviously.

Retainer equity grants offered to IC6+ engineers resigning

Meta started to offer large retainer equity grants to those resigning and leaving for Google, Anthropic, and OpenAI: a practice not done before. Talking with long-time Meta engineers, Meta simply didn’t make counteroffers or negotiate when an engineer handed in their resignation. But this practice has been abandoned. Here’s a story of a senior, long-tenured engineer I confirmed to have happened:

• This dev was force-reassigned to one of the AI data labeling teams
• They thought “oh no, this is not what I want to do” and started to interview
• Got a Google offer, but compensation was below their current one. Google basically lowballed them.
• Decided to leave anyway. Told their new manager they were resigning.
• New manager asked for some time, then presented a large, one-off, retainer equity grant, vesting over 3 years
• The engineer went back to Google, telling the company, unfortunately, they are a key engineer and showed them the retainer amount Meta is paying for them to stay
• Google upped their offer, out-bidding Meta’s retainer offer
• The engineer who was desperate to leave – even for lower compensation! – was thrilled, and happily left for Google.

Retainer equity seems to be offered for most IC6 engineers who resigned in the last few weeks. I talked with seven such people, who were all IC6 (staff-level engineer) and IC7 (principal-level engineers), who shared more details. They are not aware of IC4 (mid-level engineer) or IC5 (senior-level engineers) getting such offers.

Retainer equity offered is $400K to $1M+, and vests over 3 years, for the folks who got these offers. The $1M+ offers were all for engineers with Anthropic or OpenAI offers, and I confirmed a $400K and a $600K grant for engineers with an offer for other, smaller AI startups.

A current Meta engineer who is interviewing asked me if these folks needed to present their offer letter to get a counteroffer. I asked two of these folks with counteroffers, and neither had to do so, but they both indicated they were handing in their resignation. Upon receiving this notification, their director, together with HR, offered a discretionary retainer equity, should they stay.

Anthropic and OpenAI on a hiring spree from Meta

Anthropic and OpenAI seem to be on a hiring spree, signing Meta engineers that Meta wants to retain now that it is too late! I have confirmed that three engineers who received an offer from Anthropic were also offered large, $1M+ equity grants (vesting over 4 years) as counter-offers to stay. Two out of the three rejected it and joined Anthropic, and the third one initially accepted the offer and then still left for Anthropic a month later, forfeiting this grant.

OpenAI is having similar success, and the two AI labs seem to be the main destination for now ex-Meta engineers to bounce to. It makes sense: these companies can match Meta’s total compensation, and although neither OpenAI nor Anthropic is publicly traded, both companies organize secondary equity sales, and their respective IPOs are likely to happen in 3-12 months’ time.

AI startups also successful in hiring infra experts from Meta

I had an in-depth discussion with another long-tenured AI infra engineer at Meta, who was considering whether to stay, or to go. This person wrote up their situation, and shared it with me, writing:

“With the current situation at Meta (layoffs, forced drafting to do AI work, general sense of disruption at work, low moarale, and almost zero productivity across teams, plus multiple reorgs) I started to look around, on the market, a few weeks before the 20 May layoffs. I figured I have a 50% chance of being let go.

I did not get laid off, but I got offers from startups, and one from a high-growth AI startup I like, and also an offer from Google.

Now, the pros to staying a Meta:Context, relationshipsScale that few others haveBig Tech perks, relatively “chill” WLB in terms of oncallHigh TC

The cons:Terrible morale, hard to “move” work through mostly unmotivated teams all around meRe-orgs continue, I have very low confidence in middle management (Directors, VPs) to navigate this churnFurther layoffs are likely: it’s clear that there’s more bloat, surely Zuck will do another round in 2027. And no one is safe during a layoffA ‘doom loop’ for anyone who stays: low morale → unmoitivated bith highly compensated engineers → a bloated organization that moves slow”

After a new reorg, this engineer got a supportive manager, who made it clear they would help them thrive inside of Meta. So the engineer rejected both the AI startup’s offer and that of Google, deciding to stay… but only for a month!

The AI startup’s founder spent more time with this engineer, increased the equity in the package significantly, and convinced this engineer that at the startup they matter as a person, while at Meta they would be doing soulless work, worrying about the next layoff, probably early 2027.

So this engineer also handed in their resignation. And it all started with the layoffs: the engineer assumed there was a 50% chance of being let go and wanted more career stability!

Those deciding to not resign: stay for the money?

There are engineers staying – at least until the end of the year, to wait for the stock refershers — which we’ll talk about in just a moment.

Another engineer I talked to decided to quit Meta with morale so low, and start his own thing. Management convinced him to stay until the end of the year, hinting at large equity refreshers being handed out around January, which could be in the $1M+ range for this specific engineer.

The 2022 stock price causing a TC decrease for many?

One reason there’s speculation about large equity refreshers being handed out at the end of the year is that Meta’s stock price was very low in 2022 and 2023, when equity refreshers and new hire grants were handed out. In both March 2022 and March 2023, the stock price was around $200 - which is about a third of this year’s stock price (which has been fluctuating between $520- 680, currently sitting at $540.) By March 2024, the stock price rose to $500, and then to $580 in March 2025.

Because of this, those who joined Meta in 2022 have seen their equity grants nearly triple, and the 2023 equity is worth nearly 3x as well. But for these lucky folks, total compensation is set to drop by the end of this year, and their equity refresher would need to be higher than in past years to avoid a compensation drop.

Inside of Meta, engineers who I talked to are split between expecting further attrition (those leaving whose total compensation drops steeply) and those who think that there will be more attrition unless there are higher-than-usual equity refreshers.

A “mercenary” culture at Meta?

I wonder if Meta made a massive mistake by treating engineers as “commodities” and turning their culture into a “mercenary” one. Between March and June, Meta treated engineers as replaceable commodities that could be thrown out or moved between teams, while stripping them of any autonomy. The reassignments to AI labeling were not explained, nor were managers or individuals asked about preferences: from above, someone said that Alice and Bob, starting on Monday, are no longer with the team, but are training up Meta AI. Never mind that these were the two most experienced engineers on the team, or that Bob was the team’s infra expert, and Alice the “fixer” engineer on the team.

And all this damage was done, for what? It was to allow Meta to restart its AI coding model development efforts, and release Meta Muse Spark: an AI model that is currently the 7th most capable AI model as per Artificial Intelligence Analysis, tied with Grok 4.5. Credit where it’s due: with this model, Meta is now well ahead of Google’s Gemini in capability.

With the damage done, there is little motivation for anyone to stay at Meta – unless they are on a visa, or if it’s about the money. You can have an OK engineering culture with a team who is mostly interested in making more money than what they would elsewhere: but this is more of a mercenary culture, and companies run by mercenaries are more easily out-executed by teams where people believe in the mission of the company.

If you work at a company that would love to hire from Meta: now is (still) your opening! So go for it.

And if you’re at Meta: you are so in-demand, outside of the company, at places where engineering is still treated as a profit center. As Ryan Nystrom, currently building AI at Notion, put it:

“Meta friends: the industry is exhilarating right now. Get out. Go build something. The handcuffs are an illusion.

I’m having the time of my life at Notion. Come, and have fun again.”

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. More on the “great engineering leader career break.” The industry is changing fast, and the VPE and CTO roles also need to adapt. And don’t forget that these are the roles from which you can drive change that reorganizes engineering in ways that work better.
2. We need to talk about migrations with AI. Asana needed to migrate off testing framework Enzyme, but it meant doing a massive rewrite of test cases. With AI, the project was completed in two weeks: without AI, this work would surely have been kicked down the road. Airbnb and Uber share similar stories, and AI seems like a superb fit for framework migrations.
3. Are AI startups making the Gartner Magic Quadrant irrelevant? Gartner ranked AWS, Microsoft and IBM above Anthropic, Cursor and OpenAI in their “AI code modernization tools” ranking. This is most likely because the first three pay large sums of money to Gartner, but AI labs and vendors refuse to pay this “Gartner tax.”
4. Industry Pulse. Another hours-long GitHub outage, GitHub alternatives are here and fighting for market share, Slack launches Slack Code, text generated by Claude to be watermarked, and Uber open sources SubmitQueue.

Read the full The Pulse.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from a previous The Pulse issue. Full subscribers received the article below four weeks ago. If you’ve been forwarded this email, you can subscribe here.

Last week, xAI (Elon Musk’s AI company, now part of SpaceX) released the Grok 4.5 model, built by Cursor (an acquisition), trained on SpaceX GPUs, and branded “Grok.” It’s a pretty good model; benchmarking as close in coding capability to Opus 4.8 and GPT 5.5 – while being 60-70% lower cost.

Grok 4.5 can be used via API, but is easiest used via the Grok Build coding CLI. So, that’s what many devs did. Some of them have noticed something really weird: the CLI is uploading all their local files in their working directories! Here’s an independent AI safety researcher known as ‘Cerblab’ documenting what is happening (emphasis mine):

“xAI’s official Grok Build coding CLI (grok), on a normal consumer login, does three things worth documenting precisely:It transmits the contents of files it reads — including a .env secrets file — to xAI, verbatim and unredacted. The secret appears in two channels: the live model turn (POST /v1/responses) and a session_state archive uploaded and accepted (HTTP 200) via POST /v1/storage — the endpoint the binary routes to the grok-code-session-traces GCS bucket (see section 5).It uploads the whole repository — every tracked file’s content plus git history — independent of what the agent reads. Grok packages the workspace and uploads it via POST /v1/storage. Proven directly: on a real codebase, with the prompt “reply OK, do not read any files”, Grok uploaded the entire repo as a git bundle (POST /v1/storage → 200); git cloning the captured bundle recovers a file the agent was told not to open — src/_probe/never_read_canary.txt — with its unique marker verbatim, plus the full git history (appendix uploaded_repo.bundle). And it scales: on a 12 GB repo of never-read random files, /v1/storage moved 5.10 GiB, all HTTP 200 (truncated mid-stream), while the model-turn channel moved just 192 KB — a ~27,800× ratio that pins the upload to the codebase, not to what was read. No storage upload failed; the only non-200s were a model-usage quota (402/429) on /v1/responses and one unrelated 404 — not a storage size cap.The storage destination is a Google Cloud Storage bucket, grok-code-session-traces (not AWS S3) — named verbatim in the binary and in a captured metadata.json (gs://grok-code-session-traces/…). I did not find this mechanism surfaced in the CLI’s install/quickstart materials (not an exhaustive docs audit — §7), it is active by default, and disabling “Improve the model” does not turn it off (/v1/settings still returned trace_upload_enabled: true; §6).

None of this proves xAI trains on the data — that is a policy question addressed in [section] 6. What is proven is transmission, acceptance, and storage.”

There’s so much wrong with this approach! To name a few:

• Sending your codebase over the context window is not normal. All AI agents send over their context window to the server that runs the LLM. That’s where tokenization happens and the context is appended to the session. This means that other AI agents send over some part of the code that they read in their context window.
• No need to send over the source code to index it. Indexing the codebase is important for efficient code lookup, and we covered how Cursor does this in a privacy-conscious way by indexing a user’s codebase locally, creating embeddings, and sending those embeddings to the server. Cursor’s server does not store any of the user’s codebase though. Except that Grok CLI transferred all users’ codebases to a cloud bucket! Given Cursor and Grok are now combined as part of SpaceX, it’s a real head scratcher why the Grok CLI isn’t doing what Cursor always has.
• Sending over unencrypted .env files is reckless. Local .env files store secrets, database access tokens, service access tokens, and more. These are sensitive pieces of information that need to be handled with care. If transmitted, they should be encrypted at the very least. Grok / SpaceX storing them on the GCP storage bucket – likely unencrypted – is flat-out unacceptable and reason enough for any sensible company to ban usage of Grok CLI.
• No good reason to upload git history. Sure, seeing Git history could be helpful when training an AI model.
• It’s malicious to not tell devs anything about it. Developers using Grok CLI haven’t been asked to opt into this data upload, nor notified of it. Most evidently had no idea this has been happening, and are understandably furious after Cerblab’s writeup went viral.

SpaceX throws devs “under the bus”

Caught red-handed, Grok CLI disabled file uploads with a remote feature flag. AWS engineer Wes Eklund started tracking upload functionality with the CLI, finding that the uploads suddenly stopped due to a feature flag being flipped by the Grok team, pausing data collection.

But the code functionality to stream all local files to the server, unencrypted, remained present in the CLI, even in later updates. Read an in-depth analysis by Wes.

SpaceX’s official response was pretty laughable, not explaining why .env files and .git history were uploaded, and adding that enterprise customers with zero data retention (ZDR) enabled were the only ones unaffected by underhanded, secret uploading of users’ local files:

My initial reaction is what a condescending response by SpaceX, swiftly followed by the question: does Grok CLI even have enterprise customers? It might be few, given how reckless the team evidently is by uploading unencrypted secrets! SpaceX CEO Elon Musk chimed in with a post that seemed to be almost trolling angry users. He wrote:

“SpaceX policy regarding data retention.

It is actually helpful for debugging issues if we can retain some amount of data, so allowing this would be appreciated, but your privacy settings are always respected.”

This makes it worse because SpaceX has been secretly uploading far more data than is “useful for debugging!” Uploading the git history and sensitive .env files is not, in any way, useful for debugging. Also, Grok/SpaceX did not upload “some amount of data”; it uploaded every last file it could find in your local folder.

The developer community is justifiably upset to read SpaceX and Musk pretending that Grok has only uploaded scraps of data purely for debugging purposes. This time, even fans of SpaceX and Grok are speaking out against Musk and his company’s behavior. AWS engineer Wes Eklund:

“Elon, firstly, huge fan of everything you work on. Completely understand the need for some trace data to improve customer experiences.

From what I’ve researched, it seems to be much more than just trace debugging issues. It seems to be entire code repos with sensitive information just collected entirely.

Your Google Cloud blob storage must have petabytes of code repos from us.

Not ideal.”

Sam Altman pushes Grok to open source Grok CLI

OpenAI CEO, Sam Altman, also posted, using a term Musk often employs when commenting on things he disapproves of in society, and hinting at the benefits of Codex which doesn’t upload your whole local filesystem, or mess with .env files and the git history. The Codex harness is open source, so secretive file-upload functionality would be visible in the source code:

Musk clearly read it and responded a few hours later:

A day later, (15 July), SpaceX did indeed open source Grok CLI. Altman’s comment seemingly hit home. Also, SpaceX has stated it is deleting data from its servers, writing:

“We disabled default retention for all Grok Build users starting on July 12th. Additionally, we are deleting all coding data that was previously retained, ensuring every user’s preferences are respected. With these steps, Grok Build goes beyond other major coding products to protect user privacy.”

The open sourced repo is rushed, unsurprisingly. Kernel engineer Elliot Arledge used the repo and reported what he found:

“Out of the box, `cargo test --workspace` doesn’t compile. 190+ errors, all one bug class: cross-crate test helpers hidden behind #[cfg(test)], which Bazel’s per-target test builds tolerate but Cargo doesn’t, because a dependency is never compiled in test mode. The default-bazel feature is declared in ~20 manifests and wired to nothing. Ungating the benign helpers and putting the signing-key test seam behind a dev-dependency-only feature gets the suite running for the first time on the published tree: 24,663 passed, 28 failed, and every one of the 28 is a pre-existing bug the broken build had been hiding (tests reading your real ~/.claude/settings.json, a tool missing from the registry, macOS /var symlink breakage, one Theme::current() race).

Questions for the team:

Did anyone try downloading this repo and running it before publishing?”

In fairness, it’s clear from the outside that the dev team was instructed to open source the repo ASAP, and did just that. I would expect improvements to allow the developer community to compile the repo and run tests will come later.

On a side note, I wonder what the mood within Grok is. A mandate to open source the product – that was not built to be open sourced! – and doing it in a couple of hours hints what it’s like to work there. Some folks doubtless would find it thrilling and a big challenge, while others likely find it pretty stressful having the “Eye of Sauron” (the CEO) upon them!

Can companies trust Grok now?

To hand it to Grok/SpaceX, the last 24 hours of this incident saw some impressive execution. In contrast, everything that took place beforehand screams “amateur hour”:

• Did no one in teams that wrote the functionality to upload the complete local codebase raise concerns that this would be unacceptable to devs?
• How did uploading secrets without encryption not raise alarm bells?
• Why greenlight any of it without opt-in and do it in a secretive manner?

For sensible companies generating revenue with software, vendors who are allowed to access their codebase are limited to those that can be trusted. Grok has demonstrated it is unprepared to handle codebases with security fundamentals in mind.

There’s now frantic backpedaling and rapid open sourcing, but my impression is that this is simply because Grok/SpaceX was caught red-handed, and understands the risk of losing enterprise contracts caused by this flagrant breach of trust. This is why SpaceX’s communication mentioned that enterprise customers with zero data retention have not been impacted!

Trust is earned in drops and lost in buckets; SpaceX/Grok will likely learn this. By secretly uploading codebases and secrets, Grok CLI revealed itself as an untrustworthy coding agent that’s a risk to use. Meanwhile, all the major competitors – Claude Code, Codex, OpenCode, Gemini CLI – have never violated user trust this way.

Grok CLI can rebuild trust, but it’ll likely take years of no security-related incidents to prove they are an open source-first product (that they were not, just a day ago!), and also demonstrate that they care about “normal” developers, and not just enterprise clients with ZDR turned on.

This incident is a good reminder of why planning and process can slow shipping speed, but increase revenue generation. I would wager that the Grok team has shrunk SpaceX’s enterprise subscription prospects for the foreseeable, in the name of saving a few hours on security reviews, learning what other coding harnesses do with codebase uploads, or even just asking the Cursor team!

I predict Grok/SpaceX will have to offer very high usage limits inside the Grok CLI to convince devs to take a risk on running this software on their system. And they will have to undercut OpenAI and Anthropic API pricing massively for any security team to greenlight use of a CLI that just last week was sending .env secrets unencrypted to their GCP buckets.

Of course, SpaceX/Grok will be just fine as it has the capital to fix things. It will now just be a lot more expensive and time-consuming to fix something that was likely caused by a few engineers wanting to make debugging easier!

Read the full The Pulse issue, or check out this week’s The Pulse. The full issue additionally covers:

1. New trend: concern about massive increase in code review load. Top of mind for engineering leaders: what to do about the ever-growing code review load, and how devs are starting to review code less thoroughly than before? Many questions, but few proven solutions. Send comments about what you see working.
2. Are more devs at enterprises upset about enterprise pricing by AI labs – and does it matter? I got a message from a reader baffled to learn their company pays 20-30x the price for tokens than their own $20/month Claude Code / Codex subscription. It may show how valuable AI coding tools are.
3. Linux creator: AI “clearly useful.” Inside the Linux kernel maintainers group, the discussion veered onto whether Linux should consider banning AI contributions, similar to how some FOSS projects have done so. Linus Torvalds weighed in and made it clear that AI is useful, everyone should decide whether to use it, but no one is allowed to tell others what tools to use. Given AI is an increasingly capable tool, it would be foolish to not use it as such.

Read the full The Pulse

Bending Spoons has announced buying Airtable for $1.285B in cash this week - which is less than the $1.4B in total funding Airtable has raised in the past, and well below the $11B valuation it had during its last fundraise in December 2021.

Selling to Bending Spoons is a company admitting defeat, and its inability or unwillingness to turn its business around, and wanting to get the highest possible cash for the business. Because this is what Bending Spoons is excellent at: they pay the highest cash value for a struggling business with a well-known brand, then take over operations, and operate the product with a fraction of the staff. They often let go most or all of the original team as they move over product operations to their in-house engineering team based in Italy and Europe.

Evernote: what happens when a new team takes over a legacy application

And having talked with Bending Spoons' engineering team on the podcast: they have done impressive engineering work after a takeover, in the past! For example, upon acquiring Evernote, the Bending Spoons engineering team discovered that the note-taking service was running as a Java 11 monolith (!!), with user data sharded across 750 manually provisioned virtual machines (!!!) on top of Google Cloud - in 2023! At a time when running cloud-native setups (managed databases, microservices) was common knowledge for years.

Evernote's existing setup was weirdly inefficient and operationally very heavy, with manual interventions needed to keep the service running. Needless to say, performance was poor because some VMs were regularly overloaded. Also, oncall was brutal!

The Bending Spoons engineering team rationalized the architecture:

• Migrated user data sharded from the 750 manually provisioned VMs to a managed database
• Split up the Java 11 monolith to microservices
• Did all of the above without disrupting user experience
• Improved performance of the backend by a wide margin
• Reduced oncall load after finishing the migration to a cloud-native setup vs the previous manual provisioning setup
• Did all the above in about 6 months.

It's a fair question: would long would have the original Evernote engineering team have taken to do the same changes that made the service more reliable, more performant, and cheaper to operate? I would guess it would have taken them many years: in fact, if they did not make this change until 2023, who knows if they would have ever made these pretty rational changes? And so the "shock therapy" of Bending Spoons starting with a blank page, and a new team taking over operating the full product, with a laser focus on efficiency: well, this approach can be pretty efficient, as the Evernote example shows.

You can listen to the full podcast episode I did with the Bending Spoons team: Twisting the rules of building software: Bending Spoons.

Price increases and the existing team let go: the two most typical complaints

Bending Spoons taking over an existing product has two major criticisms:

1. Price increases. Evernote was the biggest example of price hikes: after Bending Spoons took over operating the product - and improving its performance - price hikes followed. Being Spoons kept investing in Evernote, adding new features (including AI ones), but customers paying $37/year for the Pro plan pre-2023 were charged $250/year by 2026. My take is that this is what happens when a product starts working a business maximizing profits: lots of customers will leave for competition, while others will pay more, valuing a more reliable service that gets more investment than before. Bending Spoons keeps improving Evernote since the acquisition, alongside the price increases. Clearly, the company is optimizing for maximizing revenue, not maximizing the number of customers, though.
2. Layoffs. Bending Spoons let go most/all of the Evernote team in the US, briging operations in-house. This is part of the "usual" playbook of Bending Spoons: they buy products to operate them as efficiently as possible. The re-architecting example shows benefits of starting from scratch, and not needing to deal with internal resistance for changes that result in more efficient operations. Knowing that with a Bending Spoons acquisition, letting go of all the existing team is on the table is something that comes with selling to this company.

With this, let me share my analysis of a past Bending Spoons acquisition: when they bought SteamYard from Hopin.

Below is the now un-paywalled excerpt from The Pulse #89: The end of Hopin, from April 2024, sent to paid The Pragmatic Engineer subscribers. If you'd like to get analysis like this in your inbox, weekly, subscribe to The Pragmatic Engineer.

The End of Hopin

It’s been a real rollercoaster ride for the virtual events provider:

• 2019: founded with a mission to provide a solution for hosting virtual events.
• 2020-2021: raised a total of $1B in funding during a seed round in Feb 2020, Series A in June, Series B in November, and then a Series C in March-June 2021. The company was valued at $7.75B and acquired several startups, the biggest of which was video streaming platform, StreamYard, for $250M.
• 2022: layoffs in February, when Hopin was one of the early scaleups to do large cuts (12%), followed by more in July (29%), and November (17%)
• 2023: Hopin sold its core event tech business to RingCentral for $50M. We analyzed this at the time.
• 2024: Last month, Hopin’s UK entity entered liquidation. Insiders told me it was merely a restructure, with Hopin UK employees joining StreamYard. Basically, Hopin became the business it had purchased back in 2021.

This week, Italian mobile app developer Bending Spoons acquired the remains of Hopin, which is basically the StreamYard product. All Hopin staff will soon be laid off.

The Bending Spoons acquisitions strategy

Bending Spoons has previously acquired apps such as the notes app Evernote in 2022, events app Meetup in 2024, and video-recording app FiLMiC in 2022. Their approach to these acquisitions was the same each time:

1. Take over operating the product
2. Fire most staff immediately
3. Have some remaining staff hand over services, then fire them as well
4. Operate the app with a much smaller team and raise prices.
5. Profit!

I talked with current Hopin employees for details on what will happen next, and if this model will be followed again. Unfortunately, it will.

All existing Hopin staff will be let go, eventually. This affects around 80 staff working on StreamYard, and another 70 on other Hopin products, Streamable (video sharing) and Superwave (community platform.) I’m told severance packages are generous enough, at around 3-4 months’ salary.

As with other Bending Spoon acquisitions, a subset of the team was requested by Bending Spoons to help with the transition (and then be let go afterwards.) Understandably, morale is very low for this reason, and the certainty that everyone will lose their jobs.

How much did StreamYard sell for?

From talking with current employees, I gather that circa 95% of Hopin’s revenue comes from StreamYard, and not more than 5% from Streamable and Superwave. So the only valuable asset that this acquisition priced in is StreamYard.

In 2021, Hopin paid $250M for it. Back then, the video streaming service generated about $40M in annual revenue. This has risen to about $70M per year and keeps growing in an increasingly crowded market. StreamYard was at around break even and can be easily made profitable, I’m told.

A good question is whether Bending Spoons paid $250M or more for this asset. In 2023, RingCentral paid $50M for the “core” virtual events offering which was making $20M in annual recurring revenue (ARR) at the time, I’ve confirmed with insiders. However, ARR was falling steeply, and was forecast to hit $10-15M within a year. So RingCentral paid a 2.5x multiple for an asset losing revenue.

StreamYard brings in $70M per year, and this is increasing. I’d assume the purchase price would be at least the same 2.5x multiplier, if not more. So there’s a fair chance this sale’s value is close to $200M.

Why did Hopin sell to a buyer which wants to lay off everyone?

I have exclusively learned that StreamYard’s founders actually offered to Hopin’s board of directors to buy the company back, and operate independently, as before. This would’ve been a better outcome for employees, most of whom would surely have kept their jobs. Some of StreamYard’s staff knew of this plan and naturally supported it. The Bending Spoons sale has taken everyone by surprise.

But why would Hopin choose a buyer that is guaranteed to sack existing staff? Well, the board might have had no real choice, due to Hopin having raised too much money.

Hopin raised $1B in funding, during which it almost certainly offered board seats to investors including a16z, General Catalyst, Coatue, Northzone, Salesforce Ventures, Tiger Global, Accel, and others. It’s safe to assume investors control the board, and as Hopin will never live up to its $7.75B valuation, the board-level rationale has evidently been to maximize the amount of money clawed back.

Of that $1B, here’s what’s left:

• $50M from selling Hopin’s core business
• Whatever StreamYard sells for
• Residual cash left over from the fundraising

The board serving investors’ interests had to shop around for the highest bidder, and minimize losses. I have to assume the decision on whether StreamYard’s founders could buy back their own company came down to whether or not someone else was offering more money for it. Unfortunately for Hopin’s staff (and fortunately for investors,) Bending Spoons probably offered more.

The risk of raising too much venture capital

Hopin is a reminder that raising too much venture capital can have unexpected, seemingly irrational, outcomes. Firing all staff from a company making $70M/year while being break-even or profitable sounds irrational from the company’s perspective. But it is rational for investors and a buyer:

• Hopin’s investors realized the company is a “failed bet.” They want to cash out their losses: get back whatever money they can – which is still in the hundreds of millions of dollars! – and use this capital to make new bets.
• Hopin’s buyer – Bending Spoons – wants to maximize their return. They pay $X for the company, and the goal is to generate $Y over the next several years in profit from it, where $Y > $X. So, the acquisition pays for itself. Bending Spoons has a working model that involves firing all existing staff, and operating the product more efficiently.

The biggest losers in this story are:

• Some investors. Collectively, investors poured $1B into Hopin. In October 2023, Hopin returned $581M of capital to investors (so 58% of all amount raised). It is unclear if the StreamYard purchase that could be another $200-300M, will be returned to them. It is safe to assume that investors will lose about 20-42% of the amount they invested, depending on how much proceedings of the StreamYard purchase they get paid. This is much better than in the case of one-click checkout startup Fast going bankrupt a year after raising $100M in funding, where investors most likely lost all their investment! In the case of Hopin: it’s still a loss, but it’s far from a 100% loss like with Fast.
• Employees who expected a better outcome. Shares issued to staff by Hopin are now officially worthless. At the same time, Hopin did pay above-the-market base salaries, and offered generous severance during redundancies. Unfortunately, a reality of fast-growing startups is that they can grow fast, but also go down fast.

Winners of this sale are:

• The original founders of StreamYard who sold the company for $250M cash. Even though these founders are also departing, they netted a healthy return in 2021.
• Bending Spoons, which has acquired a market-leading streaming product generating $70M per year and growing. StreamYard would normally not be available to buy, but the need of the Hopin board to “cash in” the company’s remaining assets made this sale possible.

I assume the biggest winner of the Hopin story stands to be Hopin’s founder and former CEO, Johnny Boufarhat. He sold more than £100M ($127M) of his shares in 2021 as secondaries. He probably netted more money than Hopin – excluding StreamYard – generated in its lifetime! Selling a good chunk of his shares in 2021, at the peak of hype for virtual events is a good reminder that when everyone is buying, it can be a profitable strategy to sell!

What happened to other fast-growing startups in Europe?

In 2020, Hopin was known as the fastest-ever growing startup in Europe by valuation. This visualization by Sifted went viral, and was widely shared by Hopin staff on social media:

Hopin’s current value is now zero, having sold its valuable assets. But how have other, formerly fastest-growing startups in Europe performed? I visualized this:

Excluding Hopin, the car sale website Cazoo did worst; it’s currently close to bankruptcy, valued at about $60M. The companies that managed to grow above than their 2020 valuations are:

• Food delivery service Wolt was acquired by DoorDash for €7B ($8.1B) in 2022
• Ride-hailing app Bolt was last valued at $8.5B, and is supposedly preparing for an IPO in 2025
• Neobank, Revolut, was valued at $33B, even though some investors cut their valuation of the company to around $20B in the summer of 2023
• Spotify’s current market cap is nearly $60B, and the company is trading close to its 2021 all-time-high

This chart confirms what we already know: 2020-2022 was a time when startup and scaleup valuations hit all-time highs, fueled by zero interest rates, and widespread changes in consumer spending caused by the Covid-19 pandemic. We have covered what the end of rock bottom rates could mean for the tech industry.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics of last week's The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

You can no longer watch The Pragmatic Engineer Podcast as video in the Spotify app (only as audio) because I have quit publishing video on that streaming platform. This comes after I decided that reliability takes a back seat within that team – and across much of Spotify. Unlike on other platforms such as YouTube, Apple Podcasts, and Substack, I’ve recently encountered a series of reliability issues around Spotify being unable to process video episodes. Even though I enjoyed a direct link with the Podcasts team there, things haven’t improved.

So from now, I will no longer be publishing video episodes on Spotify. You can find videos of my in-depth chats with guests only on YouTube. Apologies for any inconvenience this change causes! Audio episodes of the podcast can still be found on Spotify via the RSS podcast feed hosted on Substack.

Honestly, the decision to quit the streaming giant wasn’t hard, and I reckon there’s a point here about the risk of deprioritizing reliable operations at major companies in order to push on things like AI adoption, as Spotify seems to be doing.

Some context: for the first two years of The Pragmatic Engineer Podcast, it was published on three podcast platforms:

1. Substack’s podcast platform (audio): this is where the “master” RSS feed is served to the likes of Apple Podcasts, the web, Overcast, Pocket Casts, etc
2. YouTube (video): video episodes uploaded individually
3. Spotify (video + audio): every video episode was uploaded individually and then served as video or audio episodes from the platform.

As someone hosting a podcast, there are good reasons to bother doing three separate uploads:

• Most podcast platforms don’t support video. There will always be a need for a platform that serves the master RSS feed for audio versions while the video ones are elsewhere.
• YouTube doesn’t integrate with anything. YouTube is the leader in video podcast distribution, and uploading there directly makes sense.
• I had a direct line to the Spotify team, which was a big plus. Starting out the podcast, I had the unusual privilege of contact with the podcasts team, thanks to the newsletter gaining a decently-size audience. I was persuaded to take the plunge with them.

For eighteen months, nothing major went wrong. The admin portal for podcast publishers (called ‘Spotify Creators’) was pretty wonky; it gave intermittent errors, and was unable to remember me when I signed in, so, each Wednesday, I’d have to sign in with a code sent to my email to publish an episode.

But overall, things worked, until it all went suddenly downhill…

Unable to publish Spotify podcast episodes 3 out of 5 weeks

From late May, I did not include links to Spotify on new episode announcements because their podcasts product or platform seemingly had outages every time one published on Wednesdays at around 9am PST / 12pm EST / 6pm EU time.

Outage #1 (20 May): podcast publishing broke, my episode would not process on Spotify for 2+ hours. When uploading a video file to Spotify, there’s a processing pipeline that runs to create chunks of the podcast in different video and audio formats. This pipeline appeared to stop running, meaning new episodes were not published.

It was not just the publishing that broke: the Creator portal looked absurd, with NaN% values everywhere, during the outage:

I emailed the Spotify team to alert them about the outage and also complained online. I got a response, confirming the outage and pledging to do better:

“The issue was in one of our podcast publishing metadata pipelines. A small subset of episodes completed normal media processing but then missed a downstream publish update because a newly introduced validation signal was not correctly wired into the logic that wakes up the publishing path. In simpler terms: the episode could become eligible to publish, but the final propagation step was not reliably triggered for that class of episodes.

We identified the root cause, deployed a fix, and reprocessed the affected episodes with all-clear called early this morning. We’re also tightening the system so that fields used for publishing eligibility cannot be added without also triggering the relevant downstream updates.

Separately, we’re reviewing how partial creator-impacting publishing delays are surfaced, because even when this is not a broad platform outage, it is still a bad experience for publishers like yourself.

Apologies again that you hit this. It was a real bug, not a wide outage, but it hit some of our most relevant creators.”

Outage #2 (17 June): Spotify down. Four weeks later, when attempting to publish a video episode, all of Spotify went down for many users, including myself.

Spotify does not maintain a status page, so it’s impossible to tell how widespread the outage was. I didn’t include a Spotify link in that week’s announcement either.

Outage #3 (24 June): podcast publishing broke – again. Outage #3 in five weeks; deja vu. This time, it was episode publishing not working, yet again. After waiting two hours for the episode to publish on Spotify, I yet again sent out the announcement with no Spotify link.

I also emailed the Spotify Podcasts team, who confirmed the outage. I said I was considering stopping publishing video episodes, and to switch to audio-only publishing (which means pointing Spotify to my master RSS feed.) I said that an apology was appreciated but it wasn’t enough to make it worth publishing video episodes there.

I also asked for the incident review because I had the feeling that reliability was not all that important on this podcast product. For the first outage I got a vague description of what happened, and promises of improvements that were never done – e.g. during this second outage, there was no improved communications to creators, which I was told would happen, after outage #1.

Internally, Spotify’s team surely conducted an incident review as per usual, so I figured I’d hear back in about two weeks’ time, and assumed a reply would be forthcoming because I’d made clear I was ready to leave Spotify Podcasts if reliability didn’t improve.

No incident review three weeks later, so I quit Spotify

The incident review had never arrived as promised by three weeks later, even though there had been time for it to be completed. It was yet another sign of a platform that has become unreliable. Also, the creator portal occasionally threw up this error:

I checked my Spotify stats: stream plays had been trending downwards unsurprisingly, given the ongoing outages, while the other podcast platforms didn’t show the decline. It made me decide “enough is enough” and to move off Spotify.

Staying on their platform depended on seeing an incident review, but they didn’t prioritize transparency, still had no status page, and nobody had built a feature for episode-processing status like YouTube has had for years. So, I pulled the plug and left:

After I made the switch away from Spotify, the platform’s creators portal became buggier than ever, as in these examples:

Comments disappeared:

… even though other parts of the UI showed dozens of comments:

Episode links directed to 404 pages:

A day or two later, these issues disappeared: I assume no one had tested the flow of moving away from Spotify Podcasts to an RSS feed, and it’s why the experience was so poor.

Incident review finally published, but with a wrong timeline

A few days after offboarding from Spotify, their team published the incident report for outage #3. Reading through it, something did not add up in the timeline:

My email account confirmed that I mailed the Spotify team at around 17:30 about the outage. So, after weeks of creating this report, why did the incident report downplay the fact that customers alerted the team before their own automated alerts fired?I complained to the Podcasts team, and to their credit, the incident report was updated:

I didn’t like how high-level the report is, and how vague the promised improvements were. Specifically, this one:

“During this incident, many creators learned something was wrong from their audiences before they heard anything from us. We are improving our processes and technical capabilities so creators get notified as soon as possible when things aren’t working.”

Overall, I don’t regret the choice to leave, particularly when the focus of Spotify’s leadership is on AI, not reliability.

Does Spotify have “AI psychosis?”

Previously, I used the term “AI psychosis” differently from the usual way of describing when someone starts believing everything an AI model tells them, however outlandish. I applied it to Meta’s rush to develop its own AI model at the cost of the reliability of its profitable business activities. This was based on Instagram’s most embarrassing-ever account takeover incident, which occurred when the team responsible for Instagram’s Trust & Safety was slashed. Soon after, AI-generated, AI-reviewed code caused the hacking of a former US president’s account.

At Spotify, it should have gone the other way. In March, I had the opportunity to meet its Head of Technology & Platforms, Tyson Singer, who said the company puts reliability far ahead of AI adoption, and doesn’t adopt AI for its own sake. So, it was somewhat surprising to read the summary below of a podcast Spotify did with Anthropic:

“Spotify now ships 4,500 production deploys a day, and 73% of PRs are now AI-assisted.

Niklas Gustavsson (VP of Engineering at Spotify) keeps 5 to 10 Claude sessions running in tmux, one per git worktree, agents working in the background. All of it inside a 20M+ line monorepo. He expected agents to struggle at that size, but it’s worked well.

Spotify’s migration codemods grew into thousands of lines of edge cases. Code has too much API surface for static rewrites. Early LLMs barely did better. Adding a judge took PR success from ~25% to 80%.

All of this leans on verification, the single most important thing when agents are used and the place most companies underinvest

Spotify rebuilt their test automation around it so engineers can confidently guide and supervise agents, rather than manually execute repetitive tasks.”

It seems to me that all the talk is about usage of AI, and none about reliability, all while Spotify’s platform becomes less reliable than ever, at the same time as the streamer is going all-in on AI; with AI judges and devs running 5-10 parallel Claude sessions.

All things considered, it’s worth asking if Spotify has the corporate variant of “AI psychosis”, whereby the reliability of a successful operation gets torched in the chase for the next big thing by executives. I don’t even think Spotify is all that different from Meta and other companies in this!

Things look bad, based on the quality and reliability degradation of products. Annoyingly, in many cases, customers don’t really have the choice of going elsewhere. My podcast is an exception, as video podcasts on Spotify never truly took off, so quitting the platform wasn’t a big deal. Even so, I’m particularly disappointed that Spotify has prioritized AI usage over reliability. I know some executives there pushed against this, but I feel safe in assuming that they lost that battle.

Value of staying reliable & “sucking less”

Max Kanat-Alexander, distinguished engineer at Capital One, has written about how a software project can become wildly successful just by “sucking less” in his reflections upon the success of the Bugzilla project, (2004-2009):

“All you have to do to succeed in software is to consistently suck less with every release.

Nobody would say that Bugzilla 2.18 was awesome, but everybody would say that it sucked less than Bugzilla 2.16 did. Bugzilla 2.20 wasn’t perfect, but without a doubt, it sucked less than Bugzilla 2.18. And then Bugzilla 3.0 fixed a whole lot of sucking in Bugzilla, and it got a whole lot more downloads.

Why is it that this worked?

As long as you consistently suck less with every release, you will retain most of your users. You’re fixing the things that bother them, so there’s no reason for them to switch away. Even if you didn’t fix everything in this release, if you sucked less, your users will have faith that eventually, the things that bother them will be fixed. New users will find your software, and they’ll stick with it too. And in this way, your user count will increase steadily over time.

But what happens if you release frequently, but instead of fixing the things in your software that suck, you just add new features that don’t fix the sucking? Well, eventually the patience of the individual user is going to run out. They’re not going to wait forever for your software to stop sucking.”

Personally, I got tired of Spotify’s Podcasts product continually going in the wrong direction on Max’s scale: the poor reliability, frequent errors on the Creators site, and the sense that they don’t really care about improving existing things.

Read the full issue of last week's The Pulse. The full The Pulse additionally covers:

1. Will Kimi K3 trigger US push for closed-source AI models? Moonshot AI’s latest open model, Kimi K3, is on par with Anthropic’s Fable 5. Could it lead to the US government regulating or banning Chinese open models to protect US labs?
2. AWS laughs off “heart attack” billing error. AWS customers were billed trillions more than they should have been, due to what was likely a conversion error. But instead of sharing an incident report, AWS saw the funny side.
3. Industry pulse. OpenAI’s unreleased model tried to hack HuggingFace to improve its test scores, X took more than a year to develop its new Android app, Google’s new AI model flops, and more.

Read the full The Pulse.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics of last week's The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

One thing I am hearing that’s top of mind for many engineering leaders is what is being done to deal with the continuous increase in code review load. It’s been a topic for a while, and more such conversations seem to be taking place.

For me, it began in January, when Opus 4.5 and GPT 5.4 started to write more and better code at most companies. Around then, Director-level folks started talking about the bottleneck of building software moving from coding to the review phase.

There’s been a boom in AI code review tools to deal with the increase in load since February, and an explosion of experimentation with and adoption of dedicated AI code review tools like CodeRabbit, Greptile, Qodo, SonarQube (now also Gitar). There’s also tools offered by coding harnesses themselves like Claude Code review, Cursor review, GitHub Copilot review. And then tools previously not involved in code reviews – but which have context on the codebase – are also adding this, like Sentry’s Seer AI reviews, Linear code reviews.

Larger companies are building in-house tools to improve the code review experience. Uber’s Code Inbox is one case:

Smart assignments are a feature inside Code Inbox for having reviews progress:

Then there’s Risk Profiles which estimate the impact of a change, and encourage devs to pay extra attention to risky ones:

We covered how Uber uses AI for software development, and it’s not just Uber: companies like Cloudflare (AI Code Reviewer), Faire (Fairey), and HubSpot (Sidekick) and many others have also built tools to make their code review flows more fluid, after finding that an in-house implementation worked better than integrating a vendor.

Another approach is thinking about how to verify code, instead of reviewing. This is easier said than done; in theory, thorough testing should be able to verify that code works as expected. But how much testing is ‘thorough’? What type of tests are we talking about? Integration and end-to-end as well? What about fuzz testing? Or formal methods? What about verifying that new tests exercise the functionality as expected? And how do we connect all of this with observability?

Too much thorough code review is burning out engineers, and resulting in sub-par code reviews. I hear a lot anecdotally that devs see others as no longer able to review code with intent, whereby, if the AI code review has no real comments, they just approve it. Meanwhile, those devs who put the same effort and energy into code review as before feel overloaded by AI slop PRs sent their way.

The problems exist, and the solutions feel more like experiments.

What are you seeing inside your company, and how are you dealing with the increase in code reviews? Share your ideas for practical, workable “replacements”?

Read the full issue of The Pulse this excerpt is from, or check out the latest The Pulse from today. Today’s issue covers:

1. Moving video podcasts off Spotify due to constant reliability issues
2. “Kimi K3” moment & a US lobby for closed-source AI models
3. AWS laughs off huge billing error
4. Industry Pulse

Read the full issue here

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s The Pulse issue. Full subscribers received the article below eight days ago. If you’ve been forwarded this email, you can subscribe here.

We’re used to highly reliable systems which target four-nines of availability (99.99%, meaning about 52 minutes of downtime per year), and for it to be embarrassing to barely hit three nines (around 9 hours of downtime per year.) And yet, in the past month, GitHub’s reliability is down to one nine!

Here’s data from the third-party, “missing GitHub status page”, which was built after GitHub stopped updating its own status page due to terrible availability. Recently, things have looked poor:

This means that for every 30 days, GitHub had issues on 3 days, or issues/degradations for 2.5 hours daily (around 10% of the time.)

GitHub seems unable to keep up with the massive increase in infra load from agents. One software engineer built a clever website called “Claude’s Code” that tracks Claude Code bot contributions across GitHub. Growth in the past three months has been enormous:

Stream of GitHub outages from infra overload

GitHub’s CTO, Vladimir Fedorov, addressed availability issues in a blog post and covered three major incidents:

• 2 February: security policies unintentionally blocked access to virtual machine metadata
• 9 February: a database cluster got overloaded
• 5 March: writes failed on a Redis cluster

Software engineer Lori Hochstein did a helpful analysis of these outages and the CTO’s response, and has interesting observations:

• Saturation: the database cluster incident (9 Feb) was a case of the database getting saturated, due to higher-than-expected usage. Databases are harder to scale up than stateless services. GitHub also underestimated how much additional traffic there would be.
• Failover + telemetry gap: the 2 Feb incident was a combination of an infra issue in one region failing over to a healthy region, and making things worse with a telemetry gap (incorrect security policies were applied in the new regions which blocked access to VM metadata)
• Failover + configuration issue: the 5 March incident was uncannily similar: after a failover, a configuration issue blocked writes on a Redis cluster

It is certainly nice to get details from GitHub on these outages. It feels to me that infra strains are causing more infra issues → they trigger constraints faster → failovers are not as smooth as they should be. Could it be because GitHub keeps changing their existing systems?

Startup shows GitHub how it’s done

While GitHub struggles to keep up with the increase in load from AI agents generating more code and pull requests, a new startup called Pierre Computer claims to have built an “AI-native” solution for AI agents pushing code, which scales far beyond what GitHub can do. Pierre was founded by Jacob Thornton: formerly an engineer at Coinbase, Medium, and Twitter, and also the creator of the once-very popular Bootstrap CSS library.

Here’s what Pierre supports, which GitHub does not:

“In October [2025], Github shared they were averaging ~230 new repos per minute.

Last week we [at Pierre Computer] hit a sustained peak of > 15,000 repos per minute for 3 hours.

And in the last 30 days customers have created > 9M repos”

These are incredible numbers – if also self-reported – and something that GitHub clearly cannot get close to, at least not today! There are few details about customers, while the product – called Code.storage – seems to be in closed beta.

Still, this is the type of “git for AI agents” that GitHub has failed to build, and the type of infrastructure it needs badly.

Has GitHub lost focus and purpose?

GitHub’s reliability issues are acute enough that, if it keeps up, teams will start giving alternatives like small startups such as Pierre a try, or perhaps even consider self-hosting Git. But how did the largest Git host in the world neglect its customers, and fail to prepare its infra for an increase in code commits and pull requests?

Mitchell Hashimoto, founder of Ghostty, and a heavy user of GitHub himself, had advice on what he would do if he was in charge of GitHub, after growing frustrated with the state of its core offering. He writes (emphasis mine)

“Here’s what I’d do if I was in charge of GitHub, in order:

1. Establish a North Star plan around being critical infrastructure for agentic code lifecycles and determine a set of ways to measure that.

2. Fire everyone who works on or advocates for Copilot and shut it down. It’s not about the people, I’m sure there’s many talented people; you’re just working at the wrong company.

3. Buy Pierre and launch agentic repo hosting as the first agentic product. Repos would be separate from the legacy web product to start, since they’re likely burdened with legacy cross product interactions.

4. Re-evaluate all product lines and initiatives against the new North Star. I suspect 50% get cut (to make room for different ones).

The big idea is all agentic interactions should critically rely on GitHub APIs. Code review should be agentic but the labs should be building that into GH (not bolted in through GHA like today, real first class platform primitives). GH should absolutely launch an agent chat primitive, agent mailboxes are obviously good. GH should be a platform and not an agent itself.

This is going to be very obviously lacking since I only have external ideas to work off of and have no idea how GitHub internals are working, what their KPIs are or what North Star they define, etc.

But, with imperfect information, this is what I’d do.”

My sense is that GitHub has three concurrent problems:

• GitHub and Copilot are entangled with Microsoft’s internal politics. GitHub’s Copilot in 2021 was the first massively successful “AI product.” Microsoft took the “Copilot” brand and used it across all of their product lines, creating low-quality AI integrations. Simultaneously, internal Microsoft orgs like Azure and Microsoft AI were trying to get their hands on GitHub, which is one of the most positive developer brands at Microsoft.
• GitHub has no leader, seemingly by design. GitHub’s last CEO was Thomas Dohmke, who stepped down voluntarily, and Microsoft never backfilled the CEO role; instead carrying out a reorg to make GitHub part of Microsoft’s AI group and stripping its independence. It seems the “Microsoft AI” side won that battle.
• GitHub has no focus, and is stuck chasing Copilot as a revenue source. GitHub has no CEO and is caught up in internal politics, so, what can GitHub teams do? The safest bet is to increase revenue and the best way to do that is by investing more into GitHub Copilot, and ignoring long-term issues like reliability.

I agree with Mitchell: GitHub has no “North Star” and we see a large org being dysfunctional. That lack of vision – and CEO – is hitting hard:

• GitHub Copilot went from the most-used AI agent in 2021, to be overtaken by Claude Code, and is soon to be overtaken by Cursor.
• As a platform, GitHub has no vision for how to evolve to support AI agents. Sure, GitHub has an MCP server, but it has no “AI-native git platform” that can handle the massive load AI agents generate.
• GitHub keeps shipping small features and improvements without direction. For example, in October 2025, they started to work on stacked diffs. However, when it ships, the stacked diffs workflow might be mostly obsolete – at least with AI agents!

It’s easy to win a market when you do one thing better than anyone else in the world. Right now, GitHub is doing too many things and doing a subpar job with Copilot, its platform, and AI infra.

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse.

Catch up with recent The Pragmatic Engineer issues:

• Scaling Uber with Thuan Pham (Uber’s first CTO — podcast). We went into topics like scaling Uber from constant outages to global infrastructure, the shift to microservices and platform teams, and how AI is reshaping engineering.
• Building WhatsApp with Jean Lee (podcast): Jean Lee, engineer #19 at WhatsApp, on scaling the app with a tiny team, the Facebook acquisition, and what it reveals about the future of engineering.
• What will the Staff Engineer role look like in 2027 and beyond? What happens to the Staff engineer role when agents write more code? Actually, they could be more in demand than ever!

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Inside Meta, an engineer created a “token leaderboard” that ranks employees by token usage. Last week, The Information reported:

“Employees at Meta Platforms who want to show off their AI superuser chops are competing on an internal leaderboard for status as a “Session Immortal”— or, even better, “Token Legend.”

The rankings, set up by a Meta employee on its intranet using company data, measure how many tokens — the units of data processed by AI models — employees are burning through. Dubbed “Claudeonomics” after the flagship product of AI startup Anthropic, the leaderboard aggregates AI usage from more than 85,000 Meta employees, listing the top 250 power users.

The practice is emblematic of Silicon Valley’s newest form of conspicuous consumption, known as “tokenmaxxing,” which has turned token usage into a benchmark for productivity and a competitive measure of who is most AI native. Workers are maximizing their prompts, coding sessions and the number of agents working in parallel to climb internal rankings at Meta and other companies and demonstrate their value as AI automates functions such as coding.”

I spoke with a few engineers at Meta about what’s happening, and this is what they said:

• Massive waste. Plenty of devs are running an OpenClaw-like internal agent that burns massive amounts of tokens for little to no outcome.
• Outages caused by AI overuse. A dev mentioned that some SEVs were caused by what looked like careless AI code generation; almost like a dev behind the SEV was more concerned with churning out massive amounts of code with AI than with product quality.
• Gamified leaderboard. Those at the top of the leaderboard produce throwaway, wasteful work. This is painfully clear to anyone who checks Trajectories (AI prompts), which can be viewed.

As per The Information, Meta employees used a total of 60.2 trillion AI tokens (!!) in 30 days. If this was charged at Anthropic’s API prices, it would cost $900M. Of course, Meta is likely purchasing tokens at a discount, but that could still come in at $100M+ – in large part from senseless “tokenmaxxing”.

After backlash on social media, Meta abolished the internal leaderboard last week. One day after The Information revealed details about the incredible tokenmaxxing numbers, I confirmed that Meta has taken down its leaderboard; perhaps they realized that the incentive created enormous and unnecessary waste. If so, it’s a bit surprising that it took media coverage for the social media giant to reach that conclusion.

One engineer at Meta told me they think Meta had a different goal with the token leaderboard. A long-tenured engineer suspects increasing AI usage actually was the real goal. They said:

“Putting a leaderboard in place was always going to incentivize much more AI usage. And more AI usage means producing a lot more real-world traces. These traces can then be used to train Meta’s next-generation coding model better.

I believe this was the goal, even if no one said it out loud.

It’s an expensive way to generate data for training, but if any company has the means to do so, it’s Meta.”

Microsoft: full-force tokenmaxxing

Similarly, Microsoft has had an internal token leaderboard like Meta’s since January, and it started pretty well, as I reported back at the time: there’s an internal token dashboard that displays the individuals who use the most tokens in order to promote the use of tokens and experimentation with LLMs. At the Windows maker, this leaderboard is interesting:

• Very senior engineers – distinguished-level folks – are in the top 5 across the whole company, despite the fact that this group generally wrote little code in the past.
• VP-level folks make the top 10 and top 20, despite often being in meetings for most of the day and rarely writing code.

However, what starts as a metric for performance reviews or promotions can quickly become a target for devs. I talked with a software engineer at the Windows maker who admitted they’re full-on “tokenmaxxing” – not to get on the leaderboard, but rather because they don’t want to be seen as using too few tokens:

“We have internal dashboards and metrics tracking AI usage, token usage, percentage of code written by AI vs hand-written code.

I am conscious of not wanting to be seen as “uses too little AI,” and I’m not ashamed to say I need to do tokenmaxxing to do this. Things I do to inflate my token usage metrics:Ask AI questions about the code already in the documentation. The AI pulls up the documentation, processes it, and gives me results 10x slower, but while burning lots of tokens. I could use “readthedocs” [an internal product], but then my token numbers would be lowerAsk the AI to prototype a feature that I have no intention of working on. Prompt it a few more times, then throw the whole thing awayDefault to always using the agent, even when I know I could do the work by hand much faster. Then watch it fail”

This engineer is relatively new at the company, so is concerned about job security, and is playing this game to avoid being tagged as insufficiently “AI-native” by burning far more tokens than necessary.

Salesforce: burning tokens to hit “minimum” & “ideal” targets

Elsewhere, Salesforce has created “tokenmaxxing” incentives, as well. Talking with an engineer there, I learned that the company built two tools that effectively incentivize excessive spending on tokens:

1. “Minimum” incentives with a tracking tool. There’s a Mac widget that shows your own spend, updated every 15 minutes. It also displays minimum expected spend. Last week, the target was $100 on Claude Code, and $70 on Cursor.
2. Showing everyone’s spend. A web-based tool to see the token spend of any colleague. It’s used to check where team mates’ usage is at.
3. “Maximum” spend limits that can be exceeded. Up to a week ago, there was also a maximum monthly limit of $250 for Claude Code and $170 for Cursor. However, this can be exceeded with the simple press of a button if the limit is reached. I’ve learned that last week, some engineering organisations at Salesforce had their “maximum” limit removed in order to “remove any friction from the development process.”

The message Salesforce sends to staff is clear: “use a minimum of $170/month tokens or be flagged.” Who wants to get flagged for using too few tokens? The outcome is somewhat wasteful token spend:

• Burning tokens for nothing. Devs ask Claude or Cursor: “build me X,” where X is a project or product with nothing to do with their work, and not something they’d ever ship. It’s just a way to burn tokens
• Calibrating token spend to be above average. Plenty of devs browse peers’ token spend to figure out the slightly-above average point, then use the tokens needed to hit that mark

Shopify: an example on how to avoid tokenmaxxing

The first-ever token leaderboard that I’m aware of was built by Shopify in 2025. And it worked well! Last June, the Head of Engineering at Shopify, Farhan Thawar, told me on The Pragmatic Engineer Podcast:

“We have a leaderboard where we actively celebrate the people who use the most tokens because we want to make sure they are [celebrated] if they’re doing great work with AI.

[And for the top people on the leaderboard,] I want to see why they spent say $1,000 a month in credits for Cursor. Maybe that’s because they’re building something great and they have an agent workforce underneath them!”

I asked Farhan for details on how it’s gone since. Here’s what he told me:

“We have since renamed the token leaderboard to usage dashboard: for obvious reasons, as we don’t want to encourage “competing” to make it to the top of this board. We have token spend on our internal wiki profile as well as on the usage dashboard.

We also have circuit breakers to catch “runaway agents.” So if personal spend spikes within a day, we can cut off access immediately, and you can renew if the usage spike was deliberate, or if it was a runaway agent. The circuit breaker worked well for us: we’ve not only caught runaway agents, but found bugs in our infra this way!”

Shopify’s approach seems to have worked for a few reasons:

• The usage dashboard served as a “push” for devs to use AI tools, early-on. Last year, devs were mostly experimenting with AI tools because they were not as performant as today. The usage dashboard encouraged developers to try new tools, and highlighted power users.
• Circuit breakers helped. Cutting off spend when usage spikes helped catch “runaway agents.”
• High usage is looked at. Farhan checks-in with top-spending individuals to understand the use cases. Any tokenmaxxing would likely have been spotted at this stage, which would have been a bit embarrassing for the user!

One more interesting learning Farhan shared with me: it’s more interesting to not look at “who spent the most in overall token cost?” but instead, “whose tokens cost the most?” Devs who generate tokens that come out as expensive have turned out to do in-depth work that was interesting to learn about!

Tokenmaxxing: great for AI vendors, bad for everyone else

I see very few rational reasons why incentivizing tokenmaxxing makes sense for any company. It results in increasing AI spend – by a lot! – in return for little to no value. Heck, in some cases it actually incentivises slower work – as shown by devs using the AI to answer questions when documentation is readily available – and encouraging ‘busywork’ where devs prompt projects that they don’t even want to ship. Tokenmaxxing seems to push devs to focus on stuff that makes no difference to a business.

It feels to me that a good part of the industry is using token count numbers similarly to how the lines-of-code-produced metric was used years ago. There was a time when the number of lines written daily or monthly was an important metric in programmer productivity, until it became clear that it’s a terrible thing to focus on. A lines-of-code metric can easily be gamed by writing boilerplate or throwaway code. Also, the best developers are not necessarily those who write the most code; they’re the ones who solve hard problems for the business quickly and reliably with – or without – code!

Similarly, the number of tokens a dev generates can easily be gamed, and if this metric is measured then devs will indeed game it. But doing so generates a massive accompanying AI bill!

—-

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. New trend: token spend breaks budgets – what next? In the past 2-3 months, spending on AI agents has exploded at many tech companies, and the ramifications of this are starting to dawn on engineering leaders. We’ve sourced details from 15 companies, including the different ways they are coping with this realization.
2. New trend: more AI vendors can’t keep up with demand. Related to massively increased spending, GitHub Copilot and Anthropic are starting to limit less-profitable individual users, so they can serve business users whose spend has easily 10x’d in the last few months. The exception is OpenAI and Codex.
3. Morale at Meta hits all-time low? Business is booming but devs at Meta are furious and worried due to looming layoffs, and an invasive tracking program rolled out to all US employees.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of three topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Last week, we covered the slightly perverse trend of “tokenmaxxing” across the industry, where devs run agents with the sole aim of boosting their personal “token stats” in an effort to rank higher on internal token leaderboards, and not be seen as a Luddite who doesn’t use AI tools enough compared to peers.

This week, I spoke with a software engineer at a large company and another at a seed-stage place. Both shared almost identical stories: at their latest all-hands, company leadership expressed concerns about the fast-rising costs of tokens. At both places, token spend has increased by ~10x in the last six months – with no signs of slowing down.

I wanted to find out about this trend, so I talked to devs at 15 businesses. Below is what I learned about what’s happening in workplaces of all sizes. Names are anonymized.

Large companies

Setting the default model to a cheaper one: 10,000+ person SaaS company, offices on all continents

Inside a large SaaS company, most devs use an internal background coding tool for coding. This model defaults to Claude Sonnet, which is the cheaper Claude version. Model selection is not persisted, so devs who prefer working with Opus, for instance, must reselect it on every subsequent startup.

This tool supports all major frontier models such as Sonnet, Opus, GPT, and Gemini. Devs at the company whom I talked to are very heavy users of the tool and have not encountered usage limitations.

Fintech company, US, Series D, ~8,000 people. Staff engineer:

“The cost in token spend is off the charts – and leadership has shared this trend with us. They have not said anything beyond showing growth in spend, and mentioning that this won’t be sustainable. So, nothing specific yet, but my sense is that something will have to change. Limits or prioritizing cheaper models, cutting back on hiring? Who knows.”

Infra company, US, publicly traded, ~5,000 people. Engineering Director:

“We’re monitoring but not restricting. We are spot checking the heaviest users, but we are seeing the business cases working out.

We are offering some guidance on model selection - e.g., turn off the new high-effort setting in Claude. Some users are trying open source models – but open source model usage is a bottom-up initiative, not a top-down one.”

Information technology, US, 10,000+ people. Director of Engineering:

“We have already had to raise our API budget limits multiple times in April. We recently switched to a much higher-effort level for Claude, which significantly increased the cost per PR.

One reason for the cost spike is using state-of-the-art models for demanding tasks. We are using that high-effort setting even for fairly trivial tasks that could have been handled by much cheaper models, or even by lower-effort Claude loops. Despite a few of us pointing this out, leadership has basically said budget is not the concern right now.

I sense that the budget increase has not been forecasted, and we’re in for a reckoning. I suspect the attitude changes once finance and other cost-conscious parts of the org realize we are spending hundreds of dollars per day, per highly-engaged developer. For now, fear of missing out and not wanting to fall behind seems to be outweighing cost discipline.”

Games studio, US+Europe, ~5,000 people. Senior developer:

“What budget increase? It’s very hard to get a budget for AI here! Claude Code is still not rolled out because $200/month/dev is seen as too high a cost. I talk with people at startups where $1,000/month in spending is totally normal, and it’s night and day here.”

Fintech company, US+Europe, late stage, ~5,000 people. Staff engineer:

“Some developers are now spending $500 a day (!!) on Claude Code. Practically speaking, this means that employee costs have doubled. Productivity has increased, in my view, but now the bottleneck is code reviews. AI can spit out code quite quickly, but we still have human reviews in place. Leadership encourages using AI for code review, but my team will not blindly trust AI.

The push from AI is coming from the top. This year’s performance review had a section on AI, rating devs by how well they used AI, so this is another reason everyone just uses it as much as they can.”

Mid-sized companies

SaaS industry, US, ~2,000 people. Dev Productivity Lead:

“Model routing helped keep our costs growing less dramatically. For example, changing the default model reduced cost by 30%. This is our strategy with AI spend, summarized:Short term: spend, spend, spend! Experiment and use whatever models make sense.Measure the impact. Measure key outcomes and report on spend, monthly.When spend vs results diverge: adjust. When our spend increases dramatically, but outcomes don’t follow: see what we can do to adjust the delta. More spend should mean better outcomes. If not, we are doing something wrong.”

Finance industry, US, ~2,000 people. VP of AI:

“We have Cursor and Claude Desktop, both of which have around 800-1,200 total users. Token usage is growing somewhat unexpectedly. Estimates are being adjusted on the fly; the initial plan to have strict limits (say, $100 per user) is breaking when reality hits, and people exhaust them in 3-5 working days.

Using expensive models is a problem. In regards to Cursor, many devs are defaulting to the most expensive models without realizing that going with Opus gives single percentage gains in intelligence compared to Sonnet, for example, while exhausting their budgets almost immediately.

We are working on blocking/managing out the most expensive models [with Cursor], as going into thousands of dollars per user, per month is not sustainable on our scale. Cursor is a good partner and we’re working with them to switch to a “pooled spend” model where heavy users can tap into a pool of extra spend.

Claude is a similar story. We were at $100 of Claude Desktop limit for everyone, but as we are moving forward, I can see that we would need to go much higher, especially for business-critical use cases.”

Infra company, US, late-stage, ~700 people. Founder:

“We haven’t had much of an issue. Most folks police themselves for runaway costs; for example, we had someone hit like $10K in a week because they messed up caching, but it was caught and they corrected their harness.

For the most part, we don’t see our high-end folks spending more than ~$1K/week. Now, to be clear, this is not a small amount! BUT it’s already a small subset of the population.

We’re just factoring it into engineering costs at this point: if it’s, say, $2K/month per employee, that’s $24K per year.

Who cares, then, when engineers already cost $200-400K/year in cash comp? Okay, so what if it’s $5K/month. That’s $60K/year.

Our bet is that token costs will stabilize and we’ll eventually end up with local-ish models.

Now, it could be five years before they stabilize, but overall, spend today isn’t that insane to me.

There’s a lot of people who are just dumb about it, but most legit execs push back on this. Take the Ralph loops or other insanity where someone spends $1K/day, $5K/week or stuff like this. That’s all just people being fools thinking they’re doing “R&D,” or somehow that they’re smarter than everyone else, but they’re just producing junk that never ships or is not useful.

We saw a bit of “stupid overspend” in the first couple months, but that’s all gone now. Costs could go up even more if we would “crack the whip” in wanting to see even more output, but we’re not doing that.”

Healthcare industry, US, ~500 people. Senior engineering manager:

“We are not holding back on spend, and have a monthly spend leaderboard. And we WANT devs to spend more on tokens! For example, one of my engineers spent $1,400 on a long Claude Code session in a single day.

We are seeing massive leverage, and we do more with the same number of people. This is why we are okay with our spending spiking. Our traffic is growing more than 10x, year-on-year, and we have managed to keep things running with the same team, and these AI tools.

Engineering is now blocked on Product and Design – which never happened before! This is how fast execution has become. We now have Staff+ engineers writing Product PRDs so we can move faster.

I’ve been in tech for close to 15 years and I never saw dramatic change like this. I just came back after a 3-month break, and every single thing is different in my day! I feel these AI agents are the biggest change in the industry since high-level languages became widespread.”

E-commerce company, US & Europe, ~2,000 devs. Head of Engineering:

“The increase in spend is INSANE. It’s about usage going up, with no signs of stopping. Usage is off the charts.

We currently do not have limits in place, and are not pausing now. Our CEO is AI-pilled and won’t let us slow down.

We do buy tokens at a discount. They start from 5% and go up with usage with the vendors we use (the usual suspects.)

We don’t let devs use anything lower than Opus 4.7 for coding. Cheaper models might work better, but a slight error pushed to prod would result in hours of toil.”

Small companies

Series A, US, ~50 people. Principal Engineer:

“About 15 devs are heavy users of AI and costs are rising very fast. Almost everyone uses Claude and Claude Code. We are considering four potential options:Increase AI budget, and start measuring more. Continue doing what we are, but allow devs to use more tokens instead of hiring limits. The precise ROI is hard to quantify, but we’ll start to measure and track both AI adoption and impact.Optimize token consumption. Use cheaper models for simpler tasks, review token usage, and see where we can cut usage. Downside: this approach could become one with diminishing returns, fast.Integrate more AI providers in the company. Find wrappers to abstract LLMs. The problem is: how do you replace Claude Code, for instance?Pivot to local models: such as Kimi, Qwen, and so on. The problem is it’s a big investment in high-end hardware or cloud GPUs. Upside: it offers better long-term cost control, once done.

We are likely to go with option #1: increase spend BUT maintain momentum and put the right measurements in place. We can do #2, #3 and #4 later. But if we kill AI usage momentum inside the company, the outcome will probably be worse.”

AI infra, US, seed stage, ~15 people. Founder:

“We saw a 15x increase in 6 months:Six months ago our spend per developer was ~$200/monthToday, it’s around $3,000/developer/month, for our seven devs
We’re not slowing usage, especially as we are building an AI infra product. The increase was much faster than expected, though.”

Small, bootstrapped company, Europe. Founding engineer:

“Our current strategy in dealing with the increase in costs is to switch to a cheaper model; unfortunately, from Opus to Sonnet in our case. That said, Sonnet is quite decent.”

How businesses manage token spend

Regardless of company size, there seems to be two strategies for how companies deal with increased spending. A summary:

Strategy #1: “let it rip and start measuring.” Around half of respondents say AI spend is rising dramatically, and they have decided to do nothing about it. They want devs to use AI as much as it makes sense to, and to help the work as much as possible.

However, because the cost is rising dramatically, these companies are now starting to measure usage and attempting to measure the impact of their AI tools.

There’s a few companies where the impact seems to be very positive, already. Smaller startups whose business is exploding in numbers of customers, load, and revenue, see that they don’t need to hire more staff because existing engineers can keep supporting the growth with AI tools.

Strategy #2: curb spending. Commonly mentioned cost-saving approaches:

• Use cheaper models for simpler tasks
• Set default models to less capable ones
• Set a spending cap and make it hard for engineers to exceed it, or require consent for doing so

Most companies using strategy #1 have briefly considered going with this approach, but threw it away, because they see this approach as optimizing on the wrong thing: cutting costs before the productivity impact of using state-of-the-art tools is even known!

Discounts exist when the spend is in the millions of dollars. I asked several people if they are getting discounts from vendors when buying tokens at scale. There were no exact numbers, but this is what I gathered in aggregate about possible custom agreements:

• Cursor: open to discounts above a few million dollars in spend. Companies have negotiated discounts with Cursor after crossing $1M of spending. Some companies negotiated tiered discounts from this level, starting at 5% and going higher as their spend goes up.
• Anthropic: no discounts. I talked with companies spending $5M+ per year on Claude which have received no discounts. If Anthropic offers discounts, it will likely be at a much higher tier.
• All discounts are custom, so try to negotiate – it’s free! Pricing discounts are on a per-customer basis, and highly custom. The easiest way to see if a discount is available is to ask the vendors!

—-

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. Load from AI breaks GitHub – but why not other vendors? GitHub’s reliability is less than one nine, and getting worse. Prolific open source contributor, Mitchell Hashimoto, is quitting GitHub because he thinks it’s not suited for professional work. GitHub’s leadership blames the 3.5x increase in service load as the cause of degradation – or it might be self-inflicted.
2. Anthropic’s speedrun to destroy trust. Anthropic could do no wrong until recently, but in the past month, that’s all changed. Silently nerfing Claude Code, banning companies from Claude, and baffling price rises all add to a sense that Anthropic is in its “extraction” era of generating more revenue for the same or worse service.
3. Industry pulse. Dramatic price increases at GitHub Copilot, explosive growth at Codex, Google scrambling to build a good coding model, Cursor might be bought by SpaceX, AI agent deletes car business, and more.
4. Mitchell Hashimoto & the “building block economy.” Ghostty’s creator finds that open source “building blocks” are the best way to win massive adoption by software components – but it’s got harder to build a business on top of open building blocks.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

GitHub’s reliability has been beyond unacceptable recently: last month, third party measurements pinned it at one nine (right at 90%). This month, reliability has been down to zero nines – 86% – as per a third-party tracker, and last week, things got even worse: a frankly embarrassing data integrity incident, more outages, and a partial explanation from GitHub, eventually.

Data integrity incident

Last Thursday (23 April), this happened: PRs merged via the merge queue using the squash merge method produced incorrect merge commits, when the merge group contained more than one PR. Commits were reverted from subsequent merges: basically, commits were “lost” in the code that was merged!

Thanks to a bug GitHub introduced, the service broke its integrity promise that pull requests would be merged as expected when using squash merge, which is a technique typically used to merge multiple small commits into a single, meaningful commit. This is a big deal: as data integrity promises are some of the most important ones, for services like GitHub.

A total of 2,092 pull requests were impacted, and companies hit by the outage included Modal and Zipline. Effectively, GitHub pushed a bunch of work on affected customers who had to manually untangle and recover lost commits, which GitHub could offer zero assistance with.

Customers had to manually go through their git history and restore missing code. After following manual recovery steps (reverting the squash commit and re-applying commits one by one), all commits should have been recovered.

GitHub later emailed the list of affected commits to customers, but it’s odd that GitHub executives seemed to downplay the nature of this outage. After all, an outage that messes with data integrity is a much bigger deal than something like a fall in availability where no data is corrupted.

Can Duruk, software engineer at Modal, was unhappy about GitHub’s muted response to the outage:

“The COO going out of their way to find a huge denominator to make the impact appear small feels very dishonest; versus a sincere apology about how this invalidates their entire promise to their customers. We had to dig into their status page about this to even realize they just casually f***ed up our repo.”

Outages don’t stop

On Monday (27 April), pull requests and issues disappeared from GitHub’s web UI:

This had to do with an Elasticsearch outage on GitHub’s backend: the cluster became overloaded and went down. So, while pull requests, issues, and projects didn’t vanish altogether, they also didn’t show up during the 6-hour-long outage.

There were other outages this week:

• Some pull requests not showing up (Tuesday, 28 April)
• Problems with some GitHub Actions (the same day)
• Incomplete pull requests in repositories (Wednesday, 29 April)

Also on Tuesday (28 April), security firm Wiz disclosed a critical security issue, where a bad actor could get access to all repositories on GitHub and GitHub Enterprise server by using only a git push command. GitHub fixed the issue on GitHub.com within six hours, but GitHub Enterprise servers that were not updated remain vulnerable.

Famous open source contributor quits GitHub in frustration

On Tuesday, Mitchell Hashimoto, founder of HashiCorp, creator of Ghostty, announced GitHub was unfit for professional work and that he was moving off to Ghostty, the open source terminal that’s his main focus. Mitchell’s reasoning was dead simple: being on GitHub makes him unproductive (emphasis mine:)

“The past month I’ve kept a journal where I put an “X” next to every date where a GitHub outage has negatively impacted my ability to work. Almost every day has an X. On the day I am writing this post, I’ve been unable to do any PR review for ~2 hours because there is a GitHub Actions outage. This is no longer a place for serious work if it just blocks you out for hours per day, every day.

It’s not a fun place for me to be anymore. I want to be there, but it doesn’t want me to be there. I want to get work done and it doesn’t want me to get work done. I want to ship software and it doesn’t want me to ship software.

I want it to be better, but I also want to code. And I can’t code with GitHub anymore. I’m sorry. After 18 years, I’ve got to go. I’d love to come back one day, but this will have to be predicated on real results and improvements, not words and promises.”

Mitchell’s experience suggests that GitHub’s official status page is inaccurate from the point of view of a heavy user like himself. The third-party “missing GitHub status page” is likely to be a better estimation: where GitHub’s reliability is at zero nines: at 85.51% uptime. That means that a part of GitHub was down for 2-3 hours, per day, on average, for the last 90 days (!!)

Mitchell’s complaint sounds straightforward:

1. As a professional software engineer, it’s important to have tools that help you get work done
2. For months, GitHub has got in the way of his work on open source projects via a flood of outages
3. It makes no sense to use a product unfit for professional work.
4. As GitHub shows no signs of improvement, it’s worthwhile to move to a different solution which just works

CTO blames AI agent-fuelled load spike

GitHub CTO, Vlad Fedorov, shared an update on why reliability has been terrible for months at GitHub. He identified the load from agents being much bigger than expected as the culprit. Charts illustrating this were shared by GitHub:

This chart looks eye-catching – but there’s just one tiny issue: no Y axis! So, while it tells the story of the load going up slowly and then very fast, we’re not told by how much. However, I managed to get data from GitHub, and below is the chart showing the actual load increase over two years:

A load increase of ~3.5x, spread across two years, doesn’t seem so brutal at first glance. It is nothing like a load increase of 10x in a month, and a good chunk of it occurred in recent months. So, why can’t GitHub handle it? In a blog post, Fedorov said:

“A pull request can touch Git storage, mergeability checks, branch protection, GitHub Actions, search, notifications, permissions, webhooks, APIs, background jobs, caches, and databases. At large scale, small inefficiencies compound: queues deepen, cache misses become database load, indexes fall behind, retries amplify traffic, and one slow dependency can affect several product experiences.”

Here’s how the per-second load numbers from January 2023 and today compare:

GitHub took 15 years to achieve the 2023 numbers, and maybe it expected to continue growing in a comparable way in the future. If so, some engineering decisions about long-term infrastructure improvements would have been made obsolete by the arrival of AI agents.

To add to GitHub’s challenges, the company is in the midst of a migration from its own data centers → Azure. In October last year, GitHub started to move over to Azure – a project expected to take 12 months – because it already had constraints on its own data center capacity.

Such large-scale infrastructure migrations are hard enough when the load on a service is relatively stable; just making sure nothing breaks takes a lot of effort. But moving at a time when load is spiking means that bugs can cause more visible outages. Of course, GitHub can secure a lot more compute capacity on Azure, now they know what to expect.

But other major companies prepared for a 10x increase in infra load, so why not Microsoft / GitHub? A year ago, I did research on how Big Tech was preparing to respond to the impact of AI on their business. Google was improving its internal systems to accommodate for a 10x increase in load. As we covered in The Pragmatic Engineer, in July last year:

“Google is preparing for 10x more code to be shipped. A former Google Site Reliability Engineer (SRE) told me:

“What I’m hearing from SRE friends is that they are preparing for 10x the lines of code making their way into production.”

If any company has data on the likely impact of AI tools, it’s Google. 10x as much code generated will likely also mean 10x more: code review, deployments, feature flags, source control footprint and, perhaps, even bugs and outages, if not handled with care.”

Predicted enormous load increases were not secret knowledge within the industry, yet it seems GitHub was blissfully ignorant of their potential size. According to Vlad, GitHub did eventually plan for a need to increase capacity by 10x, but this was in October 2025, months later. In February 2026, the company is now adjusting that expectation to 30x. He wrote:

“We started executing our plan to increase GitHub’s capacity by 10X in October 2025 with a goal of substantially improving reliability and failover. By February 2026, it was clear that we needed to design for a future that requires 30X today’s scale.”

There’s also the question of whether GitHub miscalculated how much time it had to prepare for explosive load growth, and whether it was caught off guard when that growth materialized months sooner than expected at the start of this year.

Given GitHub only started to prepare for a major load increase in October, its current problems are unsurprising. At the scale of GitHub, it’s common enough for each team owning a service to plan a year ahead on how much load their service will have, and hardware resources like storage, VMs, and networking are allocated accordingly. Load planning can account for up to half of the preparations, and when reality doesn’t conform to plans, some systems can struggle to scale up.

So, on one hand, dealing with a 3.5x increase in load over 2 years should not be such a big deal for most services; especially not ones which can be horizontally scaled (when there’s not much state, and scaling is achieved simply by adding new nodes.) But GitHub probably stores a lot more state with pull requests, workflows, projects, etc. This probably makes scaling more tricky when it comes to databases and systems running workflows.

GitHub also has 18 years of tech debt on its hands, and thousands of staff to align as “organizational overhead.” As its service load grows faster than before, responding is harder due to all that accumulated “debt”:

• Tech debt: many systems at the company are 10+ years old and are likely patched up, making them more difficult and risky to change
• Organizational debt: around 4,000 people work at GitHub, of whom 1,000 are engineers. Teams have dependencies with each other, and even seemingly simple work can require dozens of engineers to work together
• Customer expectations: GitHub cannot break customer workflows, even if doing so would mean changes to systems happen faster

GitHub finds itself in the ‘innovator’s dilemma’: the company became successful because it built developer workflows that made sense, pre-AI, and it used to be able to accurately forecast service load changes. But now that engineering teams’ workflows include AI agents, GitHub’s own workflows are not necessarily the best fit, and the company failed to forecast service-level changes.

Other vendors floored by AI load? Not really

One thing that doesn’t add up about the situation is that other vendors who are presumably experiencing similar load spikes don’t appear to be suffering with reliability issues as much. Vercel, Linear, Resend, Railway, Sentry, and other infra providers see record-level growth thanks to AI, but keep up with the load.

Yes, it’s true that AI vendors like Anthropic, OpenAI, and Cursor have some reliability issues, but it’s not at the scale of GitHub’s. GitHub’s direct competitors, GitLab and Bitbucket, presumably see load going up similarly, but they’re not going down as much.

An obvious question is how much of GitHub’s pain is self-inflicted? With Microsoft as owner, it has more resources at its disposal than any competitor or startup, and yet failed to predict load increases and is too big to respond with the nimbleness of a startup.

It’s undeniable that solving for a major load increase is a hard challenge; it’s when the difference between average and standout engineering teams is apparent. GitHub hasn’t been responding like a world-class engineering org.

GitHub alternatives?

Every regular user of GitHub feels the pain of ongoing outages. As a dev, you can either hope Microsoft will eventually improve reliability, or seek alternatives. As covered above, Mitchell has chosen to quit and is currently deciding where to take Ghostty.

The obvious alternatives are GitHub’s biggest competitors, GitLab, and Bitbucket. Each offers Git hosting, and neither comes with the uptime woes that GitHub is suffering from.

Self-hosted solutions are also an option, like self-hosting your git repo, or going with a self-hosted forge like Forgejo, which is an open source, local-first GitHub alternative.

I also suspect that, soon enough, we’ll see startups offering GitHub-like code hosting capabilities, while offering more robust uptime and being architected to handle the 30x-or-more scale which GitHub hopes one day to support.

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. Did Anthropic turn hostile on devs because capacity was running low?
2. Amazon finally allows Claude Code and Codex usage
3. Meta forcefully assigns engineers to data labelling ahead of job cuts
4. New trend: small “AI-forward” teams
5. Industry Pulse: why Meta tracks employees’ computer activity, OpenAI starts to move off Datadog, Apple lets slip it uses Claude Code, GitHub → Xbox transfers at Microsoft, VS Code inserted “coathored by Copilot” even when Copilot did nothing, analysis of the Coinbase layoffs

tl;dr: TechPays is joining Levels.fyi: so the leading tech salary site in Europe gets the love and care it deserves. Thanks to Zsombor for building this project with me for so many years.

Pay transparency has always been an issue in tech, especially in Europe. For a while, I assumed that the most that a senior+ software engineer could make in London or Amsterdam would be in the realm of £100K / €100K. Once you reach that level, you've made it. You’re now at the very top of the market! Or are you?

So when I was making £93K in London, working as a principal engineer at Skyscanner in 2016, I was not expecting that I could be compensated meaningfully better. Pay surveys kept confirming that I'm well above the median, and into the 90th percentile of pay grades.

Imagine my surprise when I got an offer from Uber, in Amsterdam, that effectively doubled by compensation, into the realm of around €220-250K ($260-295K). By year four, I made €283K ($332K):

It felt like I discovered a "secret, upper-tier" of the market that no one else knew about. When I became a manager at Uber, and started hiring for my team, several strong software engineers were hesitant to move forward with the process, because they assumed that they were at the very top of the market – but they still made ~half of what we would have offered! I had no way of telling them "your data is wrong, this place pays a lot more!" and so several of them just never bothered to interview, assuming the most raise they would get would be 5-10%. When they could have potentially doubled their compensation…

I saw first-hand that not having good compensation information works against us, developers, and decided to try and change this. I collected data points from closer to 200 engineers working in the Netherlands, and explained that there's a third, "hidden" tier of compensation in The Trimodal Nature of Software Engineering Salaries in the Netherlands and Europe.

After the success of the article, I decided to "open source" compensation data points I collected, and thus TechPays was born:

I built this site together with Zsombor Erdődy-Nagy. We paid attention to support compensation anonymization, capture freelancer compensation, and break down how compensation packages were put together. We've received so many heart-warming stories on how you've been able to negotiate better compensation packages, thanks to having access to this information.

Knowing that we're making a difference kept us going for a few years, as a side project. However, over time, both Zsombor and I got busier with other projects. For me, it was The Pragmatic Engineer taking up more of my time. We wanted to find a way to keep TechPays running, and get the care it deserves.

Levels.fyi will be taking over operating TechPays – and taking learnings about European compensation packages, and integrating into their global pay transparency platform. I've known Levels.fyi founders Zuhayeer and Zaheer for years, and we share our drive to make compensation as transparent as possible, across the tech industry.

With TechPays, there are no changes: you get to browse the data, as before. And expect even more, high-quality data points on Levels.fyi, for Europe, and globally.

To get more details on compensation, check out Levels.fyi. And read the Trimodal nature of tech compensation in the US, UK and India, based on Levels.fyi data points:

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of five topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Last week, we reported on Anthropic seemingly being on a speed run to break devs’ goodwill by silently “nerfing” Claude Code, banning corporate accounts without warning, and a weird growth experiment involving revoking Claude Code and then restoring it. This week, a dev on the $20/month Pro plan had Claude Code removed just days into their subscription:

This week, Anthropic announced a big data center expansion, and relaxing previous usage limitations, while Elon Musk’s SpaceX / xAI ( a single company after a merger) is renting its complete Colossus 1 data center to Anthropic. From the announcement:

“Colossus 1 features over 220,000 NVIDIA GPUs, including dense deployments of H100, H200, and next-generation GB200 accelerators. The cluster delivers extreme parallel performance for large language models, multimodal systems, scientific simulations, and generative AI at frontier scale.

Anthropic plans to use this additional compute to directly improve capacity for Claude Pro and Claude Max subscribers.”

In parallel with this release, Anthropic announced:

• Doubling Claude Code’s current 5-hour limits for Pro, Max, Team, and seat-based Enterprise plans
• Removing peak hours limit reduction on Claude Code for Pro and Max plans
• Substantially raising API rate limits for Opus models

Is it possible that capacity issues are what led Anthropic to make Claude worse? It’s confirmed the company has struggled with capacity for months. Conveniently, Claude Code being “nerfed” led to lower compute load, while removing Claude Code access from cheap plans could look like rate limiting. Even the banning of corporate accounts could be seen as scaling back at a time when the business has struggled to serve existing growth. Yesterday, (6 May), at the Code with Claude event hosted by Anthropic, CEO, Dario Amodei, said:

“We originally planned for 10x growth, and we’ve seen something more like 80x growth in revenue and usage over the last period of time.”

SpaceX / xAI renting a good chunk of its capacity to Anthropic is ironic, considering that xAI (Musk’s AI startup) builds Grok, a frontier model and direct rival of Claude, and also in January, Anthropic banned xAI developers from Claude. As covered at the time:

“It’s common for an AI lab to not allow another AI lab to use its model, like at OpenAI, Anthropic, and Google. On the other side, there’s also the pertinent question of why a leading AI lab would even want to use a rival for its own day-to-day work?

Turns out, xAI (Elon Musk’s AI lab) was relying on Cursor to write code, which we know because they got cut off.”

Anthropic likely banned xAI to stop Claude from being potentially distilled while it tried to improve Grok’s coding capability. Meanwhile, Musk called Anthropic “misanthropic and evil” earlier this year, and said the new tenant “hates Western civilization”. But both parties seem happy to put that behind them and strike a deal, so perhaps there’s something else at play.

Could SpaceX / xAI be checking out of the frontier-AI model wars? Leasing a good chunk of its data center capacity might suggest that. SpaceX / xAI has two data centers: Colossus 1 and Colossus 2. Colossus 1 represents somewhere around 45% of current SpaceX / xAI capacity, and 20-25% of planned total capacity.

Giving up as much capacity as this might indicate a lack of demand, or capacity sitting idle. It also means Grok is losing out in market share to Claude, ChatGPT, and other leading models. In February’s AI tooling survey we found scarce mention of Grok, which lagged in usage behind open models like DeepSeek and Qwen.

To be fair, unlike Anthropic and OpenAI, Grok never had a B2C nor B2B business that took off. The biggest consumer use case for Grok seems to be its integration into the social media platform, X; at least, I don’t know of any tech company using the model for serious work.

“The enemy of my enemy is my friend”, says the maxim, and if there’s one company Musk hates, it’s OpenAI. He is currently suing OpenAI, claiming it betrayed its founding nonprofit mission to develop safe AGI for humanity’s benefit by shifting to a profit-driven model backed by Microsoft. Musk also claims that despite investing about $40M, he has no ownership of the company.

He wants $150B in damages, the removal of Sam Altman and Greg Brockman, and for OpenAI to return to a full nonprofit, as per when he invested in the company. We covered more about OpenAI’s own ethical challenges between nonprofit and for-profit right after the firing of Sam Altman in 2023, in the deepdive What is OpenAI, really?

Similarly, Anthropic may well have an issue with OpenAI, if CEO Dario Amodei’s failure to join hands with Sam Altman while sharing a stage with the Prime Minister of India earlier this year is anything to go by.

Capacity issues hurting Anthropic would benefit OpenAI, and so by offering significant capacity to Anthropic, Musk is making it harder for OpenAI to win the market. That would be ironic, given he’s a former investor.

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. Forward deployed engineering heats up again. Massive demand for the role at Google, OpenAI, and Anthropic. The latest version of the FDE role looks like the consultant / solution architect role done by many early-junior engineers.
2. Why are layoffs spiking? Tech job cuts are higher than since early 2023 for various reasons: smaller teams prompt reorgs and reduce the need for middle management. Meanwhile, poorly performing companies make layoffs without the influence of AI.
3. New trend: self-reporting 100% AI generated code at Microsoft. With mid-year performance reviews looming, some managers advise their reports to claim they use AI for everything.
4. Industry Pulse. Tokenmaxxing at Amazon, too, SaaS companies grow faster than before – perhaps partly due to AI, Bun rewritten in Rust with AI works well, Anthropic overtakes OpenAI in enterprise spend, and more.
5. Vibe coding & agentic engineering get uncomfortably close. A relatable observation by software engineer, Simon Willison, about reviewing AI agents’ code less than would be ideal.

A $124B fund in Australia would have lost all data stored with Google Cloud, had they not relied on a third-party backup. A rare blunder from GCP, where regional replication did not stop the deletion – and a just as rare statement from Google Cloud’s CEO taking the blame.

The below is an excerpt from The Pulse #93: OpenAI makes Google dance, originally published on 16 May, 2024. I am republishing it because on 20 May 2026 Google Cloud has done it again: they took offline cloud infra provider Railway by blocking Railway's Google Cloud account. The below is a warning: if you are on GCP, have a plan B, should GCP delete your account and data, or block your account.

Following on from Google Cloud being a distant third among cloud providers, a recent event could cement its reputation as the least reliable of the top three. 

UniSuper is one of the largest retirement savings accounts in Australia, used by 615,000 citizens, that’s also known as a “superannuation fund.” UniSuper has $124B of assets under management and is one of the biggest in the country.

On 29 April, the service suffered an outage. Members could not log into their online accounts, or manage their funds until two weeks later, on 15 May.

The reason was that Google Cloud accidentally deleted UniSuper’s subscription, which also deleted all data associated with the subscription. UniSuper had set up replication across two regions in Google Cloud to protect from a regional failure, but Google Cloud deleted the replica as well!

UniSuper could only avoid data loss thanks to having a backup on another service provider outside Google. In a surprising admission, UniSuper would have lost all data with Google thanks to the failure of the cloud provider. The only reason UniSuper could restore services was by having another provider with whom they’d backed up the data. Basically, UniSuper not trusting Google’s replication across two regions turned out to be a 100% correct assumption. Whoever pushed through the decision to spend additional resources in “a backup in case Google fails” saved the day at the retirement fund.

The incident is incredibly embarrassing for Google. UniSuper seems to have forced Google Cloud’s hand by issuing a joint statement with Google Cloud CEO Thomas Kurian, in which Google Cloud takes all the blame for this failure. In my experience, the situation is rarely this black-and-white, as it usually takes two parties to cause such a major outage. I would not be shocked if it turned out UniSuper’s staff played a role in this failure, but Google Cloud made enough mistakes that the press release could dump all blame on it. I asked Google Cloud if the press release really was a joint release, and if they had more to add. The company confirmed the press release is correct and added nothing else.

Whoever was at fault, two weeks of downtime is still very long for a major fund. As I understand, the damage to UniSuper is mainly reputational because the funds are safe and secure. 

Users could not see their balances for a few weeks, and were told Google Cloud had messed things up. This means there are up to 615,000 Australians in whose minds UniSuper and Google Cloud are indelibly linked with unreliability.

I keep seeing that Google Cloud has no apparent strategy for what it wants its cloud to offer. A few months ago, we dived into how AWS, Azure and GCP respond to regional outages, and I concluded it’s hard to see a strategy at GCP beyond following processes, while doing the least impressive job of all three cloud providers. It’s hard to gain market share if you remain the slowest to respond to regional outages, and the provider for whom a zone outage takes down a region, or which loses all customers’ data, despite regional replication, by deleting it.

This incident is a reminder you shouldn’t fully trust your cloud provider. UniSuper was smart to have backups elsewhere for its data in Google Cloud. And while it’s tempting to point fingers at Google Cloud: there are no definite assurances that another vendor would not make a similarly unprecedented mistake in the future!

The learning is that if you have really valuable data, keep a backup somewhere else. If you use any cloud provider, use another cloud, on-prem backups, or something else.

Read the full The Pulse issue

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of five topics from last week’s The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Last August, we covered a sudden trend of high demand for forward deployed engineers (FDEs), and now there are signs demand is increasing more.

Google: FDE recruitment spike

Google is doubling down on FDEs and making the interview process much simpler. Google Cloud CEO, Thomas Kurian, has announced a new, AI-focused organization within the Go-To-Market team, and is hiring a bunch of FDEs for it.

I’m hearing the hiring process has been shortened from 4-6 interviews held over the course of weeks, to as few as two interviews in just two days. It looks like Google is unusually eager (desperate?) to fill this job.

OpenAI outsources FDE hiring spree

On Monday (11 May), OpenAI announced The OpenAI Deployment Company, a standalone entity funded by $4 billion of private equity from TPG, Advent, and others at a $14B valuation. It appears OpenAI is not an investor and holds a partner role.

The announcement mentions FDEs and says their job will be to “work closely with business leaders, operators, and frontline teams to identify where AI can make the biggest impact, redesign organizational infrastructure and critical workflows around it, and turn those gains into durable systems”.

Based on that, the FDEs will play an important role in OpenAI’s enterprise sales activity by ensuring the company’s AI systems work and deliver value for customers. Outsourcing this to the new Deployment Company should also free up OpenAI to focus on developing better AI models, while the partner company and its FDEs take care of the customer-facing side of things.

In a related development, OpenAI has acquired Tomoro, a UK-headquartered AI company founded in 2023, which employs 150 FDEs across the UK, Asia, and Australia. Tomoro is the first acquisition of the OpenAI Deployment Company.

Anthropic plans outsourced FDE recruitment

Anthropic is doing the same by creating its own distinct FDE consulting company. Last Monday (May 4), Anthropic issued an unusually hand-wavy announcement about the new business without a name and with few investment details mentioned.

Investors are Anthropic, Blackstone, Hellman & Friedman, and Goldman Sachs, and the new business will work with “mid-sized companies across sectors to bring Claude into their most important operations.”

Anthropic’s approach seems to be the same as OpenAI’s: create a standalone company with external funding, in which FDEs integrate Claude into enterprises that will then presumably start purchasing more Claude tokens than ever.

FDE or a consultant?

These FDE roles seem very similar to those of an external consultant or a systems integrator. A year ago, I talked with FDEs at OpenAI and Ramp whose jobs seemed a genuine mix of platform engineering – with an FDE contributing back to the platform – software engineering, in that they built new solutions, and also solutions engineering: integrating into customers’ services.

But today, it looks like the role is about to become indistinguishable from a solutions architect or consultant, especially given that these new FDE jobs are in quasi-external companies and separate organizations from where AI products are built.

Job adverts are increasingly clear about the role, but it still helps to read between the lines. Here’s one for an FDE at Google Cloud. At first glance, it’s impressive (emphasis mine):

“You are an embedded builder who bridges the gap between frontier AI products and production-grade reality within customers. Unlike traditional advisory roles, you function as an “innovator-builder,” moving beyond high-level architecture to code, debug, and jointly ship bespoke agentic solutions directly within the customer’s environment. Your role is designed for high-agency engineers with a founder’s mindset. You will address blockers to production, including solving the integration complexities, data readiness issues, and state-management challenges that prevent AI from reaching enterprise-grade maturity. By embedding with strategic accounts, you serve a dual purpose: providing “white glove” deployment of complex AI systems and acting as a critical feedback loop, transforming real-world field insights into Google Cloud’s future product roadmap.”

Translated into plain English:

You are a contractor who codes at a customer’s office. The actual job is around ~25% coding-related, 50% integration/plumbing, 25% meetings and customer hand-holding. Anything else will be assorted admin and internal process-related stuff.

Here’s what I reckon some of the terms in Google’s job advert will add up to on the job:

• “Founder’s mindset”. No one will provide a spec, and scope creep is your problem to deal with. If your project doesn’t ship, that’s also your problem
• “High-agency”. There are no resources besides your own
• “White glove”. Do not say “no” to anything the customer suggests, even when they should probably listen to your feedback about whatever it is
• “Critical feedback loop transforming real-world field insights into Google Cloud’s future product roadmap”. You will file tickets and a few PMs at Google may read some of them

But in all fairness, this FDE job looks like a great fit for some folks:

• Those at the early-career stage who want Google on their resumes, but who might struggle to land a software engineering job with the tech giant
• Those who enjoy shipping end-to-end, can work well with ambiguity (“founder’s mindset” is spot on!) and will own outcomes

On the other hand, I suspect this FDE role will not be a good fit for those who:

• Like to build well-engineered systems and value the time to do it well
• Like building greenfield systems
• Prefer longer-term projects and working with other software engineers

In the cases of OpenAI and Anthropic, the outsourcing of FDEs is even clearer. Google at least hires FDEs to the company, and they will be issued some stock as part of their compensation package. But at OpenAI and Anthropic, new FDEs will be hired to a standalone company, and if they get stock, it will likely not be OpenAI or Anthropic stock. So, if OpenAI or Anthropic benefit greatly from FDEs’ work, then the FDEs won’t see the upside!

Putting it more simply: FDEs hired in these external companies will not be seen as “core”. If they were, then the companies would hire more FDEs, as in the past.

Opportunity for new grads?

As mentioned above, the new FDE roles could be a great opportunity for early-career software engineers entering the industry, according to Box CEO, Aaron Levie:

“If I were a college career counselor or in career services, I’d quickly be figuring out how to get students to understand these forward deployed engineer jobs exist and how to get them.

The requirements are a mix of deep technical skills, often CS majors or minors. You must be great at understanding problem solving, how to have systems thinking, and have a strong business acumen. The kicker, of course, is to make sure you’re very deep in AI agents; you need to have fluency in coding agents, MCP, CLIs, Skills, and so on.

Hundreds (thousands?) technology companies will be hiring for these roles, same with any consulting and IT services company, and the vast majority of mid-size and large enterprises will be hiring for this talent internally as well.”

Historically, tech consultancies hired many new grads for consultant roles, which are not so attractive to experienced engineers, but are great, real-world, paid learning opportunities for more junior ones. With product companies hiring fewer new grads, new grads will increasingly find FDE roles that they have a chance at getting.

All things considered, I expect demand for FDE roles to increase, industry-wide. They speed up AI rollouts, which several parties have an interest in doing:

• AI labs: the faster that AI solutions roll out, the more revenue they make!
• AI vendors: any company selling AI products will, similarly, want FDEs to help integrate the software with customers, so they can sell more
• Non-AI companies: these will want to hire FDEs for an “AI transformation” and to integrate AI into workflows and products
• Non-AI vendors: even SaaS companies that don’t sell AI products will be able to close larger clients if they hire FDEs who can roll out their software faster, and for more use cases, inside enterprises they work with.

FDE was the hottest tech role in 2025 and this trend seems set to continue this year. Demand for this role is high and rising, but it’s likely to stay unattractive to experienced devs for whom being a consultant may feel like a step down – especially after you’ve learned to love building products!

Read the full issue of last week’s The Pulse, or check out this week’s The Pulse. This week’s issue covers:

1. Antigravity 2.0 takes the ‘IDE’ out of its new IDE. Feedback about the redesigned IDE is overwhelmingly negative due to bugs, poor UX & model support, and eating through Gemini token quotas. Also: a clue that Antigravity’s own devs use other tools for their work?
2. Why is Google’s product ecosystem chaotic? The range of products on display at the Google I/O conference made a messy, incoherent impression. But Google’s “let a thousand flowers bloom” approach might be giving the search giant an underrated advantage in the AI race that no other Big Tech giant has.
3. Meta cuts 8,000 jobs. Morale is very low inside the social media giant as thousands lose their jobs, just as revenue and profits hit record levels. Meanwhile, those assigned to dull data labeling work are spared the axe.
4. Industry pulse. Anthropic pays $15B/year for SpaceX compute, SpaceX’s financials and IPO filing, more woes for GitHub, court dismisses Elon Musk’s “hypocritical” OpenAI lawsuit, and Spain may stop blocking its internet during La Liga football games.
5. How to get a job at a frontier lab in 2026. A Distinguished Engineer at Google recommends focusing on developing particular skills

Read the full The Pulse.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from The Pulse issue from 21 May 2026. Full subscribers received the article below three weeks days ago. If you’ve been forwarded this email, you can subscribe here.

Yesterday, Google launched a full redesign of its flagship AI IDE, Antigravity 2.0. The “original” Antigravity came out in November 2025, as pretty much a clone of Windsurf, the IDE whose team Google acquired for $2.4B last July.

Google has turned Antigravity into two distinct applications, “Antigravity IDE” (its former incarnation) and “Antigravity 2.0”. This new version itself resembles a clone of Codex’s desktop app. When you install Antigravity 2.0, there are two different applications. From Google’s launch post:

“If you already have installed the Antigravity IDE, when that application next updates, it will automatically update to Antigravity 2.0. At this point, you will be asked if you would like to still keep the Antigravity IDE, which is recommended for developers:

My sense is that the team at Google may have struggled to decide whether to keep supporting “original” Antigravity while investing in the Codex-like experience, and so kept both. Whatever the reasoning, it has created confusing naming, and it feels to me like the team’s true focus is 2.0.

The upgrade feels rushed, sloppy, and poorly thought out. I had Antigravity on my machine, and installed Antigravity 2.0 separately. I wanted to use them side-by-side, but when I tapped “Restart to update” on Antigravity 1.0, it upgraded itself to Antigravity 2.0 (the non-IDE version). Suddenly, I had two applications with different names, but neither is an IDE:

Google has introduced an “Agent Manager” concept that feels unintuitive, and perhaps suitably, its creator struggles to explain it (emphasis mine:)

“When we launched the Google Antigravity IDE in November 2025, there was no agent-first GUI surface in the market. We wanted to prove that such a surface worked, at least for software development. So, while the core of the Antigravity IDE was a familiar agent-powered IDE, we introduced the Agent Manager, a second surface that stripped away much of the “IDE” UI. This allowed users to focus on the agent conversations themselves, the artifacts the agents produced, and multi-agent management.

Even without this separation, we have been pleasantly surprised how many people have adopted the Agent Manager in the Antigravity IDE for such non-development tasks, but it is not particularly intuitive”.

The “Agent Manager” is basically a way to launch several agents, and the most intuitive interfaces for doing so are inside Claude and Codex desktop apps and Claude CoWork. Antigravity 2.0 copies them by starting new agent tasks on the right hand of the UI, and keeping track of them.

Google looks indecisive about what to do with the IDE part of Antigravity. The release announcement suggests they’ll keep on confusing users (emphasis mine:)

“Although Antigravity 2.0 is the future, we won’t disrupt your workflows right away. For now, both the Antigravity IDE application itself and the Agent Manager in the Antigravity IDE will remain available. In an upcoming release, we will remove the Agent Manager from the Antigravity IDE, turning the IDE into a purely agent-powered IDE.”

Basically, the Antigravity IDE (not “the future” in Google’s vision) will become more limited over time. It’s unclear what a “purely” agent-powered IDE will be once agentic functionality is removed, especially as Antigravity IDE is not the future, as per Google.

Not only that, but the announcement also encourages devs to use Antigravity 2.0 with other IDEs! From the launch post (emphasis mine):

“We recommend dual-wielding Antigravity 2.0 with your IDE of choice, whether it is the Antigravity IDE or otherwise. Googlers have already been dual wielding Antigravity 2.0 with a whole host of IDEs! We will have compatible extensions and plugins into other popular IDEs shortly”.

To me, this suggests Google will retire Antigravity IDE and recommend VS Code, JetBrains, Cursor, or Zed, with Antigravity. Then again, why would Cursor or Zed support Antigravity? The messaging is extremely confusing: Google’s still the king of opacity.

Feedback on Antigravity 2.0 has been negative due to bugs, poor UX and model support, more bugs, and eating up Gemini token quotas rapidly. Antigravity does not support state-of-the-art Anthropic or OpenAI models (no Opus 4.7 or GPT 5.5). Not supporting OpenAI’s models like this is sensible as they’re competitors, but Google is an investor in Anthropic, so not supporting Opus 4.7 (while supporting the legacy 4.6 model) is a bit odd.

Gemini 3.5 Flash is Google’s cutting-edge model, but it gets lots of complaints from devs for editing files without asking, and seems like an inefficient model. Another common complaint is that Antigravity uses up the $100/month Ultra subscription daily quota in minutes. Basically, it seems like a poor-quality product that wasn’t polished due to lack of time or inclination.

In context, it’s embarrassing for there to be a “Codex” folder in the launch video if it suggests that Google’s own Antigravity devs are using Codex for day-to-day work. It also suggests that the launch video was not reviewed properly, otherwise this obvious detail would presumably have been caught and fixed:

To upset devs even more, Google is replacing its open source Gemini CLI with the closed source Antigravity CLI. There are a few issues with this move:

• Antigravity CLI does not support Google’s own Agent Client Protocol (ACP), used for programmatic control, primarily for IDE and other developer tool integrations. This is protocol which IDEs like JetBrains and Zed have adopted, so Antigravity CLI becomes incompatible with them
• Google offers no migration path from Gemini CLI settings/skills/MCPs into Antigravity. Figure it out on your own!
• Devs using Gemini models are forced to move as Google has removed support for Gemini 3.5 Flash model from Gemini CLI. It can only be used from Antigravity CLI. Clearly, this was done to force a move. Why not offer a migration path?

My sense is the Antigravity team is moving fast, breaking things, and shipping a broken product. It feels like the Antigravity 2.0 and Antigravity CLI products have been rushed to meet the annual Google conference (Google I/O) deadline, this week. Google deprecates existing products to attempt to get users to switch to the new version. But the new one is broken.

And this is a big reason why I don’t believe Google will become a serious player in the dev tools space – not even with AI dev tools. Every six to twelve months they remind devs who onboarded to their dev tools that it was a mistake to do so. I would expect the majority of Google CLI and Antigravity users to go and try products from other vendors – be that Cursor, Anthropic, OpenAI, GitHub, or others – and for few to stick around after their workflows are broken.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from The Pulse issue from two weeks ago. Full subscribers received the article below fourteen days ago. If you’ve been forwarded this email, you can subscribe here.

The below The Pulse is interesting, as a week after the original was sent it out, OpenAI CEO Sam Altman also said how AI budgeting is a huge issue for some companies – echoing findings from this analysis.

In mid-May, Uber president, Andrew McDonald, was on the Rapid Response podcast for a conversation about the ridesharing giant with host Bob Safian, who raised the lack of hoped-for efficiency leverage from AI, citing the language learning app, Duolingo.

“When you hear companies talking about 25% of code commits over the last quarter were AI-driven, or how their token usage went from X to Y percentage of employees: all these numbers are amazing. I think it’s a massive transformation of society”, McDonald said.

“But, then you go and you talk to your senior engineering leaders, and you’re asking: “how many projects that were “on the cutting room floor” got moved above the line [of being done] because of the productivity gains? Because 25% of our code commits were via Claude Code last quarter.”

That link [of improved productivity thanks to AI] is not there yet. I mean, maybe implicitly there’s more that is getting shipped, but it’s very hard to draw a line between one of those stats and more useful consumer features.

Over the coming quarters and years, maybe that will become clearer. But today it’s hard, even if some of the underlying metrics are trending in a really astronomical direction.

Our CTO, Praveen, went viral because he said in an interview that we had blown through our AI budget for 2026 and it was the middle of March. We’re going to have to start talking about token consumption and the associated cost versus headcount, and making tradesoffs on that as an engineering organization.

If you’re not able to draw a direct line to [how many] useful features and functionality you’re shipping to your users, that tradeoff [on AI spend] becomes harder to justify because AI is not free.

If you’re just a user [of AI tools] sitting there and coming up with interesting use cases, and you don’t pay the bill, it can feel [like AI is free]. But somebody’s paying the bill”.

My hunch is that pretty much every company is starting to, or will do soon, ask questions about the massive growth in AI spend; starting with AI coding tools. I talked with a few folks at larger and smaller companies about it:

• OpenCode: customer demand for optimizing spend is spiking. Yesterday, on the podcast episode with OpenCode creator, Dax Raad, he said demand for OpenCode’s hosted inference service (OpenCode Zen) surpassed all expectations because larger companies want cheaper, but still capable, AI models. He revealed that over the past month, every single inbound enterprise request was about optimizing spend. So, there’s some widespread concern about AI bills.
• Companies with cutting-edge AI bite the bullet with model routing. I talked with a CTO and a Head of Engineering at two cutting-edge tech companies. They also do not have an obvious return on investment (ROI) as yet. Still, they feel they have no choice but to pay the “intelligence premium” for state-of-the-art models or increase the number of bugs shipped. To reduce costs, both are considering “smart” model routing based on use case and prompt. These places pay top-of-market for the best engineers, so similarly, there are expectations of access to the best tools and models.
• DoorDash: More knowledge-sharing sessions and responsibility for devs. The leading food delivery company gives responsibility for spending to devs: everyone has a high monthly token usage limit. To exceed it, you need to justify why, and also share the plan for being more efficient next month. Many regular in-house knowledge-sharing sessions are about efficient AI use.
• Traditional company: monthly limits and dumb-model downgrades. One month ago, one of the largest retirement-savings companies in the US updated its AI usage policy for all devs, a current engineer told me, imposing a monthly GitHub Copilot token limit. Once gone, devs must use the less capable “0x” models on Copilot, which are not charged extra: GPT‑5 mini, GPT‑4.1, and Grok Code Fast 1.
• Startups: signing up for multiple Claude / Codex Max subscriptions. I talked with several smaller startups that are generating meaningful revenue, and don’t want to pay expensive API prices. So, they’ve made it a practice for devs to get subsidized Claude Code Max or Codex Max subscriptions.

There’s a new bottom-up focus on AI efficiency. Most tech companies do a variety of internal knowledge-sharing things like regular team demos, lunch-and-learn sessions, and engineering all-hands. I’ve been noticing more AI efficiency-focused sessions in the past couple of months, coming from engineers: no top-down mandate!

Engineering all-hands, CTOs, and even CEOs have started to raise concerns about increasing AI token costs, and now more engineers are experimenting with cheaper models for simpler tasks, model routing, more efficient token usage, etc.

I’d expect that during the next performance review and promotion cycles, engineers who helped save on token costs might be rewarded, like two years ago, when engineering teams were rewarded for saving on third-party vendor bills.

For an engineer, the best way to show impact in your work is to translate it to money: revenue generated, or costs saved. With AI spending as high as (or higher than) on observability, it should be straightforward to show massive savings with smart optimizations. There’s a touch of irony in how any savings – for which there might be promotions and pay rises – will come from the places that actually did the rocketing spending.

Read the full issue in the previous The Pulse. Or check out this week's The Pulse: Did Anthropic’s new model just boost rival Codex’s market share?

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from this past The Pulse issue. Full subscribers received the article below two weeks ago. If you’ve been forwarded this email, you can subscribe here.

On the evening of Thursday, 7 May, trading at Coinbase went offline and stayed that way for nearly 10 hours (!!). Customers could not buy, sell, deposit, receive, or withdraw. Basically, the core services of Coinbase were unavailable.

The outage coincided with a regional AWS outage. But no other company suffered a global outage; the most I observed was a few infra companies like Datadog noting that some regions had issues, and were failing over to a healthy region.

It’s weird that Coinbase – a $40B company! – told customers to monitor AWS’s status pages for recovery. This made it pretty clear that the company fully depends on a single AWS zone. Unusually, Coinbase deleted this information from its status page, but I got a screenshot first:

Coinbase later confirmed that it does indeed have a single-availability zone dependency. From its postmortem:

“Our matching engine was pinned to a single building. The Coinbase Exchange matching engine runs as a Raft-based replicated cluster inside an AWS Cluster Placement Group. We make this choice deliberately. A matching engine that meets the latency and throughput demands of a serious market cannot tolerate inter-zone network hops between voting cluster members. The physics of distributed consensus and the economics of running a fair, liquid order book point to the same answer, which is co-location.”

A quick recap on the difference between an availability zone (AZ) and region:

• Availability zone: One or more data centers (in the case of AWS, it is usually several data centers) located close enough to have low latency between them. Data centers in different AZs must be independently resilient. In the same AZ, there is no such requirement.
• Region: Within AWS, this consists of at least three isolated, physically separated AZs, usually 10-30 miles apart. It’s unlikely they’ll go down simultaneously, even in extreme circumstances.

Coinbase is saying that running from more than one availability zone (AZ) (building) would introduce too much latency to their product. This makes sense for low-latency activities like trading. But what about preparing for a failover as and when the AZ goes down? After all, an AZ is not guaranteed to have high uptime!

Turns out, Coinbase did not prepare for a failover for an AZ. Also from its postmortem (emphasis mine):

“We lacked an automated ability to fail over to another availability zone. When AWS terminated EC2 instances inside our placement group at 9:29 PM ET, three of five matching-engine nodes went down and we lost quorum. There was no automated cross-zone failover. Recovery required an emergency code change shipped during the incident to remove a startup assumption that all five cluster nodes were resolvable, the creation of a new node group outside the impaired placement group, and a careful sequence to restore a 3-of-5 quorum. This allowed us to reopen markets: first cancel-only, then auction mode, and finally full trading.”

Having no automated failovers is incredibly amateurish for an operation of Coinbase’s scale. Coinbase moves about 5.2 trillion dollars per year, and is valued at around $40B. The outage interrupted around $7 billion-worth of financial activity, based on my napkin math.

Back in 2016, Uber was valued at roughly as much as Coinbase, and handled circa $40-50B yearly. It had two data centers on the east and west coasts, and operated more as if it ran out of two zones. I worked at Uber at the time and there were regular failover drills to another data center (another region), in preparation should a region go down. Uber’s business, in terms of the financial figures, was a fraction of Coinbase’s!

My impression of Coinbase’s engineering culture has sunk after this incident, and it’s almost comical that CEO Brian Armstrong is boasting that non-technical teams now ship production code, thanks to AI. This feels like the wrong thing to focus on when Coinbase’s infrastructure basics seem to be in far worse shape in 2026 than Uber’s were a decade ago in 2016!

It seems Coinbase did not learn lessons after getting burned by previous regional AWS outages. In October 2025, the company suffered a three-hour-long global trading outage due to issues with AWS’s DynamoDB service. Following that outage, Coinbase engineering said (emphasis mine):

“To be better prepared in the future, we are exploring all options, including reviewing our regional deployment strategy to implement immediate and long-term fixes to reduce the impact of these types of outages.”

That process of reviewing the regional deployment strategy evidently missed or ignored the risk of a single-zone dependency of the heart of the business, with no cross-zone failover.

Read the full The Pulse issue.

In 2022, I wrote about the damning fall of events tech company Pollen. The short of it:

Pollen seemed to have pulled off the improbable feat of building a business in the notoriously low margin industry of events, surviving Covid-19, and building a solid software engineering organization. In April this year, the company announced it had raised another $150M in fresh funding.

But just three weeks later, Pollen laid off about 200 people, a third of staff. Leadership assured employees all was well. However, from that point on, things got worse. Leadership later pulled the plug on Slack, employees were not paid wages, pension contributions went missing, and vendors were not paid. Some vendors took matters into their own hands; on 9 August 2022, JIRA was suspended when Atlassian tired of the company’s failure to pay.

On 10 August 2022, Pollen went bankrupt, collapsing into administration.

The article looked bad on Pollen's founder, Callum Negus-Fancey. He was ultimately responsible for lying to staff, not paying salaries, the missing pension contributions, and the unpaid health insurance for US employees. The story was so bad that the BBC created a documentary titled Crashed: $800M Festival Fail. 

And then there was the $3.2M double charge for customers, manually initiated by CTO Bradley Wright, detailed extensively in the documentary Crashed: $800M Festival Fail. That double charge would have been trivial to reverse, but the reversal never happened, customers never got their money back, and the postmortem of the incident was never released to staff.

Four years later, Pollen and Callum Negus-Fancey are attempting to erase this shameful story from the public record. The article is my original writing, and thus I am the copyright holder of it. So imagine my surprise when I was notified that Google removed the article from its search results thanks to a copyright infringement claim it received:

It seems that anyone can file a bogus copyright claim to get an article they don't like removed from Google's search index. This happened in this case. I have no information on who filed the copyright claim. Even less so on who claims to be the copyright owner? Because I am the only possible copyright owner!

And Google has gone ahead and removed my article about Pollen's shameful collapse from its search results.

I have the option to appeal, which I have done so.

Google's copyright removal system is clearly being abused, to a comical degree. Someone doesn't like that I went into extreme detail about the events at Pollen - all of which are facts. And, for some reason, bogus copyright requests can be weaponized to remove information like this from Google's search index.

I managed to find the bogus DMCA complaint submission, after Google removed my site from search results. It is absolute BS: it claims that my original article is a copy of a The New York Post article. Which is absolute nonsense!

This "Ellie Piee" claimed that this 1998 article titled Band Leader Hits Winning Chord was copied by my article Inside Pollen’s Collapse: “$200M Raised” but Staff Unpaid - Exclusive. The two do not even share a single sentence!

The fake DMCA is made by a fake profile from a country with zero inhabitants. The removal requests by this "Ellie Piee" are made from the country called Bouvet Island, an uninhabited Norwegian dependent territory in the South Atlantic/Southern Ocean near Antarctica. It has zero inhabitants, and is referred to as the "world's most remote island."

Why does Google allow fraudulent DMCA notices to be filed with no penalty? My own speculation is that it is clear enough that either Pollen, or its former CEO Callum Negus-Fancey, or its cofounder and COO Liam Negus-Fancey or someone else related to the company hired reputation firms to remove Pollen articles from Google. This firm then files the most bogus requests under fake names supposedly residing in uninhabited regions of the world, and Google complies.

I never thought I would have to revisit the shameful history of Pollen, but someone at the company felt the need to prompt me to do so.

Lawsuits are still ongoing against Pollen, by the way. Now that someone from Pollen tried to erase the record of this story, I got a bit of renewed interest in what has happened since. In California, the lawsuit Tayler Ulmer vs Pollen is still in progress, summarized as:

• Tayler Ulmer and five other named former employees, on behalf of themselves and “all similarly situated employees” claim to have been laid off without paid wages and benefits, plus claiming possible fraud
• The filing says that Pollen executives Callum Negus‑Fancey, Liam Negus‑Fancey, and James Ellis are personally liable in this lawsuit
• The lawsuit wants to reclaim unpaid wages, unpaid severance, restoration of lost 401(k) contributions, and a uling that all the named entities and individuals are jointly liable, including successor entities, so employees can collect regardless of how Pollen shuffled assets and dissolved subsidiaries

I am wishing best of luck to the claimants - former Pollen employees - and we will see how the judge rules in this lawsuit. The more Pollen wants to silence me writing about this, the more I'll likely pay attention.

Pollen executives should have read what the Streinsand effect means!

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from a previous The Pulse issue. Full subscribers received the article below three weeks ago. If you’ve been forwarded this email, you can subscribe here.

Two weeks ago, I covered a trend of companies trying to reduce spending on AI within their engineering departments. While talking to my sources about this, one head of engineering at a larger company told me that they wished there was an ‘intelligent’ router that picks the right model for the right task.

The reason for such a wish is clear; prices for tokens vary greatly per model, and there can easily be a 10-20x difference between a cheap, average model, and a state-of-the-art one.

I did some digging into whether any solutions like this currently exist because the benefits look obvious, and what I found is listed below. Usual disclaimer: I have no affiliation with these vendors, and have not been paid to mention any of them!

Vendors:

• Factory Router: automatically selecting the right model per session, claiming 20-25% cost savings. More details.
• Not Diamond: auto-selection of coding models, claiming around 30% cost savings. Used by OpenRouter, under the hood. More details.
• Vercel AI gateway. Hundreds of AI models, smart routing and billing in one place. More details.
• Prism by Augment Code. Choosing the “best” model automatically for coding tasks. More details.
• Model Router by Morph. An API to suggest model selection for a prompt, based on a list of models. More details
• Weave router: a token router that works inside Codex, Claude Code and Cursor. “Hard” requests stay on frontier models, while “easy” ones go to open source ones. More details

AI gateways with routing built in. API gateways are popular ways to use LLMs in workplaces.

• OpenRouter: comes with “auto router” functionality where, after analyzing the prompt, the best one is selected. Uses Not Diamond under the hood. More details
• Kilo Gateway: route requests the model considered the best price-per-value. Supports using your own model keys, and using the service only as a router. More details
• Requestly.ai: automatically route requests to the right model based on cost, latency, and availability, and tons of configuration. More details
• LiteLLM: define routing rules that automatically select the best model, based on input content with the “auto routing” functionality. The setup is more manual, but you get more control than with many other AI gateways. More details
• Envoy AI Gateway: an open source gateway that offers some routing configuration, though it feels that the routing engine focuses more on availability, not cost optimization and smart model routing. More details

Cursor and GitHub Copilot also have an “Auto” model selection that does automatic model selection. For Cursor, it’s a fixed-price model where any savings made are for Cursor: they are not passed on to customers, but the model is cheaper than most others. For Copilot, the Auto mode results in intelligent model selection – but I’ve not heard much positive feedback about this mode from the few devs I asked about it. For Pro plans, Copilot supports pretty old models: GPT-5.5 and Opus 4.8 are not available. These are, however, available on the Pro+ and above plans.

Demand seems to be extremely high for intelligent routing. I asked Matan Grinberg, cofounder and CEO at Factory AI, who told me:

“Demand has been off the charts, especially from the enterprise [from large companies.] I’ve met with practically every bank CEO since we launched this offering, because they want a layer to control spend, while still generating high-quality code.

Pretty much everyone in tech is starting to see that open models are often sufficient. We’re seeing open model usage strictly increasing the last six months. My guess is that hosted open models are sufficient in performance for around 60% of coding-related work, in terms of token spend.”

It feels to me that “intelligent routing” will become table stakes, and so we can expect pretty much all AI vendors to build some version of it, and many new vendors to offer this kind of functionality.

If you know of any additional vendors not listed, you can add a comment on the original The Pulse article, and see more options there.

Read the full issue The Pulse that this excerpt was from, or check out all The Pulse issues.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics a past The Pulse issue. Full subscribers received the article below five weeks ago. If you’ve been forwarded this email, you can subscribe here.

Cursor has just released a new report based on two years of its aggregated usage data, and there are some interesting findings:

Power users generate 10x as many lines of code vs the median

The median dev using Cursor (the p50) generates about 700 lines of code per week with it, while for the 90th percentile, it’s closer to 9,000 lines.

Top 1% of users create incredible volume of code

The p99 data is pretty stunning:

The top 1% of users generate around 30-40K lines of code per week! That’s the equivalent of what ~45 “median” devs generate in the same period.

It’s worth asking how these top 1% of users are different. Are they writing a lot more greenfield code, do they have a bias for not using libraries, are they tokenmaxxing to get to the top of leaderboards? Do they generate 45x as many bugs, and importantly: are they adding a lot of business value with the software they ship?

Cursor consumes 10x more input tokens than it generates in output tokens

This is surprising: 90% of Cursor’s token usage is input tokens! This means that most of the tokens used are for reading the existing codebase and documentation. Outputting of code is a minority usage:

In some ways, this usage makes sense: as devs, we always spent far more time on reading the code, compared to lines of code we typed out. The “10:1 read-to-write” ratio is a classic. Here’s Robert. C. Martin (aka “uncle Bob”) sharing this observation in 2008, in his book, Clean Code:

“Indeed, the ratio of time spent reading versus writing is well over 10 to 1. We are constantly reading old code as part of the effort to write new code… [Therefore] making it easy to read, makes it easier to write.”

I find it amusing that we’re now seeing this 10:1 read / write ratio for token usage with AI agents!

Input tokens become the main AI token cost

Input tokens are priced at a fraction of output tokens: for example, Opus 4.7 charges 5x more for output tokens than for input tokens ($5 per 1 million input tokens and $25 per 1 million output tokens). Still, thanks to input tokens dominating token usage, Cursor is seeing input tokens account for closer to 70% of the cost of AI coding agents:

Without caching context, token cost would be 10x higher

Cursor does smart caching of context, to avoid re-generating old context with more new input tokens. When taking cache usage into account, Cursor only spends 0.6% of tokens on output tokens. The remaining 99% is split between cache read (90%), cache write (2.5%), and input tokens (7%):

I wonder if context reuse and caching will be a key AI efficiency component in the future? AI tokens are expensive to generate, so any form of reuse will make a lot of sense, especially in workflows like coding where a lot of existing context is reused.

Of course, Cursor sharing this detail also makes sense, as they remind everyone that building an efficient AI agent harness is far from trivial. Indeed, if you roll your own agent harness, you also need to put an efficient caching layer in place to match the efficiency of tools like Cursor.

Opus is the most expensive model & could hurt Anthropic

At the time of publishing, Opus 4.7 was still considered the most capable coding model. However, it’s also very expensive, and Cursor’s own data shows it’s close to 10x more expensive than its own Composer 2.5 model:

It’s significant that Cursor compares the cost of a single agent request; it’s not a direct token-to-token comparison. And it’s worth noting this benchmark is being shared by Cursor, which has an incentive for its Composer model to appear the lowest-cost.

Still, assuming you can get similar-enough results with a 10x cheaper model, it is a saving that’s hard to ignore, especially for mid-sized and above companies. I would not be surprised if more tech companies find ways for devs to use less capable – but cheaper – models for less critical work.

More expensive models result in higher acceptance rates

An interesting metric Cursor shares is cost-per-line-added, per model:

This metric is a more realistic cost because it correlates to output: “smart” models that are expensive, but which produce code that is frequently accepted, are penalized by the cost-per-agent-request metric, but they’re not here.

Indeed, Opus 4.7 has the same cost-per-line-accepted as GPT 5.5 at half the cost per agent request. In this comparison, Cursor’s Composer model is “only” 5x as efficient.

Missing from both lists are Google’s Gemini models, a strange omission by Cursor. I reached out to Cursor and they told me that Gemini was left out simply because they see very little usage of this model on their platform, similar to the sparsely used Grok model.

Almost half of AI changes accepted without manual review by devs

I’ve left the most interesting part of this report to last: in just a month, among devs using Cursor, it has gone from 10% who let AI agents create commits without a manual step, to around 40% of devs who no longer personally check the code:

The jump correlates with Opus 4.7 and GPT-5.5 being released, and around the time when many devs seem to have concluded that writing code by hand is dying after experiencing this generation of models’ capability at generating code.

Check out the full report from Cursor for more details. Thanks to the team for releasing this data!

Read the full issue of The Pulse this excerpt is from, or check out the latest The Pulse from today. Today's issue covers:

• Bun’s Rust rewrite with Fable: what can we learn?
• Anthropic’s Fable, OpenAI’s GPT-5.6 Sol, Cursor’s Grok 4.5, Meta’s Muse
• North Korean hackers keep trying to infiltrate full-remote companies
• Industry Pulse: Meta’s key logging exposed sensitive data, massive cuts at Xbox, Meta could not buy enough AI capacity from Google, Qualcomm acquires Modular, and memory price hikes hit Apple products.

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics of last week's The Pulse issue. Full subscribers received the article below seven days ago. If you’ve been forwarded this email, you can subscribe here.

Last week in San Francisco, I met Jarred Sumner, creator of JavaScript runtime, Bun, and was keen to learn more about the rewrite of Bun from Zig to Rust. But at the time, Jarred didn’t want to say too much, as the tool used for the migration, Fable, was out of action due to the US government imposing export controls. 

Jarred and I at Anthropic’s HQ, last week

Fortunately, the situation is now resolved and Fable is available globally, and Jarred has published a detailed post about the project. Before we get into the migration, some context:

Bun is a complex project, with lots of production software depending on it. Bun itself does many things:

• JavaScript, TypeScript and CSS transpiling, minifying and bundling
• A test runner
• A package manager (npm-compatible)
• Other things: module resolution, a WebSocket client, Node.js implementations and many modules

Today, Bun has 22 million monthly downloads, and software like Claude Code and OpenCode depend on it, while hosting providers like Vercel, Railway and DigitalOcean do first-party support for Bun.

Why a rewrite? 

Zig is not a memory safe language, and memory-related bugs occurred continuously. Jarred lists memory-related bugs in the latest version of Bun: memory leaks, crashes due to memory issues, heap-out-of-bounds writes, and so on. This was after the Bun team patched the Zig compiler to reduce memory-related issues, and put end-to-end memory leak tests in place. As Jarred says:

“Our bugfix list felt bad and I was tired of going to sleep worrying about crashes in Bun. I don't blame Zig for that - other users of Zig don't have the bugs we had, and mixing GC with manually-managed memory is an uncommon enough thing for software to need that no language really designs for it. (...)

For Bun, correctly handling the lifetimes of garbage-collected values and manually-managed values has been a major source of stability issues - most often small memory leaks and occasionally crashes. Every memory allocation has to be meticulously reviewed. Where do these bytes get freed? How do we ensure it only gets freed once? Did we check for JavaScript exceptions properly? Is this garbage-collected pointer visible to the conservative stack scanner? Is this garbage collected memory or manually managed memory?”

Moving to a memory-safe, yet performant language could eliminate such errors, and Rust is one such language that fitted the bill. Jarred:

“A large percentage of bugs from that list are use-after-free, double-free, and "forgot to free" in an error path. In safe Rust, these are compiler errors and RAII-like automatic cleanup with Drop. Compiler errors are a better feedback loop than a style guide.”

However, doing a full rewrite on Rust has always been a terrible idea. Or at least, it used to be, because of how unbearably long it would have taken:

There are two problems with rewrites: they take too long, and they take waaaay too long. A dev who has done rewrites probably knows how things tend to go:

1. Make an educated guess about how long it will take; say, nine months.
2. Nine months later, there’s still another ~6 months to go because new functionality is added to the original codebase, and now that new functionality needs to be added in!
3. By 15 months in, there’s still months left to go for the same reason!
4. In the end, you manage to mandate a “feature freeze” for two months and finish the rewrite in ~18 months, if lucky. The original nine-month estimate can end up taking 2+ years.

Jarred likened rewriting Bun in Zig to this:

“Historically, rewrites are a terrible idea. Excluding comments, Bun is 535,496 lines of Zig. 

A rewrite in another language would take a small team of engineers a full year.

A year of zero user-facing impact is not a realistic option we could consider. So, enforcement through code-style to fix stability issues was our best bet, and was our plan when we added Rust-inspired smart pointers to Bun's codebase.

But honestly, I didn't want to do it. Homegrown smart pointers offer worse ergonomics than Rust, with none of the guarantees.

What if, instead, I spend a week testing if Anthropic's new model [Fable] can rewrite Bun in Rust?”

Rewriting Bun with Fable

Unsurprisingly, the rewrite was not as simple as typing a prompt like: “Claude, rewrite Bun in Rust. Make zero mistakes.” Instead, this is how Jarred did it:

Step #1: Prep work. Three hours of intense prep work with Claude, explained Jarred:

“Before writing any code, I spent about 3 hours talking to Claude about how to map patterns from our Zig codebase closely to Rust. Claude serialized this discussion into a PORTING.md document, which ended up on Hacker News [as the Zig → Rust porting guide]”

This guide is a 600-line file with instructions like:

Ground rules:

• No tokio, rayon, hyper, async-trait, futures. No std::fs, std::net, std::process. Bun owns its event loop and syscalls. (Rust core/std slice, iter, mem, fmt, and core::ffi are fine — only the I/O-touching modules are banned.)
• No async fn. Everything is callbacks + state machines, same as the Zig.
• Borrow-checker reshaping is allowed. When matching Zig flow yields overlapping &mut, capture the needed scalar (.len(), index) into a local, drop the borrow, then re-borrow. Do NOT reach for raw pointers just to silence borrowck; leave // PORT NOTE: reshaped for borrowck so Phase B diff readers aren't confused.

It’s a series of instructions that makes sense to someone who’s expert in Rust. If you want to learn more, we cover Rust basics and why Rust is different, with Alice Ryhl.

Step #2: Trial run + adversarial review. Asking Claude to rewrite three files out of 1,448 total number of files. After the rewrite, Jarred ran two separate adversarial reviews with Claude to critique the result, in separate sessions than the one that Claude made the changes in.

Step #3: split up the work across 64 AI agents. Jarred split up the job so that agents worked on files independent from one another, in parallel.

Step #4: iron out issues with the run (~1 day). When Jarred attempted to run all this, agents kept getting in each other’s way:

“I asked Claude to loop the workflow on all 1,448 .zig files, and about 2 minutes in, one Claude ran git stash before committing. Another ran git stash pop. And then git reset HEAD --hard. They were stepping on each other! And if I put each Claude into a separate worktree, I would run out of disk space because Bun's git repository is too big and eventually the changes will need to be compiled and seen together.

So, I asked Claude to edit the workflow to instruct Claude to never run git stash or git reset or any git command that doesn't commit a specific file at once. No cargo either. No slow commands at all.

Then, Claude resumed the workflows. And it was working! Too slowly, so I split it into just 4 workflow shards each with their own worktree (4 worktrees total), each running 16 Claudes committing and pushing files.”

Step #5: have it run and wait ~2 days. The parallel agents went to work, and completed the rewrite of 535,496 lines of Zig code over the course of two days. Each commit was checked by two adversarial reviews, before being committed.

Step #7: fix ~1,600 compiler errors (~12 hours). The rewrite was completed, but nothing compiled. Going crate-by-crate (‘crate’ is Rust’s concept of a top-level compilation unit), Jarred had Claude fix compiler errors. This alone would be a herculean task for an engineer, but not for Claude:

“Fixing the cyclical dependencies revealed about 16,000 compiler errors. A massive number for 1 human, but not a crazy number for 64 Claude’s at once.

To maximize parallelism, the workflow looped over each crate.

• For each crate, run cargo check, group the output by file and save the errors to a file
• Fix all the compiler errors within that crate
• 2 adversarial reviewers for the crate's changes
• 1 fixer applies the fixes”

Visualizing fixing of errors, one by one, done by the agents. Source: Anthropic

The enjoyable thing about this phase of the migration was that the agents ran from midnight until 11:30am, fixing compiler bugs on their own – when Jarred and the team were getting some sleep.

Step #8: run tests locally (~2 days). Bun has a large test suite. The next step was to get these tests to run without compilation errors. 

Step #9: get the test suite to pass CI (~3 days). Once the tests were running (and failing), the next step was to fix the code, so that the tests could pass. This took two days.

Step #10: Done in 11 days! After all the tests passed and Jarred verified that everything worked as expected, he merged the changes. The whole process took 11 days, from planning to the finish.

The rewrite: porting ~550K lines of code, in 6,500 commits, over 11 days, with 64 agents

How repeatable is this process?

The rewrite cost a whopping $165,000 with API pricing. With Fable’s API prices, the rewrite consumed 5.9 billion uncached input tokens, 690 million output tokens, and 72 billion cached input token reads. Anthropic sells API tokens at a margin as its business, so the cost of the rewrite for it was lower. It’s a large amount: the equivalent of the annual base salary for a software engineer at a mid-tier company in the US!

But then again, could have an engineer done all this work in a year? Probably not, and Mitchell Hashimoto says the same:

“On the cost, I think $165,000 at API pricing for Fable (didn’t verify) is an incredible deal. There’s absolutely no way an engineer with that salary would’ve been able to achieve the milestones Claude did in 11 days. No way. (Even if you break it down to N engineers paid $165K total in 11 days it doesn’t math out)

This does, however, also reconfirm my own biases which is that Fable in particular is most excellent at hard, focused tasks with clear reward functions. I’ve been tweeting about this recently.”

What if AI enables rewrites and migrations that wouldn’t have been considered before? The idea of rewriting Bun in Rust without AI was impractical, admits Jarred:

“By hand, I think this would've taken three engineers with full context on the codebase about a year, during which time we wouldn't be able to improve Node.js compatibility, fix bugs, fix security issues or implement new features. We never would've done that. The realistic alternative was to do nothing and keep fixing the bugs at the top of this post forever.”

A rewrite or migration taking months or years is why so many of these projects never happen. Let’s take aside the cost for a minute and consider this question: if AI can shorten a one-year rewrite to a week: would you do it? 

If the answer is “hell, yes:” a blueprint now exists in the form of the Bun migration on how to do it. There are some caveats not detailed in the post, though:

1. You need an engineer who is very motivated and knows the codebase very well
2. You need an extremely robust test suite, so when the test suite passes, you know it works
3. You need to be willing to invest a lot in tokens, not knowing how well it all will work

In fairness, #3 is the weakest point because we know LLMs are pretty good at “mundane” work like code migrations. With a good test suite (#2) and a motivated engineer to iron out things (#1), you’ll more likely than not succeed.

The remaining question is how much can be spent. It will likely not be $165K: and costs can be reduced with a simpler project, or by being thoughtful about model usage. For example, do high-level planning with the most expensive model, and cheaper ones for coding and review tasks.

Migrations with AI are surely speeding up, but only when projects are well-engineered like Bun’s has been.

Read the full issue of The Pulse this excerpt is from, or check out the latest The Pulse from today. Today’s issue covers:

1. Grok’s CLI uploaded all your local files to the cloud, then got caught.
2. New trend: concern about massive increase in code review load.
3. Are more devs at enterprises upset about enterprise pricing by AI labs – and does it matter?
4. Linux creator: AI “clearly useful.”

Read the full issue here