Technical Debt and Realistic Expectations for AI

I ran into an interesting use case the other day and it made me stop and think about the current rush to AI and how utopian ships often run aground on the reefs of reality.

The customer was pretty typical: a long lasting Google Workspace account that held multiple custom domains used for gmail and drive. The user had started as an individual who needed hosting for business emails, been nudged into workspace for custom domain support, and then expanded the domains and services used as they worked their business up through a dba to an LLC.

The challenge came when they tried to set up a phone number using Google Voice. Workspace customers cannot set up Google Voice for Workspace users if the Workspace account was originally set up as an individual account: for reasons, the Workspace account must be of a commercial account type in order for a Workspace user to use the commercial version of Google Voice.

No problem they thought: I'll just change that online, drop the LLC registration into the Workspace account details, I'll be all set. ChatGPT even told them to do this to resolve the issue.

Predictably, this was the outcome:

Any moderately technical person will tell you that, while this is admittedly not very customer-friendly, it is also not very surprising. Workspace is used all over the world. Legal registration confirmation flows are a land of corner cases. Working billing systems are incredibly complex and almost always very brittle. If you have something that is working, best to leave it alone.

BUT at the same time, there is a tell in the fact that you can pay someone a non-zero amount of money to make this problem go away. There is a workflow, that people have been trained to accomplish, to log in to the back end and migrate the account and its data from one state to the other. It hasn't been automated for you, but you can pay humans to do it.

This is the sort of thing that is the core promise of AI. Remove bureaucracy and automate the complex workflows that require judgement and experience and skill. Eliminate the need for human intervention, relying instead on systems that understand the architecture and code of the system you are trying to live and expand within. Make it cost less, happen faster, and don't require phone calls. Above all, no phone calls. Ever.

That promise is clearly not going to be realized here. Instead of automated splendor the customer got an education packet on the back end architecture and an invitation to solve their own problem by finding someone to pay to make it go away.

Its unlikely this will change any time soon. These systems are beyond baked, and the idea of letting an unsupervised AI loose to make changes in the database of a $Billions payments system is going to seem unacceptably aggressive for a very long time.

This is the point of this essay. AI might put humanity on a path toward untold riches, breakthrough science, astounding medicine. One wonders if AI will ever be able to directly migrate a customer in the currently existing Google Workplace offer from one account type to another. Technical debt makes simple-to-imagine things like this incredibly hard to actually do, and monetary incentives disallow the sorts of risky ideas, like using an AI, that might make it easier. It is entirely possible that AI will solve cancer before it is allowed to take on this trivial task.

In the meantime the customer will be getting their online voice service elsewhere. On the reefs of the way things actually are, change happens slowly indeed.