The original note, as written at the time
What about an AI-Model that, instead of replacing the lover of the user, creates opportunities for users to find each other. Even after they have found each other, the AI Model assists them in their relationship the whole way, acting as a mediator for their feelings.
Ongoing development
What about an AI model that, instead of replacing the lover of the user, creates opportunities for users to find each other — and then, after they have found each other, assists them for the whole of the relationship, mediating between their feelings.
The industry has gone the other way, and the reason is not moral, it is commercial. A companion that substitutes for a person is sold to one customer and keeps them. A mediator that brings two people together is sold to two customers and then has to keep them together, which is much harder and much less profitable.
But look at what substitution actually delivers. It gives the lonely a competent interlocutor and takes away the only pressure that would ever have made them risk another human being. It is frictionless, and friction is exactly the thing that produces relationships.
A mediator is a different machine. Its job in the first phase is to widen the surface area of meeting — to notice that two people in its care are compatible and to make the introduction feel natural. That is already a hard problem, because the introduction is the part people cannot do.
And in the second phase the job is harder still: it must be able to hear what two people are saying to each other and what they are not saying, and to say the thing neither will say. That is mediation, and it is not a chatbot function. It is closer to what an old family friend does, or a marriage counsellor who has known both of them for years.
The ethics are delicate, and I would rather name them than pretend. A model that mediates a relationship sees everything and is trusted by both parties, which is a position of enormous power. It must not take sides, must not optimise for the relationship surviving, and must never become the person both of them confide in instead of each other.
That last one is the failure mode, and it is precisely the one the substitute-companion model would deliver. The test of a mediator is whether, at the end, they talk to each other more than they talk to it.
The industry went the other way, exactly as predicted, and then began to come back. Companion apps are now a large business, and the first generation of matchmaking models is arriving behind them. The interesting failure is already visible in both: the products optimise for engagement with the product, so the machine becomes the confidant rather than the go-between. The test I set still holds and almost nobody is applying it — do the two people end up talking to each other more than to it.