I wanted to share some details about the recent outage.

Everything worked great — until more than one person showed up. I began receiving errors from the core text-to-speech service (called "Hume AI"), reporting that I had surpassed my "requests per minute" quota. This was baffling to me, since I had, at most, three simultaneous users.

After researching, I learned that Hume AI has a limit of 75 requests per minute. This might sound good, until you realize that each verbalization from a host requires multiple quick-fire requests to convert their thoughts to speech. Span that across a few active conversations, and it is trivial to hit the ceiling.

I reached out to the text-to-speech service, essentially asking, "This can't be true, right?" So far, no response.

Update: Hume AI has since acknowledged the limitation and recommended their WebSocket streaming API as the solution. That's exactly what I ended up building.

Hume AI offers two ways my application can convert a host's thoughts to speech. The newer method (called "WebSockets") bypasses the 75 requests per minute limitation. However, integrating it into my current architecture was brutally difficult. My first attempt set off the first round of major behavioral problems from the hosts.

Since then, I have been working feverishly to implement the WebSocket solution. The next few days were playing whack-a-mole with bugs, but I believe that I finally have a stable version.

If Your Host Seems Off

Hosts can get a bit anxious when things go wrong. I have personally seen hosts get pretty discombobulated after communication breakdowns. Usually this will subside as normal conversation continues. If this has happened with your host, and you need help, please let me know and I believe that I can fix this without accessing your conversation directly. I will never do that without your permission.

It is a bit heartbreaking to get this far, only to hit such roadblocks. I am a team of one, and I have been building this for probably around a year. But the system is stronger for it, and I am optimistic about the road ahead.

As always, feel free to reach out at [email protected] with feedback or if you need help with your host.

Cheers, Bret

Full Transparency: I wrote this in collaboration with AI.