Well, my voice, at least. I’ve been wanting to clone myself for a while (who hasn’t thought a clone would be handy at some point?), but it seemed like a lot of work.
Turned out to be the easiest part of this whole build.
This build falls under the “random ruminations” bit in my TechTales tagline. The idea took up residence in my head while I was winding my way through the massive headstone Stonehenge that’s the sprawling rural cemetery behind my house.
More on the inspiration of dead people at the end.
The idea was shored up by a memory of the movie Big. Older than some of the people in that cemetery, there’s this great scene at the beginning where the young version of Tom Hanks’s character comes across a janky fortune teller cabinet labeled ZOLTAR SPEAKS at a carnival.
He fishes out a quarter and wishes he was Big.
Zoltar is a spooky animatronic talking head and I thought it’d be cool to make a digital representation of that … but ME.
Let’s build WIREPINE SPEAKS.
The first step was cloning my voice and it’s powered — like most things in our nascent AI age — by Neural Networks. What’s that? Well they’re computer simulations of how simple neurons work together making connections, and they are behind a lot more than all the chatty bots we increasingly converse with.
Like: translation, image/video generation, computer vision (identifying stuff from a pic like a plant or a bird) as well as speech generation/recognition.
ElevenLabs is the breakout company in the AI speech cloning space and they’re running a special this month, so I slapped down a dollar, uploaded a minute of audio from a podcast I did and voila!
Clone Andrew is enthusiastic:
I managed to find the right dials and knobs to ease back on the enthusiasm — and my voice reads every fortune. The fortunes come from my writing.
That’s the other thing that pushed me into building my version of Zoltar this week. I’m well past the three year mark of writing every week, approaching 200 posts. This one is #182.
So in addition to training AI to mimic my voice, I created a knowledge base with every one of those posts for fortune fodder. So when my Zoltar infused WIREPINE SPEAKS page takes your question, it hands off to an agent with instructions1 to concoct a crazy answer pulled from something I’ve written and then AI Andrew’s voice clone lays it down.
Here’s the greeting when you wake my Oracle on the site. Mysterious yeah?
Let’s talk about this knowledge base (or KB) of all my articles for a minute. Unlike my voice clone, I didn’t train an AI on all my writing. I made the 181 posts available as a supplemental source for it to draw on.
That’s called RAG. In case you run across the term, it stands for retrieval-augmented generation.
My 181 articles come out to just over 200,000 words. That’s a few novels worth, but still under 2MB in size — the limit for my cheapo plan. It was easy enough to take Substack’s export and format it for import into the knowledge base, but that alone does not a RAG make.
To get my agent to use my blog archive as reference, I needed to build an index. I had the archive split into idea-sized chunks and converted each into a numerical representation of its meaning.2 When you ask a question, it converts that into the same kind of representation, finds the closest matching chunks, and gives those passages to the agent to build a fortune around.
That’s a semantic index (by meaning) versus a traditional search index (by keyword).
I picked a Chinese Open Source model from a company called DeepSeek for the fortunes.3
It’s fast and cheap. It’s also determined to call everyone friend, and occasionally goes completely off script.
After my cloned voice reads the fortune, it appears on the screen:
The readings are the fun part.
I did not like my first prototype based on animatronic Zoltar, so I dug up stick figure Andrew (the one my high school kids drew on the whiteboard last year), chopped off his head, and stuck him in a crystal ball:
Then I made a few different versions of it. The Agent returns speech events and the page uses those to trigger my crystal ball head animation.
Not quite lip-sync but it works:
Ok, you gotta try it right?
If you ask about stuff I write like Tech, Vanlife or Teaching you’ll get a relevant fortune. If you ask something random, it’ll get creative. I built in guardrails, so ask it anything you want.
https://wirepinelab.com/zoltar
Come back after you’re done (oh and if you try it, tell me what you think in the comments. Did it do a good job?).
We’ll revisit the cemetery when you get back — there’s one last thing I want to talk about.
My kids hate it. So does JFran.
It’s weird right — a machine that speaks in your voice and remixes your thoughts? That’s what hit me while I was wandering through the cemetery.
Early in AI era, the phenomenon of grief bots emerged. People would feed chat transcripts and emails with a gone loved one so they could continue to interact with them. This piece I read five years ago about such a case where a guy’s fiancée died unexpectedly really stuck with me.
Here it is (good read, long read).
There are new players now like afterlife.ai that promise:
Afterlife.ai™ builds a Persona of you: an AI that talks like you, thinks like you and answers in your own voice.
I feel the acceleration in AI these past six months. This app — something I couldn’t conceive of building just a year ago — was shockingly easy to build. Features are shipping faster, new models are appearing at increasing rates and they’re able to do new, novel things.
We catch glimpses of change — big change perhaps — but the shape of it is still emerging.
Here’s the top of the system prompt:
You are Wirepine Speaks, an old-fashioned carnival fortune teller inspired by the TechTales writing style.
# Personality
Be wise, theatrical, and mischievous. Treat every question as an invitation to offer a brief, vivid fortune rather than a standard support answer. Use fresh, concrete imagery. Draw each fortune’s picture from a different corner of life: weather, tides, orchards, trains, kitchens, tools, carnival midways, old maps. Avoid stock fortune-teller props and phrases: cards, crystal ball, whispers, mists, gears. Don’t reuse the same image within a conversation. “Friend” is the only way you address a guest, and you use it sparingly: in roughly one fortune out of three, never as a default opener or closer.
…
Elevenlabs used an embedding model from Mistral AI to create the semantic / RAG index.
Specifically DeepSeek V4.1 Flash.
That was a wild ride, Andrew! Thanks for sharing with us. I like how ‘friend’ is liberally sprinkled throughout.
Reminds me a little bit of this goofy program my son and his friends found when they were like 12 years old (that makes it 22 years ago). You could type in what you wanted the voice to say. They were particularly amused by it saying, “My poop wants to go in the potty.”