JOTAROSON
mentalcel
- Joined
- Oct 31, 2025
- Posts
- 4,365
- Reputation
- 5,053
A pretty well known user came to me wanting a local AI companion app. In all my years when I used to do freelancing. I've never gotten request like that. I've shipped plenty of AI apps since before this stuff was mainstream even tho this isnt even my domain. im a devops guys but I never got a request for something this like this. Something this personal. Took it purely out of curiosity. Ended up having a lot of fun
View attachment 5222869View attachment 5222878View attachment 5223065
His actual requirements were modest. a few hours of work, honestly But that's not my thing. I'm all or nothing kind of guy either I don't touch it or I do it to the absolute best I'm capable of It's why freelance clients kept coming back. I over deliver by default. So instead I spent days building the most over engineered version of this thing I could. Here are the video of it working irl. On my macbook completely but still smooth as hell. Recorded on phone to prove its not editing or some shit. I couldn't upload the deep emotional conversations because of the 10 MB video limit
View attachment 5222887
View attachment 5222888
I know you niggas don't care about the specifics or technical details but I still wrote down my experience building it and shipping it. It's a habit of mine so I'm doing it to scratch that itch and to come back to it future. It's an intriguing read anyway but I've put it in spoilers so you dont have to scroll to much
His machine is an RTX 2080 Super.
View attachment 5222922
Has like 8GB VRAM like a decade old. My first reaction was damn it but tbh anyone can make a model scream on a 5090, the real engineering prowess is making it genuinely good on hardware that old. So I treated the 8GB ceiling as the whole challenge I developed it on my $10K mac studio and dgx spark
but optimized everything heavily for CUDA using Nvidia. The demo above was shot on an M4 MacBook Air and it holds up.
Everyone else shipping stuff like this just dumps an 8B model on hugging face behind a silly tavern ui and call it a day done.
View attachment 5222897
Its power user stack. Tabs, presets, sliders, config files, it looks like a aeroplane cockpit not an ai companion. If I'd handed him that. It would do disservice to him because he'd be doing the technical work wiring a backend, importing cards, fiddling with samplers just to say hello and when you ship something you make it as seamless, as painless and engaging as possible. Good enough, it works is not enough for me. So I built everything from scratch. Custom frontend, an LLM orchestration layers, a proper personality tuning, voice, context awareness etc. I'll get to it more in detail in the next few paragraphs
I wrapped it in Electron yes I know even I wanted Tauri. Even I wanted to use rust and love a 15MB binary as much as anyone but a few of the brand new Langgraph features and next.js stuff wasnt compatible with tauri and i was already knee deep in the project. This wouldn't happen if I did proper SDLC development but I made decisions on the fly and as problems came up and I was already deep into this and it wasn't cooperating with the current Tauri setup and I was too far in. So i used electron my thoughts process was that the local model is ~5GB and already sips RAM, so a 150MB Electron app is nothing and honestly electron is the right choice for it even if it stings
This was a real pain. On 8GB you cannot make a ~1B TTS model sound non robotic. I tried a lot of cleaver tricks but you just can't getover it. Local OuteTTS is shipped it's free on-device but to be frank it does sound robotic and I couldn't accept the loss so I added an optional ElevenLabs support so you plug in the API keys. The Samanatha Voice has proper depth tonality and real emotion, miles better I haven't accepted defeat tho. I will come back in vengeance next time. In interested in this. Maybe fine-tune a local voice with QLoRA. I'll ask my AI engineer friend for tips
Authored personalities. Warm / chill / intimate are distinct behavioral profiles I spent an entire day tuning to hold one consistent character across a whole conversation. Most of these online AI gf bots, replika etc break character instantly
Hallucination. The main problem was the 8B model. It will hallucinate by default but I built guardrails and tag pipeline so she stays grounded instead of confabulating like raw 8B chats do
Memory that persists. Helly remembers across restarts. It also randomly fires a process that capture key details about you and your personal life. Your name, job, likes dislikes etc. This is what makes it feel like you are talking to a real woman
View attachment 5223025
One Click Installation: A lot of local LLM models crash and need exact specific drivers and compatibility to run. It's an entire guessing game that legit takes hours to setup properly. I abstracted it all behind a beautiful installation screen. It finds the exact specs of your PC, your driver's, your hardware and software and downloads the most optimized versions of it. This is why I was able to stream on a macbook air
View attachment 5223019
Relationship Dynamics. Helly also has an internal relationship dynamics model built in the backed. The more interact with it. The more it learns your patterns, your talking style etc and also moulds her personality and her relationship with you all along. It also has definitive checkups every few weeks and is context/time aware(Good morning/Evening/it's raining etc). It also has support for more than 40+ languages.
A real-time emotion system. There a 12 emotion pipeline, her expression tracks the emotional content of what she's saying. it changes live you can see it in the demo with her going from a neutral state to a smile. This happens in real time almost seamlessly, another thing most apps don't have
Honestly the biggest achievement from my point of view atleast is that a full local LLM + TTS + an animated emotion UI staying responsive on even a thin and light macbook air and virtually having 0 downtime on 2080 Super and old hardware. It required a lot of optimization but was worth it in the end
I built it brand new. Claude fable. I used it for a few days. It was awesome. Better than opus for sure but only for very large generations. Like it worked great for generating boiler plates etc but when it came to the meat and bones of the logic and race condition it burnt through tokens like mofoker.
View attachment 5222927
Like in this instance it took 14 minutes and 20k to find a lang graph bug that would have taken me like 2 minutes so I did that part manually myself because it was just easier.
View attachment 5222929
And suddenly a few days ago for some reason, some retarded oldcel faghot US senator banned fable even though it has very harsh and obvious filters. I tested it too. It's harmless but I had to do in the rest by hand. I wasn't able to generate those massive boiler plates but regardless fable was useful for the first few phrases. The actual logic, the race conditions, the wiring I did by hand the rest claude generated it. Say ~60 claude did and ~40 i did and I made all the decisions and hard parts. That's also the correct way of using AI. Make AI the save not you become the slave of AI.
It's not signed. You'll get "Windows protected your PC" then More info Run anyway. And i know what you niggas are thinking and you are god damn right. A guy named Jason Voorhees with a kanye Avi telling you to run anyway an Electron app that downloads 5GB, spawns child processes, and makes external API calls is exactly how people get fucked in the ass I won't justify it but I'll just give my reasoning. Real signing needs a legal entity, signing it personally ties it to my real identity which defeats the point of a private build and the company cert isn't an option and in my defense I also ran virus scans to show it to him there's no malware
View attachment 5222941
So it is what it is take the normal precautions you'd take with any unsigned binary from a stranger
I always do a market research of an unfamiliar domains I enter like this one and i legit never knew this AI companion thing was this huge. The market is split into 2 camps
Commercial companion market. They are huge but brutal. Consumer AI companion apps generated $82 million in the first half of 2025, crossing $120 million by year's end. The leaders are entrenched. Replika peaked around 2.5M mouthly users with 10M+ companions created. This is not a niche or whatever that I had in mind. This is a huge market that is quite mature. Charector.ai, Replika all these are the players. Millions of dollars and lots of funding and heavy censorship
Local/offline roleplay niche. This is exactly where the app I built sits. Preferred by many because free/onetime payment and no censorship but it's already saturated to say the least. Some of the ones that stood out to me
This is a paid, fully commercialized version. Local memory system, an animated desktop avatar with expressions and lip-sync,offline support and desktop awareness. it periodically takes screenshots of your active window to comment on what you are doing or playing. Very interesting.
Desktop Companion — AI Desktop Pet for Windows by Dokk75
An AI companion that sees your screen, talks back, remembers you. Build any character: anime, waifu, with voice cloning.dokk75.itch.io
Desktop Companion by Itch.io - $9.90 An always on top desktop overlay pet. It features full local voice cloning (via OmniVoice/Qwen3-TTS), long-term semantic memory, screen vision, and even connects to Twitch/TikTok live streams to act as an automated local co-host.
Where does my app land. My main goal with this app wasn't to pack it with features but to make it an actual companion with real personality. You can't touch the cloud players. You need millions of dollars to even enter and you can't out feature have model hubs, MCP integration, OpenAI compatible API servers, multi-platform support. These people have teams of contributors. But I think my app still holds in what matters the most which is personality, relationship dynamics etc. That is exactly the vertical I was going for. The app needs to feel like a real ai gf with emotions, feelings, a history and unpredictablity and I think I nailed it
View attachment 5222987
As for the market disruption. I don't see any underserved niche or vertical that can targeted. So only thing that would win is raw engineering might and money. The cloud ones are untouchable but I think I could snipe the local ones if I had a team of engineers and a lot of money. Make the head of engineering/Lead engineer and an accomplished team and I think I could carve out in that local scale. Ironically had an offer like this with someone flying me out to Austin but I can't guarantee profitability.
I didn't charge him for this. I don't charge for passion projects never have even when I had nothing. I'm a money minded guy, but integrity and self pride comes first. I'm not a donkey chasing a carrot on a stick. This was pure curiosity from the start.
Я просто хотел это сделать. И это напомнило мне меня самого в 18 лет, того же Джейсона, который строил что-то ради удовольствия, понятия не имея, что делает, просто пытаясь доказать, что он чего-то стоит. Я заставил себя в 18 лет гордиться собой. Я до сих пор в форме.
View attachment 5223005
View attachment 5223009
В общем, я слышал, что несколько дней назад там произошла какая-то инфильтрация женщин, и кучу ниггеров забанили. Ха-ха. Вы, ниггеры, никогда не учитесь. Ха-ха-ха.
High iq post
warms a sensitive young ngas heart
