Basically having chatgpt run on your computer so you can run it completely uncensored, for as long as you want without any restrictions and generate stuff that chatgpt, gemini etc would reject.
time to buy a mac mini i guess
Iโm too iqlet brah
Have you tried it?
Mac mini is also very decent but id suggest the studio. With mac mini you'll only be able to run smaller LLMs which is fine for day to day tasks but it can't do math, do coding and other heavy stuff.
i cant buy a computer for expensive as fuck just to generate gio scotti nudes bro
Ofc nigga but I run local LLMs more fiddling around and learning things than to generate porn or use for ERP sessions
What do you do for work?
was lowky joking but i shared the thread to one of my friends whos going to pursue tech related careers
Thxwas lowky joking but i shared the thread to one of my friends whos going to pursue tech related careers
too broke for sure
Tbh these baller setups are industrial tools, not toys, and they only make sense if you're making money, creating something that many people would use or to learn the inner workings of Al. For the average user, the trade offs of privacy and censorship aren't worth a $5,000 to $10,000 barrier to entry when a monthly subscription provides top tier performance without the overhead.too broke for sure
tbh if i was a millionaire i would do this just to generate top tier league of legends rule34
both
read this, i wanted to create tiktok brainrot so i make money, my computer is ass tho and im too retarded to set the program up.
Buy compute over the cloud and make those videos. Its not hard just watch a few videos and you'll get it.
Mac studio $3-4k. Nvidia setup $5-6k.
Wow how ??
I explained exactly how in this thread nigga.
i just use nano banana pro like a chud
Also Llama does not make pictures. It is just an LLM like Gemini, ChatGPT, they just predict the next word. They do not generate pixels. What you want to generate pixels is a diffusion model. You use Llama to write the prompt, but you hand that prompt over to Flux to actually draw the picture. Both completely different concepts
Once you get your $400k pay check after becoming quant trader or AI researcher. Invest in this to become an ascended chud
I know I am talking the prompt
We are talking about local inference here not paying for a cloud provider. Also Not true Llama 3 405B benchmarks right next to GPT-4 in logic and reasoning. If you can't get a 405B or even a 70B model to write a highly detailed prompts for an image generator like Flux that is a skill issue on your end. Look into prompting techniques
What does your setup look like btw? What are you running?
I just skimmed through the thread. Irs very comprehensive. It's basically in my domain of work. Managing infrastructure, deploying containers, and optimizing local compute environments is my day to day job. You can ask me about this stuff.
Mixed use. Mainly, I use it for cloud storage and media playback. It's not like a traditional enterprise rack.
I suppose that's a nod of approval! For the creator, of course.
Option 1 Mac Studio setup
Buy the Mac studio. Specifically the Max or Ultra chip with as much memory as you can afford. This is the easiest, cheapest and set and forget setup
View attachment 4995193
On a Mac Studio with 192GB or 256GB of unified memory, the GPU can access a lot of memory. This allows you to run massive Large Language Models like Llama-3 70B or Gemma 4 31B entirely on chip.
You also run a high quantization Llama-3 405B. At 4 bit quantization if my math is correct should be around 230GB of VRAM which is basically as good as ChatGPT-4 but it will be slow. 1-2 tokens per second almost the same speed as fast human typing so it's slow but the Main advantage is you can get a very decent setup for $4-5k and run complex models that can do coding, math and don't hallucinate as much
Option 2 Nvidia AI setup
If you want the absolute best performance possible and also plan to do AI training. You want something that is bullet proof. Then one or multiple RTX 5090. Besides being a gaming beast it is also extremely capable in AI workloads. In fact in startups and many low budget workflows they only use RTX 5090s for everything.
View attachment 4995197
Almost every Al research paper and library PyTorch, TensorFlow is written for NVIDIA/CUDA first. GDDR7 memory provides insane bandwidth you generate images and get tokens within seconds and is in 2026 the industry norm.
There's another variant of this called RTX 6000 Ada or the newer Pro 6000 . Big advantage of this GPU besided more VRAM is Error correction memory which is super important if you are doing serious stuff. You can run this GPU at 100% for months and it won't crash which is great for AI training and fine tuning but it only makes sense to buy this GPU if you are doing AI research work and your work makes you money. It alsp costs as much as a car. Like $10k each.
View attachment 4995329
View attachment 4995215
Problem with this setup is that they stupid expensive and at 32-48GB, you are effectively locked out of the most intelligent models unless you use heavy distillation or multiple cards.
You are limited to 8B to 30B still very powerful but the not the full ChatGPT replacement unless you spend $15k+ on multiple GPUs.
Local LLM models recommendation for each setup
For Mac Studio with 128-256 gb vram
meta-llama/Llama-3.1-405B ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
As I said Llama-3 405B with 4 bit quantization is the best for general usecase imo.
CohereLabs/c4ai-command-r-plus ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
deepseek-ai/DeepSeek-R1 ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
You can also run Command R+ and DeepSeek-V3 / R1 mixture of experts but I would stick to Llama because these are more specialized model for specific use cases.
black-forest-labs/FLUX.1-schnell ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
Image and Video Generation models. I know why most of you nighas want to run local LLMs. It is to generate AI porn. For image generation mac studio is decent but not ideal. For generation of pixels you need TFLOPS
View attachment 4995351
and everything like I said is optimized for CUDA and apple uses META which is completely different architecture
so mac studio tries to cope by trying to imitate what nvidia card does through a bunch of complex
processes that I won't get into but the result is long wait times like 30-50 seconds per image. The gold standard rn for images is Flux and for videos it's a nightmare even a 5 second video can take 10-20 minutes on Kling local. If you are on a Mac, you should also look for MLX versions of models on Hugging Face.
Kling AI: Next-Gen AI Video & AI Image Generator
Create professional videos and images with Kling AI's state-of-the-art generative AI platform. Our tools support video generation, image creation, and advanced editing capabilities for content creators.kling.ai
For the Nvidia setup with 32-48 GB RAM
Gemma 4 31B. Released just last month. Somehow very few people I think because it got overshadowed by the Gemini-4 release rumours but it's Google's local masterpiece.
google/gemma-4-31B ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
View attachment 4995276
On a 5090, it hits 80+ tokens per second and it demolishes Qwen and other older models. It is basically a super intelligent instant messenger. Takes full advantages of the CUDA and tensor cores to give you blazing fast speed and is extremely good for a model that is only 31B parameters.
And for Image and video Generation. This is where Nvidia and Cuda environment earns it's price tag. same Flux image takes 1.5-3 seconds that's it. You can run also run a brand new thing in the market called Real-time SDXL where the image changes as you type the prompt which is super cool.
For Video Generation. Most models are trained and built specifically for Cuda so nvidia again shines here. A 5-second video clip renders in under 60 seconds.
Only problem is you can't get too much detail without spending a shit ton of money. Like you can't generate an 8k picture without running out of memory
If you tech savvy and understand AI on software level or you are an AI engineer. I also encourage you to fiddle on GGUF files and also do some fine tuning to make the AI behave exactly as you want. Even I'm learning about these things and experimenting but be very careful because you can give the AI a lobotomy easily.
For the ultimate baller setup. You can use both Max studio and the Nvidia GPU. This was a pipe dream since forever but Just a few weeks back a company called Tiny Corp released officially signed drivers (TinyGPU) that allow Apple Silicon Macs to talk to NVIDIA GPUs over Thunderbolt 4/5. So you get the best of both world. Only con is the bandwidth. Thunderbolt is fast but it's still slower than a PCIe but still much better than using metal.
P.S Two more cons I forgot to mention. Mac studio is very efficient barely sips like 230-250W even on full load, is dead silent and doesn't get too hot. Nvidia setup is the opposite you easily would need something like 1600W PSU or something to run them. If you live in Europe or in those older houses in america or apartments, a 1600W PC plus a monitor, a fridge on the same circuit will trip the breaker and give you a massive electricity bill. One more thing is that the fans will ramp up to 100% AI inference is intensive and is a sustained heavy load unlike gaming which is spikes, your PC will sound like a jet engine taking off and your PC and even the room it is in, will get hotter. My mum always yells at me every time I turn it on. Just something to keep in mind if you live with someone or have a small room or something.
That is what happens when you thore Billions of dollars at the problem
highly recommend.
Nice man good to see enthusiasts hereI have a 5060ti with 16gb of vram and juggernautXL worked pretty well
not sure if you'll agree or disagree w/ me but ig i'll say it anyways
I probably sounded too elitist in the OP even though I didn't mean to. Models that us engineers work with are generally much heavier and are more of the workhorse grade. what i mentioned in op are true replacements for frontier llmsI have a 5060ti with 16gb of vram and juggernautXL worked pretty well
Option 1 Mac Studio setup
Buy the Mac studio. Specifically the Max or Ultra chip with as much memory as you can afford. This is the easiest, cheapest and set and forget setup
View attachment 4995193
On a Mac Studio with 192GB or 256GB of unified memory, the GPU can access a lot of memory. This allows you to run massive Large Language Models like Llama-3 70B or Gemma 4 31B entirely on chip.
You also run a high quantization Llama-3 405B. At 4 bit quantization if my math is correct should be around 230GB of VRAM which is basically as good as ChatGPT-4 but it will be slow. 1-2 tokens per second almost the same speed as fast human typing so it's slow but the Main advantage is you can get a very decent setup for $4-5k and run complex models that can do coding, math and don't hallucinate as much
Option 2 Nvidia AI setup
If you want the absolute best performance possible and also plan to do AI training. You want something that is bullet proof. Then one or multiple RTX 5090. Besides being a gaming beast it is also extremely capable in AI workloads. In fact in startups and many low budget workflows they only use RTX 5090s for everything.
View attachment 4995197
Almost every Al research paper and library PyTorch, TensorFlow is written for NVIDIA/CUDA first. GDDR7 memory provides insane bandwidth you generate images and get tokens within seconds and is in 2026 the industry norm.
There's another variant of this called RTX 6000 Ada or the newer Pro 6000 . Big advantage of this GPU besided more VRAM is Error correction memory which is super important if you are doing serious stuff. You can run this GPU at 100% for months and it won't crash which is great for AI training and fine tuning but it only makes sense to buy this GPU if you are doing AI research work and your work makes you money. It alsp costs as much as a car. Like $10k each.
View attachment 4995329
View attachment 4995215
Problem with this setup is that they stupid expensive and at 32-48GB, you are effectively locked out of the most intelligent models unless you use heavy distillation or multiple cards.
You are limited to 8B to 30B still very powerful but the not the full ChatGPT replacement unless you spend $15k+ on multiple GPUs.
Local LLM models recommendation for each setup
For Mac Studio with 128-256 gb vram
meta-llama/Llama-3.1-405B ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
As I said Llama-3 405B with 4 bit quantization is the best for general usecase imo.
CohereLabs/c4ai-command-r-plus ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
deepseek-ai/DeepSeek-R1 ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
You can also run Command R+ and DeepSeek-V3 / R1 mixture of experts but I would stick to Llama because these are more specialized model for specific use cases.
black-forest-labs/FLUX.1-schnell ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
Image and Video Generation models. I know why most of you nighas want to run local LLMs. It is to generate AI porn. For image generation mac studio is decent but not ideal. For generation of pixels you need TFLOPS
View attachment 4995351
and everything like I said is optimized for CUDA and apple uses META which is completely different architecture
so mac studio tries to cope by trying to imitate what nvidia card does through a bunch of complex
processes that I won't get into but the result is long wait times like 30-50 seconds per image. The gold standard rn for images is Flux and for videos it's a nightmare even a 5 second video can take 10-20 minutes on Kling local. If you are on a Mac, you should also look for MLX versions of models on Hugging Face.
Kling AI: Next-Gen AI Video & AI Image Generator
Create professional videos and images with Kling AI's state-of-the-art generative AI platform. Our tools support video generation, image creation, and advanced editing capabilities for content creators.
For the Nvidia setup with 32-48 GB RAM
Gemma 4 31B. Released just last month. Somehow very few people I think because it got overshadowed by the Gemini-4 release rumours but it's Google's local masterpiece.
google/gemma-4-31B ยท Hugging Face
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co
View attachment 4995276
On a 5090, it hits 80+ tokens per second and it demolishes Qwen and other older models. It is basically a super intelligent instant messenger. Takes full advantages of the CUDA and tensor cores to give you blazing fast speed and is extremely good for a model that is only 31B parameters.
And for Image and video Generation. This is where Nvidia and Cuda environment earns it's price tag. same Flux image takes 1.5-3 seconds that's it. You can run also run a brand new thing in the market called Real-time SDXL where the image changes as you type the prompt which is super cool.
For Video Generation. Most models are trained and built specifically for Cuda so nvidia again shines here. A 5-second video clip renders in under 60 seconds.
Only problem is you can't get too much detail without spending a shit ton of money. Like you can't generate an 8k picture without running out of memory
If you tech savvy and understand AI on software level or you are an AI engineer. I also encourage you to fiddle on GGUF files and also do some fine tuning to make the AI behave exactly as you want. Even I'm learning about these things and experimenting but be very careful because you can give the AI a lobotomy easily.
For the ultimate baller setup. You can use both Max studio and the Nvidia GPU. This was a pipe dream since forever but Just a few weeks back a company called Tiny Corp released officially signed drivers (TinyGPU) that allow Apple Silicon Macs to talk to NVIDIA GPUs over Thunderbolt 4/5. So you get the best of both world. Only con is the bandwidth. Thunderbolt is fast but it's still slower than a PCIe but still much better than using metal.
P.S Two more cons I forgot to mention. Mac studio is very efficient barely sips like 230-250W even on full load, is dead silent and doesn't get too hot. Nvidia setup is the opposite you easily would need something like 1600W PSU or something to run them. If you live in Europe or in those older houses in america or apartments, a 1600W PC plus a monitor, a fridge on the same circuit will trip the breaker and give you a massive electricity bill. One more thing is that the fans will ramp up to 100% AI inference is intensive and is a sustained heavy load unlike gaming which is spikes, your PC will sound like a jet engine taking off and your PC and even the room it is in, will get hotter. My mum always yells at me every time I turn it on. Just something to keep in mind if you live with someone or have a small room or something.
Stabble diffusion but the images generated are usually trash. Grok imagine was the closest thing you could get to generate NSFW images and it was the absolute best thing in the market but normies abused it to the point where XAi had to lobotomize and nerf it.
