Software > User Zone
anyone running local LLMs?
power9mm:
Looking for advice on what hardware I should go with if I wanted to run a local llm on raptor hardware. Would an 8 core blackbird with a 48gb vram gpu be sufficient? or should I be considering one of the talos 2 options..? unsure how ROCm support is or if I should consider nvidia for the computer drives.
atomicdog:
I don't think NVIDIA supports Power anymore.
I have an NVIDIA A2 and I tried to get it working but the last supported CUDA version is very old.
I tried to get Claude Code to port it over to the newer one but it was so much different it would require a whole lot of work.
MPC7500:
You could give this a try: https://github.com/Scottcjn/nvidia-power8-patches
power9mm:
--- Quote from: MPC7500 on July 20, 2026, 01:58:01 pm ---You could give this a try: https://github.com/Scottcjn/nvidia-power8-patches
--- End quote ---
Yeah I saw that in the other thread, I suppose I'll have to give it a go if I get another power9 system before any new systems roll out whenever that will be but I don't think it'll be for awhile.. Surprised there hasn't been much interest running it on open hardware. How does claude and other LLMs work on POWER?
atomicdog:
Claude Code doesn't run on POWER, you need to use SSH or NFS from an x64.
Navigation
[0] Message Index
[#] Next page
Go to full version