Author Topic: anyone running local LLMs?  (Read 8 times)

power9mm

  • Full Member
  • ***
  • Posts: 125
  • Karma: +8/-3
    • View Profile
anyone running local LLMs?
« on: Today at 01:11:31 pm »
Looking for advice on what hardware I should go with if I wanted to run a local llm on raptor hardware. Would an 8 core blackbird with a 48gb vram gpu be sufficient? or should I be considering one of the talos 2 options..? unsure how ROCm support is or if I should consider nvidia for the computer drives.