The GX10s are clustered and run DeepSeek-V4-Flash-0731-Abliterated. The M4 Pros run various utility LLMs such as Qwen 3.8
Software:
OpenWebUI
Hermes Agent
LiteLLM
Llama.cpp
vLLM
StoryBinder - custom web app that I use for story/script writing when in the mood
LASM - Local AI Stack Manger - custom web interface to manage pretty much all aspects of my local stack. Agents run on the 4 nodes as well as my NAS(Download GGUF/HF snapshots once and copy out to the nodes)
Have yet to connect my Home Assistant into the stack.
Hardware:
2 x ASUS GX10 128GB(each)
2 x Mac Mini M4 Pro 24GB(each)
The GX10s are clustered and run DeepSeek-V4-Flash-0731-Abliterated. The M4 Pros run various utility LLMs such as Qwen 3.8
Software:
OpenWebUI
Hermes Agent
LiteLLM
Llama.cpp
vLLM
StoryBinder - custom web app that I use for story/script writing when in the mood
LASM - Local AI Stack Manger - custom web interface to manage pretty much all aspects of my local stack. Agents run on the 4 nodes as well as my NAS(Download GGUF/HF snapshots once and copy out to the nodes)
Have yet to connect my Home Assistant into the stack.
For mobile access I use Conduit https://github.com/cogwheel0/conduit