Project N.O.M.A.D.: Your Offline AI Survival Kit

 

Project N.O.M.A.D.: Your Offline AI Survival Kit (And How to Actually Get It Running)

If you've been following the homelab and self-hosted AI scene, you've probably heard of Project N.O.M.A.D. which is short for Networked Offline Machine for Autonomous Data. It's an impressive all-in-one offline-first AI platform built by Crosstalk Solutions, designed to give you a full AI assistant, knowledge base, and document processing engine that works even when the internet doesn't. Think Wikipedia mirrors, Stack Exchange snapshots, medical references, and a fully local LLM; all in a Docker stack you run on your own hardware.

This post covers what NOMAD is, how to install it, how to get it behind a reverse proxy with authentication, and (for those of you with AMD Ryzen mini PCs) how to unlock GPU acceleration so your AI assistant actually runs at a useful speed.

Offline AI LLM System (TrueNAS Hosted)

 

TrueNAS Hosted Offline LLM:
Dolphin, Ollama and AnythingLLM

Introduction

Running AI models on your TrueNAS server provides centralized access to powerful language models for your entire network. This guide walks through deploying Ollama with Dolphin models and AnythingLLM on TrueNAS, making AI assistance available to all your devices while maintaining complete data privacy.

By the end of this guide, you'll have:

• Ollama running as a Docker container on TrueNAS
• Dolphin models stored efficiently on your ZFS pool
• AnythingLLM accessible via web browser from any device on your network
• Persistent storage that survives container restarts
• Optional GPU passthrough for accelerated inference

Offline AI LLM System

Create a Completely Offline LLM:
Using Dolphin, Ollama, and AnythingLLM

Running a Large Language Model (LLM) completely offline gives you privacy, control, and independence from cloud services. In this comprehensive guide, I'll walk you through setting up a fully functional offline AI assistant using three powerful tools:

• Dolphin - Uncensored, instruction-tuned language models
• Ollama - Simple, efficient local LLM runtime
• AnythingLLM - User-friendly web interface for interacting with local models

By the end of this guide, you'll have a ChatGPT-like experience running entirely on your own hardware, with no internet connection required!!