One platform, many ways to run AI
The same console deploys, orchestrates, and operates AI workloads — whether that is a single appliance in a lab or a distributed GPU estate.
Private AI, inside your walls
SaaS LLM APIs mean every prompt, document, and usage pattern leaves your network. For regulated industries and sensitive data, that is a non-starter — but running models yourself usually means building a platform first.
How NUSAPOD does it
- Self-hosted on Kubernetes and GPUs you own — nothing phones home
- Ingest models from Hugging Face, a URL, or your own object storage into an in-cluster artifact store
- Scoped API keys per deployment, with exact request and token metering per key
- An append-only audit log of every operator action
What you get
OpenAI-compatible endpoints that never leave your network — swap the base_url in your existing code and keep working.
GPU-as-a-Service for data centers
Teams request whole GPUs, use a fraction, and the rest idles. Serving many consumers from one estate needs isolation, metering, and clean endpoints — per team, per model.
How NUSAPOD does it
- Hardware-enforced VRAM slices: one GPU serves many models without oversubscription
- Custom endpoint paths per model (/m/chat-prod/v1) with per-deployment API keys
- Requests and exact prompt/completion tokens metered per key — chargeback-ready
- Per-GPU utilization, physical VRAM, and 24h energy (kWh) in one console
What you get
One fleet, many measurable consumers — every gigabyte and token attributed.
Mixed-use GPU estates
Your GPUs already run things — training jobs, hand-rolled servers, notebooks. Most schedulers assume they own the card, over-commit the VRAM, and OOM whatever was there first.
How NUSAPOD does it
- A lightweight probe reads real per-process VRAM on every card — including workloads outside Kubernetes
- Provisioning is admission-checked against physical usage: a slice that would not fit is refused up front
- The console shows every process holding VRAM, by name, and alerts on untracked consumers
What you get
Adopt NUSAPOD without evicting anything — verified in production on a GPU shared with a hand-run llama.cpp server.
AI product backends
A model container is not a production API. You need readiness gating, streaming, capacity signals, and a way to grow a model that outgrew its slice — without rebuilding the stack.
How NUSAPOD does it
- Endpoints publish only when the model actually answers (health-gated — no 502s mid-load)
- Streaming pass-through with usage accounting; custom endpoint paths that survive restarts
- Alerts before trouble: OOM-risk (slice nearly full), request backlog, crash-with-cause
- One-click VRAM resize, latency p95 / queue / token-throughput dashboards per deployment
What you get
An inference backend with SLO instrumentation built in — not a bare container.
Research & experimentation
One experiment per GPU wastes capacity; sharing a card by convention ends in OOM roulette. Iterating on models should take minutes, not tickets.
How NUSAPOD does it
- Right-sized slice per experiment, provisioned in minutes with a suggested VRAM size per model
- Warm model caches on every node — restarts and re-provisions skip the download
- A built-in test-chat playground (streaming, reasoning traces, tok/s) on every deployment
- Release returns the VRAM to the pool immediately
What you get
More experiments per GPU, with none of the sharing accidents.
Edge & appliance-class hardware
Unified-memory GPUs (NVIDIA GB10 / DGX Spark-class) break standard tooling: NVML cannot report memory, DCGM reads nothing, and naive schedulers see 0 GB.
How NUSAPOD does it
- NUSAPOD ships its own telemetry probe: physical VRAM, utilization, temperature, power, and clocks — no DCGM required
- The installer auto-detects unified-memory GPUs and configures slicing for them
- Full observability (GPU → host → serving metrics) on hardware other stacks treat as a black box
What you get
Appliance-class boxes as first-class fleet members — verified end-to-end on real GB10 hardware.