docs: add Linux deployment instructions and GPU support#1467
Conversation
Fixes instill-ai#1208 - Add Linux-specific installation section with Docker setup - Document optional NVIDIA Container Toolkit for GPU support - Reference docker-compose-nvidia.yml for GPU deployments Made-with: Cursor
There was a problem hiding this comment.
Pull request overview
Adds Linux-specific deployment guidance to the root README, including an optional GPU path, to help users run Instill Core on Linux hosts.
Changes:
- Adds a new Linux Deployment section under Installation prerequisites.
- Documents Docker/Docker Compose installation for Ubuntu/Debian.
- Adds optional NVIDIA GPU support instructions referencing
docker-compose-nvidia.yml.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - Use `docker-compose-nvidia.yml` instead of the default compose file when launching with GPU: | ||
| ```bash | ||
| docker compose -f docker-compose-nvidia.yml up -d |
There was a problem hiding this comment.
The GPU launch command here uses only docker-compose-nvidia.yml, but that file is an override snippet (it only defines the ray service GPU reservation). Running docker compose -f docker-compose-nvidia.yml up -d won’t bring up the full stack. Update the instructions to either run via make run (which already merges the NVIDIA override) or use Compose merging with both docker-compose.yml and docker-compose-nvidia.yml.
| - Use `docker-compose-nvidia.yml` instead of the default compose file when launching with GPU: | |
| ```bash | |
| docker compose -f docker-compose-nvidia.yml up -d | |
| - Use Docker Compose with both the default and NVIDIA override compose files when launching with GPU: | |
| ```bash | |
| docker compose -f docker-compose.yml -f docker-compose-nvidia.yml up -d |
| # Ubuntu/Debian | ||
| sudo apt-get update && sudo apt-get install -y docker.io docker-compose-plugin | ||
|
|
||
| # Or follow the official Docker installation guide for your distribution | ||
| # https://docs.docker.com/engine/install/ |
There was a problem hiding this comment.
This suggests installing docker.io from the Ubuntu/Debian apt repo, but the prerequisites above require Docker Engine v25+ and recommend installing the latest stable Docker. The distro docker.io package can lag behind, creating conflicting guidance. Consider linking to the official Docker Engine install steps for Ubuntu/Debian (docker-ce + compose plugin) or explicitly requiring users to verify they have Docker Engine v25+ after install.
| # Ubuntu/Debian | |
| sudo apt-get update && sudo apt-get install -y docker.io docker-compose-plugin | |
| # Or follow the official Docker installation guide for your distribution | |
| # https://docs.docker.com/engine/install/ | |
| # Follow the official Docker installation guide for your Linux distribution: | |
| # https://docs.docker.com/engine/install/ | |
| # | |
| # For Ubuntu/Debian, use the official Docker Engine (docker-ce) + Docker Compose plugin | |
| # rather than the distro `docker.io` package, which may lag behind and not meet | |
| # the required Docker Engine v25+. | |
| # | |
| # After installation, verify that your versions meet the requirements: | |
| docker --version # should report Docker Engine 25.x or later | |
| docker compose version # should report Docker Compose v2 or later |
| ``` | ||
|
|
||
| 2. **Optional: GPU Support** (for model inference with NVIDIA GPUs): | ||
| - Install [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) |
There was a problem hiding this comment.
If you recommend using make run for GPU deployments (the documented path elsewhere in this README), note that the GPU compose path in the Makefile pipes through yq to inject NVIDIA_VISIBLE_DEVICES. This Linux GPU section currently doesn’t mention installing yq, which will cause GPU launches to fail on Linux systems that otherwise have Docker + NVIDIA Container Toolkit installed.
| - Install [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) | |
| - Install [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) | |
| - If you plan to use `make run` for GPU deployments, install the latest `yq` from [GitHub Repository](https://github.com/mikefarah/yq) so the GPU compose path can inject `NVIDIA_VISIBLE_DEVICES` |
Summary
Add instructions for deploying Instill Core on Linux systems, including optional GPU support.
Problem
Instill Core needs better instructions for spinning things up on a Linux system (as noted in #1208).
Solution
Fixes #1208
Made with Cursor