Installing Docker, Ollama, and Deepseek-r1 on Debian 12.

1. Jump into root

> sudo -s

2. Add the following “contrib non-free” to /etc/apt/sources.list

"deb http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware"

3. Update the repositories, then install nvidia-driver

> apt update
> apt install nvidia-driver

4. Reboot and test if working with the following command

> nvidia-smi

5. Next, install the nvidia developer CUDA toolkit

> apt install nvidia-cuda-toolkit

6. Confirm with the following

> nvcc --version

7. Install Neat Videocard TOP to monitor resource useage of the video card.

> apt install nvtop

Installing docker

8. Add Docker’s official GPG key:

> apt update
> apt install ca-certificates curl
> install -m 0755 -d /etc/apt/keyrings
> curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
> chmod a+r /etc/apt/keyrings/docker.asc

9. Add the repository to Apt sources

> echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null
> apt update
> apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

10. Confirm docker service is enabled and running.

> systemctl enable docker
> systemctl start docker

Installing Ollama

12. Install the Nvidia Container Toolkit

> curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
> sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list
> apt update
> apt install -y nvidia-container-toolkit
> nvidia-ctk runtime configure --runtime=docker (OR rootless mode) nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json
> systemctl restart docker

> docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

13. Testing it all out.

> docker exec -it ollama ollama run deepseek-r1:8b