How to Configure Ubuntu for Deep Learning with NVIDIA CUDA

Ubuntu is a practical base for deep learning because it combines long-term support, broad hardware compatibility, and strong documentation from NVIDIA, PyTorch, TensorFlow, and the wider Linux community. With the right NVIDIA driver, CUDA toolkit, Python environment, and testing routine, an ordinary desktop can become a capable machine for computer vision, large language models, and data science experiments.

The smoothest setup begins with a clear separation between system software and project environments. The NVIDIA driver belongs to Ubuntu’s operating system, while CUDA libraries, Python packages, notebooks, and model code should be managed carefully so one experiment does not break another. This approach is especially useful for Australian users working from home, in a university lab, or on a workstation connected through a typical NBN service.

Choose A Suitable Ubuntu And NVIDIA Setup

Ubuntu 24.04 LTS is a sensible starting point for a new installation, while Ubuntu 22.04 LTS remains a dependable choice for older research software and established tutorials. A clean installation is preferable if the computer has previously used several proprietary NVIDIA drivers, unofficial CUDA packages, or a mixture of Python distributions. Before changing anything, back up datasets, SSH keys, notebooks, and model checkpoints.

Your graphics card determines the practical size of your projects. CUDA-capable NVIDIA GPUs with more VRAM can handle larger batch sizes and models, but system RAM, storage speed, and cooling matter as well. A card with 8 GB of VRAM may suit image classification, smaller language models, and experimentation, whereas serious generative AI work often benefits from 16 GB or more. Australian electricity prices can make long training runs expensive, so monitor power use and avoid leaving a high-end GPU training overnight without a reason.

Check that the card is visible before installing software:

lspci | grep -i nvidia

If the command returns an NVIDIA device, the hardware is detected by the PCI subsystem. It is also worth checking the manufacturer’s specifications for compute capability and supported driver versions. Laptop users should pay attention to hybrid graphics, where Ubuntu may use an integrated GPU for the desktop and NVIDIA hardware for CUDA workloads.

Install The Recommended NVIDIA Driver

Ubuntu provides a useful way to identify the preferred proprietary driver. Refresh the package index and run:

sudo apt update
ubuntu-drivers devices

The output normally marks one package as recommended. Install the suggested version rather than choosing a random driver from an old blog post:

sudo ubuntu-drivers autoinstall
sudo reboot

After restarting, verify the driver with:

nvidia-smi

A working result displays the GPU model, driver version, temperature, memory use, and a reported CUDA compatibility version. The CUDA version shown by nvidia-smi describes what the installed driver can support; it does not prove that the complete CUDA toolkit is installed. This distinction prevents a common mistake where users see a CUDA number and assume that nvcc must already be available.

Avoid installing a second driver from NVIDIA’s .run file unless you have a specific reason and understand how to remove it. The Ubuntu package system is easier to maintain, particularly when kernel updates arrive. On a desktop in Brisbane or Perth, an unattended kernel upgrade followed by a mismatched manually installed driver can leave the graphical session unavailable, while packages installed through APT are generally easier to repair.

Install The CUDA Toolkit Without Mixing Sources

The CUDA toolkit contains the compiler, headers, profiling utilities, and development libraries needed to build or compile GPU software. It is separate from the display driver. For many Python users, a framework package includes the CUDA runtime libraries it needs, so installing the full toolkit is optional unless you compile custom extensions, use nvcc, or develop CUDA code.

For a straightforward Ubuntu-managed installation, first search the available packages:

apt search cuda-toolkit

If the Ubuntu repositories provide a suitable version, install it with:

sudo apt install cuda-toolkit

For projects that require a particular CUDA release, use NVIDIA’s official CUDA repository instructions for your exact Ubuntu version and architecture. Follow one repository method only. Do not combine Ubuntu packages, NVIDIA .deb repositories, Conda CUDA packages, and a downloaded installer without checking which files own the same libraries.

When installation finishes, test whether the compiler is on your path:

nvcc --version

If Ubuntu installed the toolkit but the command is not found, locate it:

find /usr/local -maxdepth 2 -type f -name nvcc 2>/dev/null

A typical environment adjustment is:

echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

Only add library paths when they are genuinely needed. Modern Linux packages often configure runtime libraries automatically, and an unnecessarily broad LD_LIBRARY_PATH can cause a framework to load an incompatible library before its bundled version.

Verify CUDA With A Small GPU Test

The first useful test is the NVIDIA management utility:

nvidia-smi

Then confirm that the compiler works:

nvcc --version

These commands test different layers. nvidia-smi checks communication between the driver and GPU, while nvcc checks the development toolkit. A machine can pass one test and fail the other, which is why both should be recorded when diagnosing installation problems.

A Python-level test gives more meaningful evidence for machine learning. Create a clean virtual environment and install a framework build that officially supports your chosen CUDA runtime. For PyTorch, use the installation command generated on the project’s official selector rather than copying a command intended for a different release. Then run:

import torch

print("PyTorch:", torch.__version__)
print("CUDA available:", torch.cuda.is_available())

if torch.cuda.is_available():
    print("GPU:", torch.cuda.get_device_name(0))
    print("CUDA runtime:", torch.version.cuda)

The reported runtime can differ from the system toolkit because PyTorch wheels commonly ship their own CUDA components. That is normal. The key requirement is that the installed NVIDIA driver is new enough for the runtime bundled with the framework. If torch.cuda.is_available() returns False, inspect nvidia-smi, confirm that the correct virtual environment is active, and check for a CPU-only package.

Build Isolated Python Environments

Use one environment per project instead of installing every package globally. Python’s built-in virtual environments are enough for many workloads:

python3 -m venv ~/venvs/vision
source ~/venvs/vision/bin/activate
python -m pip install --upgrade pip

Install PyTorch, TensorFlow, Jupyter, or other libraries inside this environment. Pin important versions in a requirements.txt file or a project configuration file so a working setup can be recreated later. This matters when collaborating across Melbourne, Sydney, or Canberra, where university and business systems may have different package mirrors and upgrade schedules.

Keep datasets separate from code and environments. A fast NVMe drive is useful for image collections and cached model files, while a larger secondary disk can hold archived checkpoints. Use descriptive directories and record the GPU model, driver version, framework version, and training command in a README. Such notes save time when a result needs to be reproduced months later.

You can watch GPU activity during training with:

watch -n 1 nvidia-smi

Low GPU utilisation may indicate a slow data loader, excessive image transformations on the CPU, a small batch size, or storage that cannot supply data quickly enough. High memory use is not automatically a problem; CUDA allocators often reserve memory for efficiency. An out-of-memory error usually calls for a smaller batch, mixed-precision training, gradient accumulation, or a smaller model.

Use Containers For Repeatable Experiments

Docker is useful when a project depends on exact operating system libraries or a specific CUDA-enabled framework. Install Docker from a trusted Ubuntu or Docker source, then add NVIDIA Container Toolkit according to NVIDIA’s current instructions. After restarting the Docker service, a basic test should expose the GPU inside a container:

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Choose an image whose CUDA runtime is supported by your installed driver. The host needs the NVIDIA driver, but the container normally supplies its own user-space CUDA libraries. This model keeps the workstation cleaner and makes it easier to move a project to a lab server or a rented GPU instance.

Containers are also useful when Australian cloud pricing is a concern. A researcher in Adelaide or Hobart might train locally on smaller experiments and rent a GPU only for a final run, rather than paying for a cloud instance continuously. Keep an eye on storage transfer fees, data residency requirements, and the time needed to upload a large dataset over a residential connection.

Tune The Desktop And Maintain The System

A deep learning workstation does not need a visually elaborate desktop, but a clear workspace helps. Keep a terminal, editor, file manager, GPU monitor, and notebook browser easy to reach. If you like changing the desktop appearance, a carefully chosen background can make the machine feel personal without adding software overhead; collections such as luxury car wallpapers can suit a performance-focused workstation, provided the files come from a source you trust.

Disable unnecessary startup applications and check thermals during a sustained workload. Dust, restricted airflow, and warm rooms can cause throttling, particularly during an Australian summer. A GPU that is technically functioning but constantly reducing its clock speed may train more slowly and consume more electricity. Use nvidia-smi to watch temperature, power draw, and memory, and set sensible power limits only if you understand the effect on performance.

Apply normal Ubuntu updates, but avoid upgrading the driver or CUDA stack in the middle of a long project without saving the current environment. Before a major change, capture useful information:

nvidia-smi
uname -r
python --version
pip freeze > requirements-lock.txt

If a kernel update causes trouble, booting an earlier kernel from the GRUB advanced options can help you recover the system. Keep at least one known-good project environment and test upgrades with a small training job before launching a run that may take several days.

A reliable configuration is built in layers: Ubuntu detects the hardware, the proprietary driver communicates with the GPU, CUDA supplies development tools when required, and the machine learning framework uses a compatible runtime. Testing each layer separately makes errors understandable instead of turning them into a confusing collection of package messages.

The key thing to remember is that a successful CUDA setup is not defined by one version number. It is a repeatable combination of a supported NVIDIA driver, an appropriate framework environment, verified GPU access, sensible thermal management, and notes that let you rebuild the same workstation when the next project begins.