PyTorch from source on NVIDIA Jetson
Build PyTorch from Source on NVIDIA Jetson (Without Losing Your Weekend)
· 2 min read
The NVIDIA Jetson AGX Orin paired with JetPack 6.2.1 (CUDA 12.6, cuDNN 9.3, TensorRT 10.3) offers exceptional potential for edge AI. While NVIDIA supplies prebuilt PyTorch packages, building from source ensures full compatibility, custom optimization, and access to the latest features like BetterTransformer and TorchDynamo.
Why Build from Source?
Prebuilt wheels sometimes default to CPU-only mode or drop CUDA operators like TorchVision’s
nms, leading to runtime errors.Community experience shows that matching PyTorch 2.6.0 with TorchVision 0.21.0 is critical on JetPack 6.2.x (NVIDIA Developer Forums).
Building yourself lets you specify GPU architectures (e.g., Ampere
sm_87) and streamline dependencies like cuSPARSELt (PyTorch Documentation).
Step 1: Flash Jetson with JetPack 6.2.1 & Install Dependencies
Flash device using SDK Manager (confirm JetPack 6.2.1, CUDA 12.6) (PyTorch Documentation).
Install build essentials:
sudo apt update sudo apt install -y python3-pip libopenblas-dev cmake ninja-build libjpeg-dev libpng-dev libtiff-dev(Optional but recommended) Add cuSPARSELt if using custom PyTorch features:
wget <cusparselt archive URL> tar xf libcusparse_lt-*.tar.xz sudo cp -r include/* /usr/local/cuda/include/ sudo cp -r lib/* /usr/local/cuda/lib64/to support new sparse tensor acceleration (PyTorch Documentation).
Step 2: Build PyTorch 2.6.0 from Source
Inside your PyTorch clone:
export PATH=/usr/local/cuda/bin:$PATH
export CUDA_HOME=/usr/local/cuda
export CUDACXX=/usr/local/cuda/bin/nvcc
export TORCH_CUDA_ARCH_LIST="8.7"
export USE_NCCL=0 USE_DISTRIBUTED=0 USE_MKLDNN=0 BUILD_TEST=0
export CMAKE_POLICY_VERSION_MINIMUM=3.5
export MAX_JOBS=2 # lower concurrency to avoid OOM
pip3 install --upgrade pip setuptools wheel ninja
python3 setup.py bdist_wheel
pip3 install dist/torch-*.whl
This ensures compatibility with your Jetson device architecture and avoids memory failures by limiting parallelism.
Step 3: Build TorchVision 0.21.0 to Match
TorchVision must align with PyTorch and include compiled ops like nms.
cd ~/torchvision
git fetch --tags
git checkout v0.21.0
export TORCH_CUDA_ARCH_LIST="8.7"
export BUILD_VERSION=0.21.0
python3 setup.py bdist_wheel
pip install --force --no-cache-dir dist/torchvision-0.21.0*.whl
Be sure to use --no-cache-dir to prevent involuntarily installing CPU-only builds.
Step 4: Verify Successful CUDA Integration
Test directly in a Python shell:
import torch, torchvision
print("Torch:", torch.__version__)
print("TorchVision:", torchvision.__version__)
print("CUDA Available:", torch.cuda.is_available())
from torchvision.ops import nms
print("NMS function:", nms)
print("CUDA Device:", torch.cuda.get_device_name(0))
Output should reflect GPU availability and confirm that nms is properly registered.
Summary Table
| Component | Version | Purpose |
|---|---|---|
| JetPack | 6.2.1 | Base system (CUDA 12.6, cuDNN 9.3, TRT 10.3) |
| PyTorch | 2.6.0 | Source-built with Ampere support |
| TorchVision | 0.21.0 | Rebuilt to match CUDA-enabled PyTorch |
| Optimizations | cuSPARSELt, TorchDynamo | Leverage full Jetson GPU capabilities |
Additional Tips
Use
MAX_JOBS=2to manage memory pressure during builds.Cross-compile on a larger x86 machine if build time or memory continues to be a bottleneck.
For newer JetPack versions, versions like Torch 2.7 and TorchVision 0.22 may apply.
Ensure
numpy <2is installed to avoid build conflicts.
Conclusion
By following this guided build process, you can ensure a reliable, CUDA-accelerated PyTorch and TorchVision environment tailored for Jetson Orin’s high-performance hardware. If you'd like a Dockerfile or automated script version of this guide, I'm happy to help!
Happy building—and happy deep learning at the edge!
Let me know if you’d like images embedded, video walk-through links, or step-by-step troubleshooting sections.