---
title: "Keeping billion-vector search fast on GPUs — Tanuj Dargan"
description: "Where does a GPU vector index actually bottleneck at billion scale: compute, memory, or interconnect? And what does it cost in recall to hold latency down?"
canonical: https://tanujdargan.com/research/billion-vector-retrieval
resume: on request
updated: 2026-11-07
---

# Keeping billion-vector search fast on GPUs

- Author: Tanuj Dargan (https://tanujdargan.com/)
- Where: DEIA Lab, UVic · Prof. Sean Chester
- When: May 2025 – Feb 2026
- Tags: cuVS, CAGRA, IVF-PQ, GPU
- Web page (with live demo): https://tanujdargan.com/research/billion-vector-retrieval

## Question

Where does a GPU vector index actually bottleneck at billion scale: compute, memory, or interconnect? And what does it cost in recall to hold latency down?

## Method

- A custom retrieval system on a modified cuVS fork (CAGRA graph and IVF-PQ indices), across 8+ NVIDIA GPUs and 2 TB of indexed data.
- Automated stress-testing pipelines on Kubernetes GPU clusters, profiling memory bandwidth and NVLink interconnect saturation.

## Results

- 1–4M QPS at billion-vector scale with 90–95% recall and under 250 ms latency.
- 4.3× throughput scaling, with memory bandwidth and NVLink saturation exposed as the bottlenecks.

## Next

- Topology-aware sharding so queries stop crossing the slowest link.
- Recall/latency frontiers published per index type, not as single points.

---

Profile: https://tanujdargan.com/index.md · Resume: on request (https://tanujdargan.com/#resume or dargantanuj@gmail.com) · All write-ups: https://tanujdargan.com/llms.txt