Benchmarks · HNSW vector search
MyVector adds approximate nearest-neighbor search to stock MySQL — no separate vector database. We put it on identical hardware across three MySQL versions, added MariaDB’s native vector type as a reference, and measured build time, throughput, latency, and recall on the same data.
askdba/myvector · live dashboard at bench.myvector.online
Why benchmark this at all
Vector search usually means standing up a dedicated database next to your primary store, then keeping the two in sync. MyVector takes the other path: it lives inside MySQL as UDFs and stored procedures, builds an HNSW index over a column of vectors, and answers nearest-neighbor queries in SQL. The obvious question is whether that convenience costs you speed or accuracy.
To answer it honestly we needed a level field. Every engine runs in its own container on one Ampere A1 host, each pinned to the same 4 vCPU / 24 GB slice, loading the same 50,000 synthetic 128-dimensional vectors (seeded, so every engine indexes identical data), and builds an HNSW index with the same parameters (M=16, ef_construction=200, L2 distance). Three of the four are MySQL running MyVector — versions 8.4, 9.7 and 26.7. The fourth is MariaDB 12.3, which ships its own native VECTOR type, included as a cross-engine reference point.
What each number means
Five measurements, each isolating one thing:
- Index build — wall-clock seconds to build the HNSW graph over all 50k rows.
- Insert throughput — rows/sec written into a live, indexed table.
- ANN QPS & latency — approximate nearest-neighbor queries per second through the index, and their median response time.
- Recall@10 — of the true 10 nearest neighbors (found by exact brute-force search), how many the approximate search actually returns, averaged over held-out queries that were never inserted. 1.0 is perfect; 0.82 means it finds ~8 of the real 10.
- ef_search sweep — the accuracy/throughput trade-off: searching more of the graph raises recall but lowers QPS.
Results
The headline is throughput. On identical A1 hardware, all three MySQL versions clear ~530–550 approximate queries per second at ~1.8 ms median — roughly 500× faster than the exact brute-force scan (~1 QPS) they fall back to without an index. 8.4, 9.7 and 26.7 land within a few percent of each other, so moving to a newer MySQL costs nothing here.
The real picture: accuracy vs throughput
A single QPS number hides the trade-off. The ef_search knob lets each query decide how hard to look: low values are fast but miss neighbors, high values are accurate but slow. Plotting recall against QPS across that sweep is the honest comparison — up and to the right is better.
| Engine | Build (s) | Insert QPS | ANN QPS | ANN p50 (ms) | Recall@10 |
|---|
Exact brute-force KNN measured ~1 QPS (~700 ms) on every engine — the baseline the index is beating.
The MariaDB surprise
The first MariaDB run looked grim: 29 ANN QPS, a 583-second build, recall aside. Taken at face value it was 15× behind MySQL. That gap was too large to be a real engine difference, so we dug in.
The cause was one default. MariaDB caches its HNSW graph in memory up to mhnsw_max_cache_size, which ships at 16 MB — but the index for this dataset is ~27 MB. The graph didn’t fit, so every search and the whole online build re-read nodes from disk. MyVector sidesteps this by loading its entire index into memory, so the default pitted a disk-bound MariaDB against an in-memory MyVector.
Sizing the cache above the index changed everything:
The lesson, not the leaderboard
Properly configured, MariaDB’s native vector search is competitive with MyVector on speed and ahead on recall. The takeaway isn’t a winner — it’s that a single cache default can make one engine look an order of magnitude slower than it is. Benchmarks are only as fair as their configuration, which is why the setup here is in the open.
Takeaways
- Native MySQL vector search is fast. ~530–550 QPS at single-digit-millisecond latency on 4 modest cores, ~500× over brute force — without a second database to run.
- Newer MySQL isn’t slower. 8.4, 9.7 and 26.7 land within a few percent of each other; there’s no regression moving forward.
- MariaDB 12.3 holds its own once its HNSW cache fits the index — higher recall, comparable throughput.
- Configuration is part of the result. The most interesting finding wasn’t a speed number; it was a default that quietly handicapped an engine 15×.
Every number here is reproducible. The live, interactive dashboard is at bench.myvector.online, and the benchmark harness and raw results live in github.com/askdba/myvector under scripts/bench-dashboard. Workload: 50,000 × 128-d vectors, L2 distance, HNSW M=16 / ef_construction=200, held-out queries for recall, each engine pinned to 4 vCPU / 24 GB on Ampere.
