Benchmarks · HNSW vector search

MyVector adds approximate nearest-neighbor search to stock MySQL — no separate vector database. We put it on identical hardware across three MySQL versions, added MariaDB’s native vector type as a reference, and measured build time, throughput, latency, and recall on the same data.

askdba/myvector · live dashboard at bench.myvector.online

Why benchmark this at all

Vector search usually means standing up a dedicated database next to your primary store, then keeping the two in sync. MyVector takes the other path: it lives inside MySQL as UDFs and stored procedures, builds an HNSW index over a column of vectors, and answers nearest-neighbor queries in SQL. The obvious question is whether that convenience costs you speed or accuracy.

To answer it honestly we needed a level field. Every engine runs in its own container on one Ampere A1 host, each pinned to the same 4 vCPU / 24 GB slice, loading the same 50,000 synthetic 128-dimensional vectors (seeded, so every engine indexes identical data), and builds an HNSW index with the same parameters (M=16, ef_construction=200, L2 distance). Three of the four are MySQL running MyVector — versions 8.4, 9.7 and 26.7. The fourth is MariaDB 12.3, which ships its own native VECTOR type, included as a cross-engine reference point.

What each number means

Five measurements, each isolating one thing:

• Index build — wall-clock seconds to build the HNSW graph over all 50k rows.
• Insert throughput — rows/sec written into a live, indexed table.
• ANN QPS & latency — approximate nearest-neighbor queries per second through the index, and their median response time.
• Recall@10 — of the true 10 nearest neighbors (found by exact brute-force search), how many the approximate search actually returns, averaged over held-out queries that were never inserted. 1.0 is perfect; 0.82 means it finds ~8 of the real 10.
• ef_search sweep — the accuracy/throughput trade-off: searching more of the graph raises recall but lowers QPS.

Results

The headline is throughput. On identical A1 hardware, all three MySQL versions clear ~530–550 approximate queries per second at ~1.8 ms median — roughly 500× faster than the exact brute-force scan (~1 QPS) they fall back to without an index. 8.4, 9.7 and 26.7 land within a few percent of each other, so moving to a newer MySQL costs nothing here.

The real picture: accuracy vs throughput

A single QPS number hides the trade-off. The ef_search knob lets each query decide how hard to look: low values are fast but miss neighbors, high values are accurate but slow. Plotting recall against QPS across that sweep is the honest comparison — up and to the right is better.

Exact brute-force KNN measured ~1 QPS (~700 ms) on every engine — the baseline the index is beating.

The MariaDB surprise

The first MariaDB run looked grim: 29 ANN QPS, a 583-second build, recall aside. Taken at face value it was 15× behind MySQL. That gap was too large to be a real engine difference, so we dug in.

The cause was one default. MariaDB caches its HNSW graph in memory up to mhnsw_max_cache_size, which ships at 16 MB — but the index for this dataset is ~27 MB. The graph didn’t fit, so every search and the whole online build re-read nodes from disk. MyVector sidesteps this by loading its entire index into memory, so the default pitted a disk-bound MariaDB against an in-memory MyVector.

Sizing the cache above the index changed everything:

Takeaways

• Native MySQL vector search is fast. ~530–550 QPS at single-digit-millisecond latency on 4 modest cores, ~500× over brute force — without a second database to run.
• Newer MySQL isn’t slower. 8.4, 9.7 and 26.7 land within a few percent of each other; there’s no regression moving forward.
• MariaDB 12.3 holds its own once its HNSW cache fits the index — higher recall, comparable throughput.
• Configuration is part of the result. The most interesting finding wasn’t a speed number; it was a default that quietly handicapped an engine 15×.

Every number here is reproducible. The live, interactive dashboard is at bench.myvector.online, and the benchmark harness and raw results live in github.com/askdba/myvector under scripts/bench-dashboard. Workload: 50,000 × 128-d vectors, L2 distance, HNSW M=16 / ef_construction=200, held-out queries for recall, each engine pinned to 4 vCPU / 24 GB on Ampere.