InferenceSep 15, 20265 min read

Milvus Explained: Vector Search and Storage Separation

Milvus: Understand how query and data roles separate ingestion from vector search. Compare index and deployment choices against your workload.

Docs analysis

Who this is for: Developers evaluating milvus.

The answer in 3 lines

  • Understand how query and data roles separate ingestion from vector search.
  • Compare index and deployment choices against your workload.
  • Evidence: linked documentation; no independent test results are claimed.

Evidence: Analysis of the linked documentation and source-reported behavior; examples and decision tables are explanatory, not executed tests.

AI-assisted documentation analysis · 2 source links

Based on linked documentation and web research. Source-reported benchmarks are attributed to their publishers; this report does not establish independent testing. How reports are produced.

Milvus Decoupled Compute and Storage Architecture

Loading diagram…

Conceptual map of the documented components, not a measured deployment or proof of failure recovery.

Concise Takeaway

Milvus is an open-source, cloud-native vector database designed to handle high-concurrency approximate nearest neighbor (ANN) search on large-scale unstructured datasets milvus-io/milvus. By decoupling compute from storage and splitting query execution from data ingestion, Milvus allows engineering teams to scale read operations (Query Nodes) independently from write operations (Data Nodes) README. The primary architectural tradeoff is operational overhead: running a fully distributed, Kubernetes-native cluster with multiple microservice roles requires significantly more infrastructure orchestration than monolithic or embedded vector stores, although lightweight local prototyping is supported via Milvus Lite README.

Decoupled Cloud-Native Architecture

The core architectural foundation of Milvus is the separation of compute and storage README. Rather than binding indexing, searching, and write processes into a single monolith, Milvus partitions these workloads into distinct stateless microservices:

  • Query Nodes: Responsible for loading vector index segments into memory and executing read queries over vector collections README.
  • Data Nodes: Handle incoming write streams, flush inserted data to object storage, and trigger segment index generation README.
  • Persistent Storage: Shared object and file storage holding vector segments, scalar attributes, and persistent index structures README.

This means that during search-heavy traffic spikes (such as bursty RAG application queries), ops teams can horizontally scale Query Nodes while assessing the effects on ingestion and shared resources separately README. Conversely, bulk ingestion pipelines can expand Data Nodes independently. Furthermore, segment replication across multiple Query Nodes improves both fault tolerance and query throughput README.

Written in Go and C++, Milvus incorporates hardware acceleration across CPU and GPU to optimize vector retrieval speeds README. The architecture isolates the core database management system from underlying vector search engines, allowing support for multiple index formats tailored to specific memory and latency requirements:

  • In-Memory Indices: HNSW, IVF, FLAT (brute-force search), and SCANN README.
  • Disk-Based Indices: DiskANN for scaling dataset sizes beyond available system RAM README.
  • GPU Acceleration: High-throughput GPU indexing such as NVIDIA CAGRA README.
  • Memory Optimization: Quantization schemes (e.g., IVFPQ) and memory mapping (mmap) to balance search precision against memory overhead README.

In addition to traditional dense vectors, Milvus natively supports full-text search with BM25 as well as learned sparse embeddings like SPLADE and BGE-M3 README. Developers can store sparse and dense vectors within the same collection and perform hybrid search combining semantic similarity and keyword retrieval with customizable reranking functions README.

Multi-Tenancy and Hot/Cold Tiered Storage

To accommodate multi-tenant SaaS platforms, Milvus provides isolation strategies across four operational granularities: database, collection, partition, and partition key README. These options express different isolation granularities. The appropriate choice depends on the access policy and workload; the hierarchy alone does not establish secure isolation at a particular tenant count.

For cost efficiency, Milvus supports hot/cold storage tiering README. Frequently queried "hot" vector segments remain loaded in memory or on high-speed NVMe drives, while less active "cold" segments transition to cost-effective persistent object storage README. For a team managing multi-billion vector indices, the useful experiment is to compare resource use and latency for the actual hot/cold access pattern.

Python Client Setup and Prototyping

For local prototyping, developers can utilize pymilvus with Milvus Lite, embedding a file-backed vector store directly inside a Python application README. Production deployments switch the client configuration to connect via endpoint URI and authentication tokens to a distributed server or cloud cluster README.

Security and Operational Governance

Milvus incorporates security primitives suitable for enterprise operations, including user authentication, TLS network encryption, and Role-Based Access Control (RBAC), which must be assessed in the selected configuration README. RBAC policies restrict user access to explicit databases, collections, and partitions README.

The supporting ecosystem includes Attu for graphical cluster management, Birdwatcher for deep system debugging, Prometheus/Grafana integrations for metrics monitoring, and Milvus CDC for cross-cluster replication README.

Decision Guidance and Tradeoffs

  • When to choose Milvus: Scale-out RAG systems requiring hybrid dense/sparse retrieval (BM25), granular tenant isolation, or dataset scales reaching billions of vectors where independent scaling meets a demonstrated workload requirement README.
  • When simpler options suffice: Single-process applications or workloads whose required features and measured demand fit a local design where embedding Milvus Lite or direct in-memory search removes Kubernetes orchestration overhead README.
  • Operational considerations: Operating Milvus at production scale requires managing Kubernetes deployments, persistent object storage dependencies, and coordinating stateless microservices README.

Test filtered retrieval before choosing a cluster topology

Suppose a documentation service stores vectors for several customers and filters retrieval to the requesting customer. Begin with correctness and the application's authorization boundary before optimizing retrieval speed. Milvus's index and multi-tenancy options offer choices, not a substitute for that policy. Search and isolation options

Workload caseInspectDecision
Similar text exists in another customer's recordsReturned IDs and applied filtersReject a configuration that crosses the intended scope
New records arrive during active search trafficIngestion state and search resultsDecide whether visibility and latency meet the application contract
A chosen index reduces memory useRecall against a reference set and request timingKeep it only if the quality/resource tradeoff is acceptable

Compare candidate indexes using the same vectors, filter conditions and query set. Increasing cluster size does not repair a wrongly scoped query. A local Lite pilot can validate portions of the data model, while a distributed deployment needs separate checks for scaling and operating behavior.

Select topology from measured demands and required features, not an arbitrary vector-count cutoff. Record the configuration and corpus revision that passed. This is an acceptance worksheet, not a claim that any isolation mode independently enforces the application's full access policy.

Evidence and scope

Primary repository documentation was refreshed on 14 October 2026. The worked scenario is a planning exercise derived from documented behavior, not a product test. Reported measurements remain attributed to their source and workload.

Keep exploring

Inference & deployment: questions, tradeoffs, and guides →

Browse all reports · Suggest a correction