AI Similarity Search And Clustering Tool

What is Faiss and what are its main features?

Faiss is a library for efficient similarity search and clustering of dense vectors. It is developed primarily at Meta's FAIR (Fundamental AI Research) team. Its main features include:

• Efficient algorithms for searching in large sets of dense vectors
• Ability to search vectors that may not fit in RAM
• Clustering support
• CPU and GPU implementations
• Python wrappers for easy use in Python
• Batch processing, maximum inner product search (MIPS), and range search
• Ability to return not just the nearest neighbor, but the 2nd, 3rd, …, k-th nearest neighbors
• Store the index on disk rather than in RAM
• Index binary vectors
• Ignore a subset of index vectors according to a predicate on the vector IDs

​

How do you install Faiss?

Faiss can be installed through Conda. The recommended commands are:

• For the CPU version: conda install -c pytorch faiss-cpu
• For the GPU version: conda install -c pytorch faiss-gpu

Note that you should install either the CPU or the GPU package, but not both, as the GPU package is a superset of the CPU package.

​

Can Faiss handle datasets larger than RAM?

Yes. Faiss builds an index in RAM from your vectors, and it can search in sets of vectors of any size, including those that do not fit entirely in RAM.

​

Which languages and interfaces does Faiss provide?

Faiss is written in C++ with complete wrappers for Python, enabling use from both C++ and Python environments.

​

Who develops Faiss and what research foundations is it based on?

Faiss is developed primarily at FAIR, the fundamental AI research team of Meta. It is based on a range of foundational research in high-dimensional similarity search, including:

• Inverted file (from Video Google: A Text Retrieval Approach to Object Matching in Videos)
• Product quantization (PQ)
• IVFADC-R (IndexIVFPQR) three-level quantization
• Inverted multi-index
• Optimized PQ
• Pre-filtering of PQ distances (Polysemous codes)
• GPU implementations for large-scale search
• HNSW indexing method
• In-register vector comparisons for PQ with SIMD
• Binary multi-index hashing
• Graph-based NSG indexing
• Local search quantization (LSQ) and LSQ++
• Residual quantization
• A general survey of product quantization methods

​

What search operations does Faiss support?

Faiss supports a variety of search operations, including:

• k-nearest neighbor search (returning the nearest, 2nd nearest, etc.)
• Batch search (processing multiple query vectors at once)
• Maximum inner product search (MIPS)
• Range search (returning all elements within a given radius)
• On-disk indexing (store indices on disk rather than RAM)
• Indexing and querying binary vectors
• Filtering index vectors by a predicate on their IDs