[Store] Add HybridStore with BM25 ranking for PostgreSQL

[ahmed-bhs]

Q	A
Bug fix?	no
New feature?	yes
Docs?	no
License	MIT

Problem

Vector search and full-text search each have limitations:

• Vector search: Great for semantic similarity, but may rank exact term matches lower
• Full-text search: Great for exact matches, but misses semantic relationships

Users often need both: "Find documents about space travel that mention Apollo"

Solution

New PostgreSQL HybridStore combining three search methods with Reciprocal Rank Fusion (RRF):

Method	Extension	Purpose
Semantic	pgvector	Conceptual similarity
Keyword	BM25 or native FTS	Exact term matching
Fuzzy	pg_trgm	Typo tolerance

Why BM25 over native PostgreSQL FTS?

Native PostgreSQL uses TF-IDF which has known limitations:

• No document length normalization (long documents score higher unfairly)
• Term frequency grows unbounded (repeating a word 100x inflates score)

BM25 fixes these issues with saturation and length normalization — that's why Elasticsearch, Meilisearch, and Lucene all use it.

Fallback strategy

BM25 requires the plpgsql_bm25 extension. For users without it:

• Default: PostgresTextSearchStrategy using native ts_rank_cd (works everywhere)
• Optional: Bm25TextSearchStrategy for better ranking (requires extension)

// Native FTS fallback (default)
$store = new HybridStore($pdo, 'movies');

// BM25 for better ranking (requires extension)
$store = new HybridStore($pdo, 'movies', 
    textSearchStrategy: new Bm25TextSearchStrategy(bm25Language: 'en')
);

Features

• Pluggable text search: BM25 (ParadeDB) or native PostgreSQL FTS
• RRF fusion: Merges vector + keyword + fuzzy rankings
• Configurable ratio: 0.0 = keyword-only → 1.0 = vector-only
• Fuzzy matching: Typo tolerance via pg_trgm
• Field boosting: title: 2x, overview: 1x
• Score normalization: 0-100 range

Configuration

framework:
    ai:
        stores:
            hybrid:
                postgres:
                    connection: doctrine.dbal.default_connection
                    table_name: movies
                    semantic_ratio: 0.7
                    fuzzy_weight: 0.3
                    normalize_scores: true
                    searchable_attributes:
                        title: { boost: 2.0, metadata_key: 'title' }
                        overview: { boost: 1.0, metadata_key: 'overview' }

Usage

$store = new HybridStore($pdo, 'movies', semanticRatio: 0.7);
$results = $store->query($vector, ['q' => 'space adventure', 'limit' => 10]);

References

• Comparing BM25, TF-IDF and Postgres FTS
• Supabase Hybrid Search

[Copilot]

Pull Request Overview

This PR introduces PostgresHybridStore, a new vector store implementation that combines semantic vector search (pgvector) with PostgreSQL Full-Text Search (FTS) using Reciprocal Rank Fusion (RRF), following Supabase's hybrid search approach.

Key changes:

• Implements configurable hybrid search with adjustable semantic ratio (0.0 for pure FTS, 1.0 for pure vector, 0.5 for balanced)
• Uses RRF algorithm with k=60 default to merge vector similarity and ts_rank_cd rankings
• Supports multilingual content through configurable PostgreSQL text search configurations

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
src/store/src/Bridge/Postgres/PostgresHybridStore.php	Core implementation of hybrid store with vector/FTS query building, RRF fusion logic, and table setup with tsvector generation
src/store/tests/Bridge/Postgres/PostgresHybridStoreTest.php	Comprehensive test coverage for constructor validation, setup, pure vector/FTS queries, hybrid RRF queries, and various configuration options

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[chr-hertel]

In general this is a super cool feature - some copilot findings seem valid to me - please check.

On top, I was unsure if all sprintf need to be sprintf or some values can/should be a prepared parameter - that'd be great to double check as well please.

[chr-hertel]

@ahmed-bhs could you please have a look at the pipeline failures - i think there's still some minor parts open

[OskarStark]

Open to finish this PR @ahmed-bhs ?

[ahmed-bhs]

Hi @OskarStark,

The work on my side is complete, you can take a look whenever you have time.

To give you a bit of context on the evolution of this work:

Initially, I wanted to propose a hybrid search implementation on PostgreSQL, combining semantic search (pgvector) with PostgreSQL’s native text search based on TF-IDF (ts_rank_cd).

However, TF-IDF has well-known scoring limitations:

• No length normalization: longer documents are unfairly favored
• Unbounded term frequency: repeating a word 100 times artificially inflates the score

That’s why I then suggested using BM25 (the algorithm used by Elasticsearch, Meilisearch, Lucene), which addresses these issues through saturation and document length normalization.

BM25, however, requires the plpgsql_bm25 extension, which is not installed by default. So I implemented a fallback architecture:

• Default: PostgresTextSearchStrategy using native FTS (works everywhere)
• Optional: Bm25TextSearchStrategy for better ranking (requires the extension)

I also extracted the RRF (Reciprocal Rank Fusion) logic into a dedicated class for reusability.

Feel free to reach out if you have any questions or feedback!