Skip to content

Conversation

@suddendust
Copy link
Contributor

@suddendust suddendust commented Nov 21, 2025

Description

This is a successor to this PR: #251 for json fields.

JSONB Query Optimization: Before vs After

Query Type Old SQL (Defensive) New SQL (Optimized)
EQ (String) "props"->>'brand' = 'Dettol' "props"->>'brand' = 'Dettol'
NEQ (String) "props"->>'brand' != 'Dettol' "props"->>'brand' != 'Dettol'
IN (String) (((jsonb_typeof(to_jsonb("props"->'brand')) = 'array' AND to_jsonb("props"->'brand') @> jsonb_build_array('Dettol')) OR (jsonb_build_array("props"->'brand') @> jsonb_build_array('Dettol'))) OR ((jsonb_typeof(to_jsonb("props"->'brand')) = 'array' AND to_jsonb("props"->'brand') @> jsonb_build_array('Lifebuoy')) OR (jsonb_build_array("props"->'brand') @> jsonb_build_array('Lifebuoy')))) "props"->>'brand' IN ('Dettol', 'Lifebuoy')
NOT_IN (String) "props"->'brand' IS NULL OR NOT ((((jsonb_typeof(to_jsonb("props"->'brand')) = 'array' AND to_jsonb("props"->'brand') @> jsonb_build_array('Dettol')) OR (jsonb_build_array("props"->'brand') @> jsonb_build_array('Dettol'))))) "props"->'brand' IS NULL OR NOT ("props"->>'brand' IN ('Dettol'))
GT (Number) CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) > 500000 CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) > 500000
LT (Number) CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) < 500000 CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) < 500000
GTE (Number) CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) >= 700000 CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) >= 700000
LTE (Number) CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) <= 400004 CAST("props"->'seller'->'address'->>'pincode' AS NUMERIC) <= 400004
CONTAINS (Array) "props"->'colors' @> '["Green"]'::jsonb "props"->'colors' @> '["Green"]'::jsonb
NOT_CONTAINS (Array) "props"->'colors' IS NULL OR NOT "props"->'colors' @> '["Green"]'::jsonb "props"->'colors' IS NULL OR NOT "props"->'colors' @> '["Green"]'::jsonb

Testing

  1. Added integration tests.
  2. Tested in a live environment.

@suddendust suddendust changed the title [Draft] Optimise IN queries for json fields in flat collections Optimise IN queries for json fields in flat collections Nov 21, 2025
@codecov
Copy link

codecov bot commented Nov 21, 2025

Codecov Report

❌ Patch coverage is 53.26087% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.70%. Comparing base (d411cbc) to head (01c40e0).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...ter/PostgresInRelationalFilterParserJsonArray.java 3.84% 25 Missing ⚠️
...PostgresInRelationalFilterParserJsonPrimitive.java 76.92% 3 Missing and 3 partials ⚠️
...ery/v1/parser/filter/PostgresInParserSelector.java 58.33% 4 Missing and 1 partial ⚠️
.../v1/parser/filter/PostgresNotInParserSelector.java 54.54% 4 Missing and 1 partial ⚠️
...tore/expression/impl/JsonIdentifierExpression.java 75.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #252      +/-   ##
============================================
- Coverage     80.50%   79.70%   -0.80%     
+ Complexity     1210     1209       -1     
============================================
  Files           224      226       +2     
  Lines          5626     5701      +75     
  Branches        487      492       +5     
============================================
+ Hits           4529     4544      +15     
- Misses          753      809      +56     
- Partials        344      348       +4     
Flag Coverage Δ
integration 79.70% <53.26%> (-0.80%) ⬇️
unit 57.32% <32.60%> (-0.52%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

// The LHS parser generates: "props"->'brand' (returns JSONB)
// We need: "props"->>'brand' (returns TEXT)
// Replace the last -> with ->> for primitive type extraction
int lastArrowIndex = parsedLhs.lastIndexOf("->");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is tweaking the parser to not do this a bigger lift?

@suresh-prakash suresh-prakash merged commit c7bb6ad into hypertrace:main Nov 21, 2025
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants