Skip to content

Conversation

@sundy-li
Copy link
Member

@sundy-li sundy-li commented Jan 13, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

pick #17245

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

…s#17245)

* fix(query): hash table scatter will always send agg meta

* fix(query): hash table scatter will always send agg meta

* fix(query): z

* fix(query): z

* update

* update

* update

* z

* z

* z

* z

* z

* z

* z
…17252)

* fix(query): increase state_rows in copy agg state rows

* fix(query): increase state_rows in copy agg state rows

* fix(query): increase state_rows in copy agg state rows
@github-actions github-actions bot added pr-backport pr-bugfix this PR patches a bug in codebase labels Jan 13, 2025
@what-the-diff
Copy link

what-the-diff bot commented Jan 13, 2025

PR Summary

  • Enhanced Payload Management: The way we manage data (payload) has been refined to keep track of state rows in addition to regular rows. This adds more precision to our data handling.
  • Adaptable Empty Block Creation: Making empty blocks is now more flexible as it can work with an optional parameter, 'fake_rows'. This allows us to create blocks populated with default values if needed.
  • Consistency in Payload Flushing: Payload flushing, the process of cleaning out data, has been made more consistent by uniformly passing 'None' to an associated function 'empty_block()'.
  • Block Pushing Streamlined: The process of pushing empty blocks to a collection (vector) has been simplified by removing unnecessary conditions, leading to cleaner code.
  • Inclusion of Empty Block Checks: New checks have been added to correctly handle data situations where a block might be empty, ensuring accurate data management.
  • Extended Tracking for Aggregate Metadata: A new tracking field 'is_empty' has been introduced to the metadata of aggregate data to record whether a block of data is empty. This contributes to more granular data tracking.
  • Serialization Strategy Refresh: The process of converting data into a stable format (serialization) has been updated to make sure there's always at least one block returned. To do this, it uses the aforementioned 'is_empty' state.
  • Intersection Check for Zero-Length Blocks: To avoid unnecessary computing, added a check that will dismiss processing blocks if they have zero-length.
  • Early Exit for Transformations with Empty Input: Implemented a feature where further processing is prevented if the input for a transformation is empty. This helps avoid computational resources on trivial cases.
  • Test Scripts Added: Introducing new testing tools. A test script called '20_0022_agg_memory.sh' is added which tests memory usage and its results are stored in '20_0022_agg_memory.result'.

@sundy-li sundy-li requested a review from zhang2014 January 14, 2025 02:35
@zhang2014 zhang2014 merged commit 4c4896d into databendlabs:backport/v1.2.680 Jan 14, 2025
72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-backport pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants