HDDS-13841. Namespace summary API gives wrong count of directories and keys.

[ArafatKhan2198]

What changes were proposed in this pull request?

A detailed Explanation on how the Issue was caught - https://issues.apache.org/jira/browse/HDDS-13841#:~:text=How%20We%20Uncovered%20This%20Issue

Bug:

Namespace summaries displayed incorrect file sizes and counts for directories. When a directory containing files was added to the tree, its immediate parent directory was not updated with the correct totals.

Root Cause:

The handlePutDirEvent method called propagateSizeUpwards(parentObjectId, ...) to propagate size changes. However, propagateSizeUpwards(objectId, ...) updates the parent of the given objectId, not the objectId itself.

By passing parentObjectId, the method updated:

• The grandparent directory (parent's parent)
• Great-grandparent and all ancestors above
• But skipped the immediate parent directory entirely

This caused the immediate parent to have missing or incorrect size and file count totals.

Fix:

Changed handlePutDirEvent to call propagateSizeUpwards(objectId, ...) instead, passing the child directory's objectId. This ensures the immediate parent and all ancestors are updated correctly.

Also simplified handleDeleteDirEvent to follow the same pattern, removing redundant code and ensuring consistency across both methods.

Impact:

Namespace summaries now display accurate file sizes and counts for all directories, with correct propagation through the entire parent chain.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13841

How was this patch tested?

Tested locally by creating a nested directory structure with files and verifying namespace summary totals at each level.

Test commands:

Using ozone fs

fallocate -l 10M testfile
ozone fs -mkdir -p ofs://om/volume2/fso-bucket1/dir1/dir2/dir3
ozone fs -put -f testfile ofs://om/volume2/fso-bucket1/dir1/file1
ozone fs -put -f testfile ofs://om/volume2/fso-bucket1/dir1/file2
ozone fs -put -f testfile ofs://om/volume2/fso-bucket1/dir1/dir2/file3
ozone fs -put -f testfile ofs://om/volume2/fso-bucket1/dir1/dir2/file4
ozone fs -put -f testfile ofs://om/volume2/fso-bucket1/dir1/dir2/dir3/file5
ozone fs -put -f testfile ofs://om/volume2/fso-bucket1/dir1/dir2/dir3/file6

Using ozone sh

ozone sh volume create /volume2
ozone sh bucket create --layout=FILE_SYSTEM_OPTIMIZED /volume2/fso-bucket2
fallocate -l 10M /tmp/samplefile

ozone sh key put /volume2/fso-bucket2/dir1/testfile1 /tmp/samplefile
ozone sh key put /volume2/fso-bucket2/dir1/testfile2 /tmp/samplefile
ozone sh key put /volume2/fso-bucket2/dir1/dir2/testfile1 /tmp/samplefile
ozone sh key put /volume2/fso-bucket2/dir1/dir2/testfile2 /tmp/samplefile
ozone sh key put /volume2/fso-bucket2/dir1/dir2/dir3/testfile1 /tmp/samplefile
ozone sh key put /volume2/fso-bucket2/dir1/dir2/dir3/testfile2 /tmp/samplefile

Verified structure with correct namespace summary totals:

volume2/fso-bucket1/                    (6 files, 60 MB total)
└── dir1/                               (6 files, 60 MB total)
    ├── file1                           (1 file, 10 MB)
    ├── file2                           (1 file, 10 MB)
    └── dir2/                           (4 files, 40 MB total)
        ├── file3                       (1 file, 10 MB)
        ├── file4                       (1 file, 10 MB)
        └── dir3/                       (2 files, 20 MB total)
            ├── file5                   (1 file, 10 MB)
            └── file6                   (1 file, 10 MB)

Before the fix: dir1 would show incorrect totals (missing counts/sizes from subdirectories)

After the fix: All directories show correct cumulative file counts and sizes, including all nested subdirectories.

[priyeshkaratha]

Thanks @ArafatKhan2198 for the patch. Code looks fine for me.

[devmadhuu]

Thanks @ArafatKhan2198 for the patch. Just few nits. Pls check.

1. Also add in PR description for ozone sh put commands testing.
2. And mention the difference in detail in PR description why this bug surface out because of difference in events from OM in ozone fs -mkdir and ozone sh put commands along with the full sequence of list of events.
3. Also check CI test failures of Recon.

[devmadhuu]

Can we also add few integration tests for both approaches including ozone fs -mkdir and ozone sh put

[sumitagrawl]

LGTM

[ArafatKhan2198]

A detailed Explanation on how the Issue was caught - https://issues.apache.org/jira/browse/HDDS-13841#:~:text=How%20We%20Uncovered%20This%20Issue

Can we also add few integration tests for both approaches including ozone fs -mkdir and ozone sh put
As discussed we can add those in a follow up PR which will be tracked by - https://issues.apache.org/jira/browse/HDDS-13854

[devmadhuu]

Thanks @ArafatKhan2198 for improving the patch. LGTM +1