[Bug] Nydusify conversion loses files or mismatches file size/hash in large images

[J-jxr]

Problem Description

When reconverting large images such as Python and MySQL using nydusify, the converted OCI images have missing files or mismatched file sizes and hashes. This leads to failures when running the converted images and errors in the nydusify check command. The issue is suspected to be related to the implementation of the nydus-image create command.

Expected Behavior

• The nydusify check command should not report file mismatches (size, hash, or missing files) between the source Nydus image and the target OCI-converted image.
• Converted OCI images (e.g., Python, MySQL) should run successfully without missing files or execution errors.

Actual Behavior

• Check Command Errors: When running nydusify check on large images like Python and MySQL, there are reports of file mismatches (different sizes, hashes) and missing files. For example, the Python image has mismatched cache files and missing symlinks, while the MySQL image lacks the /docker-entrypoint-initdb.d/ directory.
• Runtime Failures:
  • The converted Python OCI image fails to run with an "executable file not found" error due to missing symlinks.
  • The converted MySQL OCI image exits immediately with an error indicating the missing /docker-entrypoint-initdb.d/ directory.
• Relevant Logs/Screenshots:
  • For python:
    • FATA[2025-10-20T09:24:44+08:00] validate filesystem failed: file not match with mismatched file size and hash for /var/cache/debconf/config.dat-old.
    • docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "python": executable file not found in $PATH: unknown.
    • 
  • For mysql:
    • ls: cannot access '/docker-entrypoint-initdb.d/': No such file or directory in container logs.
    • Container status shows Exited (2).
    • 

How to reproduce

How to reproduce

1. Start a local Docker registry (to store source, Nydus, and converted OCI images):

   sudo docker run -d -p 5000:5000 --name registry registry:2

2. Pull the official Python image and push it to the local registry:

   # Pull base Python image
   sudo docker pull python:3.11-slim
   # Tag image for local registry
   sudo docker tag python:3.11-slim localhost:5000/python:3.11-slim
   # Push to local registry (insecure mode)
   sudo docker push localhost:5000/python:3.11-slim

3. Convert the source OCI image to Nydus format:

   sudo ./cmd/nydusify convert \
     --source localhost:5000/python:3.11-slim \
     --target localhost:5000/python:nydus \
     --source-insecure \
     --target-insecure \
     --nydus-image /usr/bin/nydus-image \
     --work-dir ./tmp

4. Reverse-convert the Nydus image back to OCI format:

   sudo ./cmd/nydusify convert \
     --source localhost:5000/python:nydus \
     --target localhost:5000/python:oci-converted \
     --source-insecure \
     --target-insecure \
     --reverse \
     --nydus-image /usr/bin/nydus-image \
     --work-dir ./tmp \
     --output-json ./python-conversion.json \
     --plain-http

5. Validate consistency between the Nydus source and converted OCI image:

   sudo ./cmd/nydusify check \
     --source localhost:5000/python:nydus \
     --target localhost:5000/python:oci-converted \
     --source-insecure \
     --target-insecure \
     --nydus-image /usr/bin/nydus-image \
     --nydusd /usr/bin/nydusd

   Observe the check command output for errors like missing files or size/hash mismatches.

6. Attempt to run the converted OCI image to verify runtime functionality:

   sudo docker run --name python-test \
     localhost:5000/python:oci-converted \
     python -c "import sys; print(f'Python version: {sys.version}')"

   Observe the runtime error (e.g., "executable file not found" due to missing symlinks).

7. (Optional) Clean up resources after reproduction (to avoid environment pollution):

   # Remove test container
   sudo docker rm -f python-test
   # Delete converted images from local registry
   sudo docker rmi localhost:5000/python:nydus localhost:5000/python:oci-converted
   # Stop and remove local registry
   sudo docker stop registry && sudo docker rm registry
   # Clean up residual local images
   sudo docker rmi $(sudo docker images -q localhost:5000/python) 2>/dev/null

Note: The same reproduction steps apply to other large images (e.g., MySQL) — replace python:3.11-slim with mysql:latest (and adjust the runtime command to mysql --version for validation).

Environment Details

• Nydus-snapshotter version:
• Nydus version:
• Container runtime: Docker
• Operating System: Linux
• Kernel version:

Additional Information

• Small images do not exhibit this issue; the problem is specific to large images like Python and MySQL.
• The issue likely originates from the nydus-image create command implementation, affecting file integrity during conversion.

Are you willing to submit PR?

• Yes I am willing to submit a PR!