feat: verify migration steps #6
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Migration Verification
Context
After implementing the three core migration steps (index generation, location claims republishing, and gateway authorization), we needed a way to verify that migrations actually succeeded. Without verification, we had no automated way to confirm that the indexing service received and stored the claims correctly.
Changes
New Verification Function
Added the
verifyMigrationfunction that re-queries the indexing service after migration to confirm:The function provides detailed per-shard verification status and returns a structured result indicating success or failure with specific details about what went wrong.
Integrated into Migration Workflow
Verification now runs automatically as Step 5 after all migration steps are complete. The migration is marked as successful only if verification passes. If verification fails, the migration is marked as completed with issues, and detailed error information is logged.
Verify-Only Mode
Added
--verify-onlyCLI flag that skips all migration steps and only runs verification. This is useful for:Summary Statistics
Migration summary now includes verification statistics showing how many uploads passed or failed verification, making it easy to identify issues at scale.
Testing
Limitations
Gateway authorization verification is not included as it would require querying the gateway's CloudFlare KV store. We rely on the access/delegate invocation response to confirm that delegations were stored.
Closes storacha/project-tracking#595
Closes storacha/project-tracking#597