Skip to content

Conversation

@fforbeck
Copy link
Member

@fforbeck fforbeck commented Oct 29, 2025

Migration Verification

Context

After implementing the three core migration steps (index generation, location claims republishing, and gateway authorization), we needed a way to verify that migrations actually succeeded. Without verification, we had no automated way to confirm that the indexing service received and stored the claims correctly.

Changes

New Verification Function

Added the verifyMigration function that re-queries the indexing service after migration to confirm:

  • Index claim exists for the root CID
  • Location claims exist for all shards
  • Location claims include the correct space DID

The function provides detailed per-shard verification status and returns a structured result indicating success or failure with specific details about what went wrong.

Integrated into Migration Workflow

Verification now runs automatically as Step 5 after all migration steps are complete. The migration is marked as successful only if verification passes. If verification fails, the migration is marked as completed with issues, and detailed error information is logged.

Verify-Only Mode

Added --verify-only CLI flag that skips all migration steps and only runs verification. This is useful for:

  • Checking migration status without making changes
  • Auditing previously migrated content
  • Filtering by space, customer, or CID to verify specific uploads

Summary Statistics

Migration summary now includes verification statistics showing how many uploads passed or failed verification, making it easy to identify issues at scale.

Testing

# Verify without making changes
node src/migrate.js --verify-only --space did:key:z6Mk... --limit 10

# Full migration with automatic verification
node src/migrate.js --limit 10

Limitations

Gateway authorization verification is not included as it would require querying the gateway's CloudFlare KV store. We rely on the access/delegate invocation response to confirm that delegations were stored.

Closes storacha/project-tracking#595
Closes storacha/project-tracking#597

@fforbeck fforbeck requested a review from alanshaw October 29, 2025 13:57
@fforbeck fforbeck self-assigned this Oct 29, 2025
@fforbeck fforbeck changed the base branch from main to feat/gateway-auth October 29, 2025 14:00
@fforbeck fforbeck requested review from alanshaw and Copilot and removed request for alanshaw and Copilot October 29, 2025 15:15
* feat: migration monitoring

* fixes

* minor fixes

* chore: reuse dynamodb connection
@fforbeck fforbeck merged commit 34b6f05 into feat/gateway-auth Nov 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants