-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Open
Description
Summary
During a GCP Memorystore for Redis Cluster maintenance, nodes were replaced as part of a create-before-destroy process. During this transition, the client’s in-memory cluster topology became temporarily inconsistent with the actual cluster state, resulting in _determine_nodes() returning a None value for a target node. It resulted in AttributeError: 'NoneType' object has no attribute 'name' originating from this line.
We reproduced this issue by scaling down our Redis cluster.
Environment
- redis-py version: 5.3.1
- Python version: 3.11
- Deployment environment: Google Cloud Memorystore for Redis Cluster
- Topology: multi-node Redis Cluster
- Client type: redis.asyncio.RedisCluster
Full stack trace
builtins.AttributeError: 'NoneType' object has no attribute 'name'
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/ddtrace/_trace/utils_redis.py", line 96, in _instrument_redis_execute_async_cluster_pipeline
yield span
File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/internal/redis/asyncio_patch.py", line 36, in instrumented_async_execute_cluster_pipeline
return await func(*args, **kwargs)
File "/usr/local/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 1488, in execute
File "/usr/local/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 1571, in _execute
AttributeError: 'NoneType' object has no attribute 'name'
Reproduction steps
1. Start a Redis Cluster with multiple shards.
2. Connect using `redis.asyncio.RedisCluster`.
3. Continuously run pipelined commands.
4. Scale up/down cluster
5. Observe intermittent `AttributeError` exceptions from the client.
Expected behavior
redis-py should handle temporary None nodes gracefully — either by:
• retrying the command after topology refresh, or
• raising a more specific error we can use to retry.
Questions
- I see the
_determine_nodes()method changed a lot in the newer versions. Could upgrading the version could fix that? - Is it a known issue or something we should fix in this lib?
Thanks!
Metadata
Metadata
Assignees
Labels
No labels