Multiple shards issue

[brayan9413]

Hello, I'm trying to use the connector for AWS elasticache Redis (cluster mode enabled) with 3 shards/primary nodes and 3 replicas for every shard (9 nodes), but I cannot see all my queues in the dashboard connection,
When I refresh the connector, I see different queues every time, it seems like the connector is getting the queues only for 1 shard at the same time

doesn't this connector support multiple shards?

[manast]

The connector does not support clusters at the moment. If you cannot use a direct Redis connection, a possibility could be to use https://ngrok.com/.

[brayan9413]

Thanks for the answer @manast,
I tried with the direct connection and AWS network load balancer publicly exposed and the behavior is the same, so that means the taskforce dashboard doesn't support shards setup even for direct connections.

I found this doc
https://docs.bullmq.io/bull/patterns/redis-cluster

In summary, to make bull compatible with Redis cluster, use a queue prefix inside brackets

const queue = new Queue('cluster', { prefix: '{myprefix}' });

However, it isn't clear to me, it looks like the queues will be always in the same node
so taskforce dashboard can only access one node as far as I can see 🤔

[manast]

Oh yes, you are right. It is not possible to specify a list of nodes in the connection at the moment, I will mark this issue as an enhancement. As you say, there is no advantage in using a cluster for BullMQ, the only reason you would need to do that is if you are using the cluster for something else and you want to reuse it for BullMQ as well. Cluster may also give you better availability guarantees, but I am not an expert on the subject.