Optimizing ESHOPMAN Cache Invalidation: Preventing Redis Bottlenecks in High-Volume Stores

Ensuring Smooth ESHOPMAN Operations: A Deep Dive into Redis Caching Performance

As an e-commerce expert at Move My Store, we often see the challenges that come with scaling headless commerce platforms. ESHOPMAN, with its robust Node.js/TypeScript backend and seamless HubSpot integration for storefront management, is designed for high performance. However, even the most advanced systems can encounter bottlenecks, especially when dealing with massive product catalogs and frequent data updates.

A recent community discussion highlighted a critical performance concern related to ESHOPMAN's Redis caching module, particularly affecting stores with hundreds of thousands of products and frequent cache invalidations. This insight is crucial for ESHOPMAN developers and merchants aiming for optimal performance and scalability.

The Challenge: Redis Blocking During Cache Invalidation

For ESHOPMAN stores managing over 300,000 products, routine operations like scheduled product imports or repricing jobs often trigger numerous product.updated events. These events, in turn, lead to frequent cache invalidations via the ESHOPMAN Redis caching module. While essential for data consistency, this process can inadvertently cause significant performance degradation.

The core issue was traced to the Redis KEYS command being used within the clear() method of the caching module. When a large number of cached tags (e.g., 330K+) accumulate, executing KEYS mc:tag:* can block the single-threaded Redis server for extended periods—ranging from 60-130ms, with observed peaks over 1 second. This blocking behavior starves other Redis clients, leading to connection errors and significant latency spikes, which manifest as warnings in the ESHOPMAN background worker logs:

{"level":"warn","message":"Redis connection error during clear operation, relying on IORedis retry mechanism","timestamp":"2026-02-18 04:03:08"}

During these operations, the Redis client output buffer can also swell, peaking at around 21MB, further indicating the strain on the system.

Why is KEYS Problematic?

The KEYS command is designed to scan the entire keyspace, which is an O(N) operation (where N is the number of keys). In a production environment with a large dataset, this command can be extremely slow and should be avoided. It's a well-known Redis best practice that KEYS should never be used in production due to its blocking nature.

The problem was identified in two specific locations within the ESHOPMAN Redis caching module's clear() method:

https://github.com/medusajs/medusa/blob/develop/packages/modules/providers/caching-redis/src/services/redis-cache.ts#L441
https://github.com/medusajs/medusa/blob/develop/packages/modules/providers/caching-redis/src/services/redis-cache.ts#L599

Interestingly, the flush() method in the same module correctly uses the non-blocking SCAN command for iterating over keys, demonstrating the preferred approach:

https://github.com/medusajs/medusa/blob/develop/packages/modules/providers/caching-redis/src/services/redis-cache.ts#L660

The ESHOPMAN Community Solution & Best Practice

For ESHOPMAN deployments, especially those leveraging HubSpot CMS for high-traffic storefronts and relying on the Node.js Admin API for frequent data synchronization, ensuring Redis stability is paramount. The expected behavior for cache tag invalidation is that it should not block the Redis server.

The clear solution, aligned with Redis best practices, is to replace the blocking KEYS command with the iterative, non-blocking SCAN command in the affected clear() methods. This change would allow cache invalidations to occur efficiently without monopolizing Redis resources, thereby preventing connection errors and maintaining high performance for your ESHOPMAN store.

Actionable Advice for ESHOPMAN Users: If you're running a high-volume ESHOPMAN store with frequent product updates, monitor your Redis slowlog (CONFIG SET slowlog-log-slower-than 5000) for occurrences of KEYS mc:tag:*. While the ESHOPMAN core development team addresses this, being aware of this behavior and its impact on your infrastructure is key to maintaining a responsive and scalable headless commerce experience.

This insight underscores the importance of robust caching strategies for ESHOPMAN's headless architecture, ensuring seamless integration and performance across your HubSpot-powered digital storefront.