- Hypothesis List:
* Hypothesis 1: Unbounded internal cache growth in `metricAggregator` module. * Justification: The metricAggregator module likely maintains an in-memory cache of recentlySeenMetricKeys to optimize processing or deduplicate data. If this cache lacks a proper eviction policy (e.g., LRU, TTL) or its size limit is incorrectly configured, it will grow indefinitely as new unique metric keys are encountered over time in a long-running worker. * Hypothesis 2: Unremoved event listeners in `EventEmitter` instances. * Justification: The metricAggregator module uses EventEmitter for inter-module communication. If listeners are added (e.g., emitter.on()) but not subsequently removed (emitter.removeListener() or emitter.off()) when the associated component or task completes, the closures created by these listeners can retain references to larger scopes, preventing garbage collection of objects they reference. * Hypothesis 3: Retained references within long-lived closures from async operations. * Justification: Asynchronous operations, particularly those involving Promises or callbacks that are chained or held onto, can inadvertently capture and retain references to large data structures or objects from their surrounding scope. If these closures persist longer than expected (e.g., due to unhandled promise rejections or implicit references), the retained objects contribute to the memory leak.
- Ordered Investigation Steps:
* Setup and Instrumentation: 1. Reproduce the leak: Ensure a controlled test environment where the worker process CSV data ingestion and transformation worker can run with representative data and workload, reliably demonstrating the RSS growth. Isolate the data_processor.js module for focused analysis. 2. Install `clinic.js`: npm install -g clinic 3. Instrument the worker: Start the worker using clinic heap to collect heap snapshots. Modify the worker's start script or command to prefix it with clinic heap. For example: clinic heap --on-port <PORT> -- node worker.js (where <PORT> is a port the worker listens on, if applicable, to trigger data collection start/stop). Alternatively, use clinic heap --collect-only -- node worker.js and manually trigger snapshots. * Data Collection: 1. Baseline Snapshot: Immediately after the worker process starts and initializes, take the first heap snapshot. This serves as the baseline (Snapshot A). 2. Intermediate Snapshots: Allow the worker to run under typical load for a period where a noticeable RSS increase occurs (e.g., 30-60 minutes). Take a second snapshot (Snapshot B). 3. Leaking State Snapshot: Continue running the worker until the RSS growth is significant (e.g., 2-4 hours, or when RSS has grown by 100-200MB). Take a final snapshot (Snapshot C). 4. V8 Inspector Snapshots (Alternative/Supplement): For more granular control, attach the V8 inspector (node --inspect worker.js), open Chrome DevTools, navigate to the Memory tab, and use the