@@ -315,6 +315,36 @@ your job does not run this long, or you find you are missing metrics that were r
315315of your job run, then add a five- second sleep before exiting: `time.sleep(5 )` . This will allow time
316316for the metrics to be sent.
317317
318+ # # Design Considerations: High Volume Metrics
319+
320+ This client is stateless, and sends a UDP packet (or unixgram) to `spectatord` each time a meter is
321+ updated. If you are performing high- volume operations, on the order of tens- of- thousands or millions
322+ of operations per second, then you should pre- aggregate your metrics and report them at a cadence
323+ closer to the `spectatord` publish interval of 5 seconds. This will keep the CPU usage related to
324+ `spectator- py` and `spectatord` low (around 1 % or less), as compared to up to 40 % for high- volume
325+ scenarios.
326+
327+ # # Caveats: multiprocessing
328+
329+ This library makes use of `threading.Lock()` to provide lazy acquisition of the socket that communicates
330+ with `spectatord` , and creates a background thread for flushing metrics periodically, if the Line Buffer
331+ is enabled. Therefore, if you want to use this library with `multiprocessing` , you have to make a design
332+ choice about how to use `spectator- py` , to ensure that you do not encounter deadlocks in your code.
333+
334+ * If you use a start method of `fork` or `forkserver` (Linux default), then you need to ensure that a new
335+ `Registry` is created after a process is started, which requires a few extra lines of code.
336+ * If you use the `spawn` start method (macOS and Windows default), then your code can use a " shared" instance
337+ of the `Registry` as written.
338+
339+ In both cases, the overhead cost will be maintaining a separate socket for communicating with `spectatord` , plus
340+ the memory associated with processing metrics for delivery.
341+
342+ See [multiprocessing - Contexts and start methods](https:// docs.python.org/ 3 / library/ multiprocessing.html# contexts-and-start-methods)
343+ for more details from the official Python documentation.
344+
345+ See [fork- is - problematic](https:// github.com/ thatch/ fork- is - problematic) for a detailed explanation of the
346+ challenges associated with forking.
347+
318348# # Debug Metrics Delivery to `spectatord`
319349
320350In order to see debug log messages from `spectatord` , create an `/ etc/ default/ spectatord` file with
@@ -327,15 +357,6 @@ SPECTATORD_OPTIONS="--verbose"
327357This will report all metrics that are sent to the Atlas backend in the `spectatord` logs, which will
328358provide an opportunity to correlate metrics publishing events from your client code.
329359
330- # # Design Considerations - Reporting Intervals
331-
332- This client is stateless, and sends a UDP packet (or unixgram) to `spectatord` each time a meter is
333- updated. If you are performing high- volume operations, on the order of tens- of- thousands or millions
334- of operations per second, then you should pre- aggregate your metrics and report them at a cadence
335- closer to the `spectatord` publish interval of 5 seconds. This will keep the CPU usage related to
336- `spectator- py` and `spectatord` low (around 1 % or less), as compared to up to 40 % for high- volume
337- scenarios.
338-
339360# # Writing Tests
340361
341362To write tests against this library, instantiate an instance of the `Registry` and provide a `Config`
0 commit comments