WIP: log dask task stream and rmm events into results dir

[kevingerman]

Adds a couple of utilities to capture metrics per query triggered in benchmark_runner.py
rmm_monitor logs Cuda allocations from each dask worker
dask_task_logger grabs the tasks into a json file

TBD:

• gate with an environment variable as the logs add up quickly.
• mark rmm logs with worker ID

[randerzander]

Thanks for this work.

Can you maybe include some README changes about this configuration? Do you have a parser that uses these logs or just eyeball them so far?

[kevingerman]

I have another commit pending which fetches the logs into a single df which can then be serialized (parquet?)
Still need an example analysis. Probably go back to a bokeh plot that works as a standalone example.