Skip to content

Conversation

@randerzander
Copy link
Contributor

@randerzander randerzander commented Jul 12, 2021

Based on, dask/distributed#4987, ran SF1k w/ GC disabled on the scheduler process, and observed total benchmark runtime improvement of about 5s. This was consistent across many runs.

Results from larger scale tests are pending.

Based on, dask/distributed#4987, ran SF1k w/ GC disabled on the scheduler process, and observed total benchmark runtime improvement of about 5s. This was consistent across many runs.
@randerzander randerzander requested review from ChrisJar and ayushdg July 12, 2021 19:54
@randerzander randerzander changed the title Disable GC on scheduler at benchmark start [DON'T MERGE] Disable GC on scheduler at benchmark start Jul 12, 2021
@jakirkham
Copy link
Member

Well the other interesting thing from that thread was pinning the Scheduler to a single CPU. Admittedly that's probably a little hacky (ok for the exploration Gabe is doing). If that is important longer term, we can investigate other ways to set that.

@gjoseph92
Copy link

@jakirkham worth noting that I only did that so the measured CPU times (particularly idle time) were easier to understand on a multi-core machine. The disabling-GC performance boost seemed the same without it. I suppose it could make a difference with memory access performance if the kernel is moving the process around between physical cores, but I don't think we're at the point of worrying about that yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants