In-process benchmarking #370
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…on times instead of a cut-off on current time
…tics, like std-dev
…nner is not legacy
… status dot printing
PEZ
added
levenshtein
This was referenced Jan 25, 2025
benknoble
reviewed
Jan 26, 2025
| You'll see that the `benchmark/run` function takes two arguments: | ||
|
|
||
| 1. `f`: A function (a thunk) | ||
| 1. `run-ms`: A total time in milliseconds within which the function should be run as many times as possible |
Contributor
There was a problem hiding this comment.
Please make sure you're using a monotonic clock for these measurements (wall clock time can jump around).
I don't know how this actually works in other languages, but in Racket you can provide a time-limit for a body of execution (see call-with-time-limit).
Collaborator
Author
There was a problem hiding this comment.
Good point. I made sure it was a monotonic clock for Java and Clojure, but I shall now check the other language implementations. It's probably fine, but I didn't pay attention to that so could have slipped in some wall clocking.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Couldn't reopen #365, so a new PR. 😄
I have:
run.shscript, from before and after the change.Description of changes
This is the start of what I said in #341 that I should try to implement in-process benchmarking for Clojure, Java, and C. Starting with Clojure, to get it as right as possible before cloning the approach to any other languages.
I think the changes are best described in the New runner section in the updated README.md, plus:
Addressing: