Add flat files to conformance tests#7853
Open
kwxm wants to merge 14 commits into
Open
Conversation
kwxm
added
Test
Conformance
No Changelog Required
kwxm
changed the title
Options were parsed against an empty test tree (to learn --format before test discovery), but tasty only accepts a provider option like tasty-golden's --no-create if the tree contains such tests, so the parse failed. Instead, discover the tests for both formats and let askOption select the right tree, parsing the options once, against the full tree.
zliu41
reviewed
Jul 17, 2026
| If a textual `.uplc` file is expected to fail to parse correctly then the `.uplc.expected` file will contain the string "parse/decode error" | ||
|
|
||
| #### "evaluation failure" | ||
| If the file is expected to parse correctly but an error is expected to occur during evaluation then the `.uplc.expected` file will contain the string "parse/decode error". |
| instance IsOption Format where | ||
| defaultValue = Textual | ||
| parseValue s = case s of | ||
| "uplc" -> Just Textual |
Member
There was a problem hiding this comment.
Why "uplc" not "textual"? This is inconsistent with the help text below.
| unDeBruijnTerm tmEvaluated of | ||
| Left _ -> Nothing | ||
| Right namedTerm -> Just $ UPLC.Program () version namedTerm | ||
| Left _ -> EvalFailure -- Shouldn't happen unless there's something wrong with the Agda code. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds
.flatand.flat.expectedfiles for most of the conformance tests to make things easier for other users.haskell-conformancestill use the textual format; however you can use the--format flatoption to get them to use the.flatfiles instead..budget.expected(it was previously.uplc.budget.expected, but the flat files will require the same budgets).constantdirectories toparserbecause those tests are really testing the parser and don't have any flat equivalent.conformance-consistencytest to check that every .uplc file has a corresponding .flat file and vice versa (but see the next point) and that the uplc and flat files decode to identical ASTs (and similarly for the results).conformance-consistencycontains a list of directories/files to skip to deal with the .uplc files that don't have corresponding .flat files.test-cases/uplc/evaluation/builtin/parser(in particular, some of thevalueones) to flat, but that'll have to be done by manufacturing the flat encodings directly (ie, not usinguplc convert, since it'll fail to parse them). This will be a bit tricky, so I'll come back to it in a separate issue.test-cases/uplc/evaluation/term/varbecause of some problems with free variables: I'll come back and look at this later.If something goes wrong during execution of a
.flatfile then the.flat.expectedfile is empty. I'm undecided about whether this is a good idea: maybe it should just contain text "parse/decode" error or "evaluation failure" like the other two.expectedfiles.