filter anyOf and oneOf alternative#180
filter anyOf and oneOf alternative#180animeshsahoo1 wants to merge 5 commits intohyperjump-io:mainfrom
Conversation
|
hi @jdesrosiers I have implemented what we discussed about, I tried to not add anything unnecessary but if there is need of changes let me know, I made sure to run the linting and tests as well. |
jdesrosiers
left a comment
There was a problem hiding this comment.
Please add tests. They need to cover all the edge cases we discussed and more if you can think of them.
|
I did run it on tests we talk about but I was unsure if I had to add the code for that, I will add it soon. |
|
I have added the test case for matchcount=0 for oneOf and anyOf test cases that were discussed, I tried to think of a failing case but didn't really find any let me know if I could add any other test case |
|
@jdesrosiers tagging you in case you didn't get notification since I didn't tag last time |
|
You don't need to tag me. I'll get the notification. In fact, bumping the issue will likely make it take longer before I get to it. I generally work on the most stale issue on my plate first. So, if you bump the issue, it effectively moves your issue to the back of the queue. |
jdesrosiers
left a comment
There was a problem hiding this comment.
Please try to match the code style used in the rest of the project. Please turn off Prettier or at least give it a much more reasonable line length limit.
I think this works! But, getting it to work is just the first step. This needs some refactoring.
Every time you do a filter operation, you're looping over everything again and creating a new array. Creating a bunch of arrays and objects gets expensive at scale. See if you can refactor so that you can loop over the alternatives only one time.
Also try to avoid things like [...instanceProps]. Again, we're unnecessarily creating an array. Have a look at @hyperjump/pact. It allows you to use filter, map and bunch of other things that work on any iterator (not just arrays) and don't create unnecessary intermediate representations. It looks like it could come in handy in a few places.
src/error-handlers/anyOf.js
Outdated
|
|
||
| /** @type (alternative: NormalizedOutput, propLocation: string) => boolean */ | ||
| const propertyPasses = (alternative, propLocation) => { | ||
| const propOutput = alternative[propLocation]; |
There was a problem hiding this comment.
alernatative and propLocation aren't used individually. Should we just pass propOutput directly instead?
src/error-handlers/anyOf.js
Outdated
| filtered = filtered.filter((alternative) => { | ||
| const typeResults | ||
| = alternative[instanceLocation]?.[ | ||
| "https://json-schema.org/keyword/type" | ||
| ]; | ||
| return ( | ||
| !typeResults || Object.values(typeResults).every((isValid) => isValid) | ||
| ); | ||
| }); | ||
|
|
||
| if (alternatives.length === 0) { | ||
| for (const alternative of anyOf) { | ||
| alternatives.push(await getErrors(alternative, instance, localization)); | ||
| if (filtered.length === 0) { | ||
| filtered = anyOf; | ||
| } |
There was a problem hiding this comment.
Is it right for this to be in an else? It seems to me that we should be doing this check always and doing it first. It's the most generic check and the best chance to filter out potential cases early.
There was a problem hiding this comment.
yeah that's true I focused too much on object cases this should be done at start to filter out many cases
|
hi I have done the changes and used hyperjump pact to avoid creating intermediate representations, let me know if there is still somewhere I should change |
ba77800 to
803ece5
Compare
|
Hi I have fixed the failing npm run lint command with new oxlint linting, please check if its ok. |
src/error-handlers/anyOf.js
Outdated
| filtered = []; | ||
| for (const alternative of anyOf) { | ||
| alternatives.push(await getErrors(alternative, instance, localization)); | ||
| const typeResults = alternative[instanceLocation]["https://json-schema.org/keyword/type"]; | ||
| if (!typeResults || Object.values(typeResults).every((isValid) => isValid)) { | ||
| filtered.push(alternative); | ||
| } | ||
| } |
There was a problem hiding this comment.
This is now duplicated code. You don't need the if/else. Remove the if wrapper and the else block entirely.
There was a problem hiding this comment.
So basically remove if else wrapper put the type check at top for each alternative then if its an object we check rule1 and rule2 right
src/error-handlers/anyOf.js
Outdated
| const declaredProps = Object.keys(alternative) | ||
| .filter((loc) => loc.startsWith(prefix)) | ||
| .map((loc) => /** @type {string} */ (Pact.head(JsonPointer.pointerSegments(loc.slice(prefix.length - 1))))); |
There was a problem hiding this comment.
Use Pact here to avoid creating intermediate arrays.
src/error-handlers/anyOf.js
Outdated
|
|
||
| /** @type (propOutput: NormalizedOutput[string] | undefined) => boolean */ | ||
| const propertyPasses = (propOutput) => { | ||
| if (!propOutput || Object.keys(propOutput).length === 0) { |
There was a problem hiding this comment.
I don't understand the purpose of the check for an empty object and the tests don't cover it. Either remove it or add a test showing it's needed.
There was a problem hiding this comment.
I found a bug. Here's a test that exposes it.
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"anyOf": [
{
"type": "object",
"properties": {
"a": { "type": "string" }
},
"required": ["a"]
},
{
"type": "object",
"properties": {
"b": { "type": "string" }
},
"required": ["b"]
}
]
}{ "a": 42 }The first alternative is getting filtered by the discriminator check (rule 2), but "a" is not a discriminator because it doesn't have a passing value in any other alternatives.
|
yeah this is definitely an edge case I was thinking we should add to rule 2: only apply this filter if the failing properties all have passing constraints in at least one other alternative
so the new rule2 is: |
|
I don't think there's a need to change rule 2. I think you just didn't implement rule 2 as it was defined.
I think what the current implementation is doing is to filter out the alternative if all the properties in the instance fail. It's surprising that the tests were passing. What we need to do is find all the properties that pass in at least one alternative. Then when we filter alternatives, we can filter the alternative if any (not all) of those passing properties are failing in the alternative. |
|
oh ok I think there might have been some misunderstanding cause I am actually knowingly implementing:
In this case the correct behavior should be showing both alternative but if we go by step1 of rule 2 the discriminator property here will be please correct me if I am misunderstanding you. |
It's correct that it's not a discriminator, but there is something that gets filtered out. Alternative two has no declared properties in common with the instance, so it gets filtered out by Rule 1. Rule 2 doesn't apply in this example.
Right. "type" filters out alternative 2 and "x" filters out alternative 1.
No. It's expected that sometimes everything will be filtered out due to contradicting values. In that case we fall back to showing all the alternatives. In this case we expect all the alternatives to be shown not because nothing filtered out, but because everything filtered out. |
|
yes I forgot to apply rule1 in the first case since we are talking about rule2, I tried to think of case where it could be wrong but didn't come up with anything, I understand it now I will implement it, if I do come up with any case in future I will bring it up. |
jdesrosiers
left a comment
There was a problem hiding this comment.
This has gotten a bit messy again. I see the same problem as before where you're looping over things multiple times. I don't think we can get away with looping over the alternatives less than three times, but we certainly shouldn't be doing it more than that. I expect one loop to determine discriminators, another to filter alternatives, and a third if all the alternatives got filtered out. Right now you have two loops for filtering.
Also, I see that you removed the use of Pact.pipe. I found that considerably harder to read.
|
I am still getting used to using Pact but I will use it, I will refactor to match the coding style again |
The anyof and oneof error handlers now filter alternatives based on the instance's properties. For object instances, an alternative is excluded from the error report if none of its declared properties exist on the instance (Rule 1), or if none of the instance's properties pass validation in that alternative (Rule 2). For non-object instances, the original type-keyword filtering is done.
fixes #175