-
Notifications
You must be signed in to change notification settings - Fork 3
SED-4340 find-usages-of-keyword-and-plan-referenced-by-attribute-is-b… #579
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
SED-4340 find-usages-of-keyword-and-plan-referenced-by-attribute-is-b… #579
Conversation
* SED-4305: unit tests for AutomationPackageManager * SED-4305: forbid libraries in local execution (CLI) * SED-4305: fix test * SED-4305: new unit test * SED-4305: remove redundant todo * SED-4350: new unit test * SED-4350: merge master into SED-4350 * SED-4350: fix test * SED-4350: unit test * SED-4350: unit test (cherry picked from commit 6be0173)
SED-4404: improve error messages (cherry picked from commit 011fbcb)
* SED-4413 added connect and read timeouts to AbstractRemoteClient * SED-4413 PR feedback (cherry picked from commit 984f98e)
…r not released (#567) * SED-4401 Application context of step-functions-handler-initializer.jar not released * SED-4413 added connect and read timeouts to AbstractRemoteClient (#563) * SED-4413 added connect and read timeouts to AbstractRemoteClient * SED-4413 PR feedback (cherry picked from commit 984f98e) * SED-4401 Application context of step-functions-handler-initializer.jar not released * Revert "SED-4413 added connect and read timeouts to AbstractRemoteClient (#563)" This reverts commit 5eff1e9. * SED-4401 Fixing NoClassDefFound * SED-4401 Fixing NPE --------- Co-authored-by: Jonathan Rubiero <30461894+rubij@users.noreply.github.com>
…fter merge of SED-4401 (#563)
#569) SED-4439 Optimize reporting request and query performance for Step 29 branch
… resources of a global project (#568) * SED-4387 delete resource services do not check write access for tenant * SED-4387 improve error message displyed in the UI
* SED-4444 timeseries-re-ingestion-doesnt-work-for-psql * SED-4444 timeseries-re-ingestion-doesnt-work-for-psql
* SED-4443 Fixing ClassCastException in FunctionMessageHandler * SED-4443 Fixing ClassCastException in FunctionMessageHandler
…#575) * SED-4418 Calling Keywords in an after section does not release tokens * SED-4418 Calling Keywords in an after section does not release tokens * SED-4418 Adding a junit test
* SED-4287 ForEach may try to use uninitialized tempWriter * SED-4287 propagating exception
Summary of ChangesHello @david-stephan, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request resolves a defect in the 'find usages' functionality, specifically targeting issues where keywords and plans referenced by attributes were not correctly identified. The changes involve a significant refactoring of the reference finding mechanism, moving the core logic into a new, dedicated class and enhancing the underlying entity dependency traversal. This ensures that the system can now accurately and comprehensively resolve all potential references, improving the reliability of usage analysis for dynamically referenced entities. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces a significant and well-executed refactoring of the reference finding logic to address an issue with locating usages of keywords and plans referenced by attributes. The key improvements include extracting the core logic into a dedicated ReferenceFinder class, introducing a more flexible VISIT_MODE for dependency traversal, and enhancing the PlanLocator and FunctionLocator to support a non-strict resolution mode. This non-strict mode is crucial for finding all potential references without a full execution context. The addition of a comprehensive test suite for the new ReferenceFinder is also a great enhancement.
My review includes a high-severity bug fix for a potential NullPointerException and a couple of medium-severity suggestions to further improve code maintainability and test quality. Overall, these are excellent changes that effectively solve the underlying problem.
step-controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
Show resolved
Hide resolved
| List<Object> referencedObjects = getReferencedObjects(entityType, object).stream().filter(o -> (o != null && !o.equals(object))).collect(Collectors.toList()); | ||
| //System.err.println("objects referenced from plan: " + planToString(plan) + ": "+ referencedObjects.stream().map(ReferenceFinderServices::objectToString).collect(Collectors.toList())); | ||
| return referencedObjects.stream().filter(o -> doesRequestMatch(request, o)).collect(Collectors.toList()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method can be simplified into a single, more efficient stream pipeline that avoids creating an intermediate list. Additionally, there's a commented-out debug line that should be removed as it's no longer relevant and references code that has been moved.
return getReferencedObjects(entityType, object).stream()
.filter(o -> o != null && !o.equals(object))
.filter(o -> doesRequestMatch(request, o))
.collect(Collectors.toList());
...ontroller/step-controller-server/src/test/java/step/core/references/ReferenceFinderTest.java
Show resolved
Hide resolved
jeromecomte
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes look good but I indeed see a risk in the changes related to the FunctionLocator and PlanLocator. The new JUnit test ReferenceFinderTest looks good but I'm not sure if it covers all the aspects of the Locators, especially the ones related to versions. In order to merge this to 29, we should ensure that we cover this properly and add some tests if required
step-core/src/main/java/step/core/entities/EntityDependencyTreeVisitor.java
Show resolved
Hide resolved
step-controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
Outdated
Show resolved
Hide resolved
step-controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
Outdated
Show resolved
Hide resolved
step-controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
Outdated
Show resolved
Hide resolved
step-plans/step-plans-base-artefacts/src/main/java/step/artefacts/handlers/FunctionLocator.java
Outdated
Show resolved
Hide resolved
david-stephan
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review, all points should be addressed with my last commits.
step-controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
Outdated
Show resolved
Hide resolved
step-controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
Outdated
Show resolved
Hide resolved
step-controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
Outdated
Show resolved
Hide resolved
| results.add(new FindReferencesResponse(function)); | ||
| } | ||
| } catch (Exception e) { | ||
| logger.error("Unable to find references for function {}", function.getId(), e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SHouldn't we rethrow this or add the error to the response?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure how relevant that is. The implementation will browse all plans and all composite keywords to see if they use the searched reference, if an exception is raised because the context could not be rebuilt for one of them, I don't think this should break the operation, adding it as a warning would be doable (with quite some effort) but I also don't think this warning would be relevant for the user doing the request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, get it. I would say it depends a bit on when we expect such exceptions to occur. If this an exception that we except, then I would indeed "ignore" it and even log it as worning or even debug. If we don't expect it, I would rethrow it. Let you decide
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked at it again, since the "expected" exceptions can only be thrown when rebuilding the context I moved the error handling to that specific location. The only error case I could see is when plans or composite KWs are orphans (i.e. a project has been deleted with leftover plans or composites).
| results.add(new FindReferencesResponse(plan)); | ||
| } | ||
| } catch (Exception e) { | ||
| logger.error("Unable to find references for plan {}", plan.getId(), e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Idem
* SED-4471 Add audit logging to parameters and schedules * SED-4471 Code simplification * SED-4471 More logging
…580) * SED-4430 Improve Agent and CLI resilience in case of network errors * SED-4430 Reverting method signatures and correct comment * EI-485 new nexus staging host * SED-4430 bumping grid and FW after PR merge --------- Co-authored-by: Jonathan Rubiero <30461894+rubij@users.noreply.github.com>
* SED-4498 Parameters priority not respected when deployed with AP * SED-4498 PR feedbacks * SED-4498 fixing junit
#585) SED-4506 Clean-up of isolated AP resources delete all revisions and files but keep the resource entry in DB
…datasets after re ingestion (#578) * SED-4451 The performance of the analytics view is very poor on large datasets after re-ingestion * SED-4451 The performance of the analytics view is very poor on large datasets after re-ingestion * SED-4451 The performance of the analytics view is very poor on large datasets after re-ingestion * SED-4451 adapting TimeSeriesExecutionPlugin * SED-4451 index optimizations and other perf fixes * SED-4451 bumping framework * SED-4451 PR Feedbacks * SED-4451 adding missing TS index and fixing default resolution * SED-4451 Bumping framework to 0.0.0-25-SNAPSHOT before merge
jeromecomte
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving with on minor comment
| } catch (Exception e) { | ||
| //The getReferencedObjects method is invoked for all entities found in the system, for some entities (for example plans that belongs to a deleted project), the context cannot be rebuilt. | ||
| //These expected errors are ignored | ||
| logger.warn("Unable to inspect the {} with id {}", entityType, object.getId(), e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Admins might not understand in which context this warning occurs. We might give some more details in order to understand that it relates to the find usage
…enced-by-attribute-is-broken
…eferenced-by-attribute-is-broken # Conflicts: # pom.xml
The base branch was changed.
…roken