-
Notifications
You must be signed in to change notification settings - Fork 3
SED-4340 find-usages-of-keyword-and-plan-referenced-by-attribute-is-b… #579
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
david-stephan
merged 33 commits into
master
from
SED-4340-find-usages-of-keyword-and-plan-referenced-by-attribute-is-broken
Feb 3, 2026
Merged
Changes from all commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
f40ae2f
SED-4350 extend automation package test scope (#558)
iegorov777 1deafe9
SED-4350 adapting Junit after merge
david-stephan be23145
SED-4409 Keyword calls with no agent response have no measurements (#…
david-stephan 23a70a3
SED-000 AP Junit test flakiness
david-stephan 7078864
SED-4403 deploy library error messages are not clear enough (#562)
iegorov777 291833f
SED-4428 Uploading an AP defining K6 keywords without scriptDirectory…
jeromecomte 5eff1e9
SED-4413 added connect and read timeouts to AbstractRemoteClient (#563)
rubij 1ff5a3b
SED-4401 Application context of step-functions-handler-initializer.ja…
jeromecomte 2f8e20e
SED-4413 Re-added connect and read timeouts to AbstractRemoteClient a…
jeromecomte 732a312
SED-4439 Optimize reporting request and query performance for Step 29…
david-stephan 480f1f9
SED-4387 from a project it is possible to use bulk deletion to delete…
david-stephan 1a1506a
SED-4444 timeseries-re-ingestion-doesnt-work-for-psql (#570)
david-stephan 0790bc5
SED-4443 ClassCastException in FunctionMessageHandler (#572)
jeromecomte 6098a47
SED-4443 Switching to explicit creation of initializer CL (#574)
jeromecomte 38ea1cf
SED-4445 bumping framework to 2.5.1
david-stephan 2407ee2
SED-4418 Calling Keywords in an after section does not release tokens…
david-stephan b5e41d7
SED-4287 ForEach may try to use uninitialized tempWriter (#576)
david-stephan 174c638
SED-4340 find-usages-of-keyword-and-plan-referenced-by-attribute-is-b…
david-stephan 5498447
SED-4340 PR feedbacks
david-stephan a431643
SED-4340 PR feedbacks
david-stephan e0c4c97
SED-4340 fixing incorrect context and predicate usage
david-stephan 340fc42
SED-4471 Add audit logging to parameters, schedules, resources (#581)
cl-exense 8c255de
EI-485 new nexus staging host
rubij d44013c
SED-4430 Improve Agent and CLI resilience in case of network errors (…
david-stephan 9ad83df
SED-4480 BE Junit test flakiness
david-stephan 443eb0d
SED-4498 Parameters priority not respected when deployed with AP (#584)
david-stephan ff8d80d
SED-4506 Clean-up of isolated AP resources delete all revisions and f…
david-stephan c6183e8
SED-4451 the performance of the analytics view is very poor on large …
david-stephan e459cf6
SED-4340 fixing logging
david-stephan 36fb4ed
SED-4340 fixing error handling
david-stephan 25fd2b8
SED-4430 improving warning message
david-stephan 9a52a2e
Merge branch '29' into SED-4340-find-usages-of-keyword-and-plan-refer…
david-stephan eac1f22
Merge branch 'master' into SED-4340-find-usages-of-keyword-and-plan-r…
david-stephan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
144 changes: 144 additions & 0 deletions
144
...controller/step-controller-server/src/main/java/step/core/references/ReferenceFinder.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| package step.core.references; | ||
|
|
||
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
| import step.core.AbstractContext; | ||
| import step.core.accessors.AbstractOrganizableObject; | ||
| import step.core.entities.EntityConstants; | ||
| import step.core.entities.EntityDependencyTreeVisitor; | ||
| import step.core.entities.EntityManager; | ||
| import step.core.objectenricher.EnricheableObject; | ||
| import step.core.objectenricher.ObjectHookRegistry; | ||
| import step.core.objectenricher.ObjectPredicate; | ||
| import step.core.plans.Plan; | ||
| import step.core.plans.PlanAccessor; | ||
| import step.functions.Function; | ||
| import step.functions.accessor.FunctionAccessor; | ||
| import step.resources.Resource; | ||
|
|
||
| import java.util.*; | ||
| import java.util.stream.Collectors; | ||
| import java.util.stream.Stream; | ||
|
|
||
| public class ReferenceFinder { | ||
|
|
||
| private static final Logger logger = LoggerFactory.getLogger(ReferenceFinder.class); | ||
|
|
||
| private final EntityManager entityManager; | ||
| private final ObjectHookRegistry objectHookRegistry; | ||
|
|
||
| public ReferenceFinder(EntityManager entityManager, ObjectHookRegistry objectHookRegistry) { | ||
| this.entityManager = entityManager; | ||
| this.objectHookRegistry = objectHookRegistry; | ||
| } | ||
|
|
||
| public List<FindReferencesResponse> findReferences(FindReferencesRequest request) { | ||
| if (request.searchType == null) { | ||
| throw new IllegalArgumentException("A valid searchType must be provided"); | ||
| } | ||
| if (request.searchValue == null || request.searchValue.trim().isEmpty()) { | ||
| throw new IllegalArgumentException("A non-empty searchValue must be provided"); | ||
| } | ||
|
|
||
| List<FindReferencesResponse> results = new ArrayList<>(); | ||
|
|
||
| PlanAccessor planAccessor = (PlanAccessor) entityManager.getEntityByName(EntityConstants.plans).getAccessor(); | ||
|
|
||
| // Find composite keywords containing requested usages; composite KWs are really just plans in disguise :-) | ||
| FunctionAccessor functionAccessor = (FunctionAccessor) entityManager.getEntityByName(EntityConstants.functions).getAccessor(); | ||
| try (Stream<Function> functionStream = functionAccessor.streamLazy()) { | ||
| functionStream.forEach(function -> { | ||
| List<Object> matchingObjects = getReferencedObjectsMatchingRequest(EntityConstants.functions, function, request); | ||
| if (!matchingObjects.isEmpty()) { | ||
| results.add(new FindReferencesResponse(function)); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| // Find plans containing usages | ||
| try (Stream<Plan> stream = (request.includeHiddenPlans) ? planAccessor.streamLazy() : planAccessor.getVisiblePlans()) { | ||
| stream.forEach(plan -> { | ||
| List<Object> matchingObjects = getReferencedObjectsMatchingRequest(EntityConstants.plans, plan, request); | ||
| if (!matchingObjects.isEmpty()) { | ||
| results.add(new FindReferencesResponse(plan)); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| // Sort the results by name | ||
| results.sort(Comparator.comparing(f -> f.name)); | ||
| return results; | ||
| } | ||
|
|
||
| private List<Object> getReferencedObjectsMatchingRequest(String entityType, AbstractOrganizableObject object, FindReferencesRequest request) { | ||
| return getReferencedObjects(entityType, object, request.searchValue).stream() | ||
| .filter(o -> (o != null && !o.equals(object))) | ||
| .filter(o -> doesRequestMatch(request, o)) | ||
| .collect(Collectors.toList()); | ||
| } | ||
|
|
||
| // returns a (generic) set of objects referenced by a plan | ||
| private Set<Object> getReferencedObjects(String entityType, AbstractOrganizableObject object, String searchValue) { | ||
| Set<Object> referencedObjects = new HashSet<>(); | ||
|
|
||
| // The references can be filled in two different ways due to the implementation: | ||
| // 1. by (actual object) reference in the tree visitor (onResolvedEntity) | ||
| // 2. by object ID in the tree visitor (onResolvedEntityId) | ||
|
|
||
| // When searching the references of a give entity we must apply the predicate as if we were in the context of this entity | ||
| ObjectPredicate predicate = o -> true; //default value for non enricheable objects | ||
| if (object instanceof EnricheableObject) { | ||
| AbstractContext context = new AbstractContext() {}; | ||
| try { | ||
| objectHookRegistry.rebuildContext(context, (EnricheableObject) object); | ||
| } catch (Exception e) { | ||
| //The getReferencedObjects method is invoked for all entities found in the system, for some entities (for example plans that belongs to a deleted project), the context cannot be rebuilt. | ||
| //These expected errors are ignored | ||
| logger.warn("Unable to inspect the {} with id {} while searching for usages of {}", entityType, object.getId(), searchValue, e); | ||
| return referencedObjects; | ||
| } | ||
| predicate = objectHookRegistry.getObjectPredicate(context); | ||
| } | ||
| EntityDependencyTreeVisitor entityDependencyTreeVisitor = new EntityDependencyTreeVisitor(entityManager, predicate); | ||
| FindReferencesTreeVisitor entityTreeVisitor = new FindReferencesTreeVisitor(entityManager, referencedObjects); | ||
| entityDependencyTreeVisitor.visitEntityDependencyTree(entityType, object.getId().toString(), entityTreeVisitor, EntityDependencyTreeVisitor.VISIT_MODE.RESOLVE_ALL); | ||
|
|
||
| return referencedObjects; | ||
| } | ||
|
|
||
| private boolean doesRequestMatch(FindReferencesRequest req, Object o) { | ||
| if (o instanceof Plan) { | ||
| Plan p = (Plan) o; | ||
| switch (req.searchType) { | ||
| case PLAN_NAME: | ||
| return req.searchValue.equals(p.getAttribute(AbstractOrganizableObject.NAME)); | ||
| case PLAN_ID: | ||
| return p.getId().toString().equals(req.searchValue); | ||
| default: | ||
| return false; | ||
| } | ||
| } else if (o instanceof Function) { | ||
| Function f = (Function) o; | ||
| switch (req.searchType) { | ||
| case KEYWORD_NAME: | ||
| return req.searchValue.equals(f.getAttribute(AbstractOrganizableObject.NAME)); | ||
| case KEYWORD_ID: | ||
| return f.getId().toString().equals(req.searchValue); | ||
| default: | ||
| return false; | ||
| } | ||
| } else if (o instanceof Resource) { | ||
| Resource r = (Resource) o; | ||
| switch (req.searchType) { | ||
| case RESOURCE_NAME: | ||
| return req.searchValue.equals(r.getAttribute(AbstractOrganizableObject.NAME)); | ||
| case RESOURCE_ID: | ||
| return r.getId().toString().equals(req.searchValue); | ||
| default: | ||
| return false; | ||
| } | ||
| } else { | ||
| return false; | ||
| } | ||
| } | ||
| } | ||
154 changes: 4 additions & 150 deletions
154
...er/step-controller-server/src/main/java/step/core/references/ReferenceFinderServices.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,182 +1,36 @@ | ||
| package step.core.references; | ||
|
|
||
| import io.swagger.v3.oas.annotations.tags.Tag; | ||
| import step.core.accessors.AbstractOrganizableObject; | ||
| import step.core.deployment.AbstractStepServices; | ||
| import step.core.entities.EntityConstants; | ||
| import step.core.objectenricher.ObjectHookRegistry; | ||
| import step.framework.server.security.Secured; | ||
| import step.core.entities.EntityDependencyTreeVisitor; | ||
| import step.core.entities.EntityManager; | ||
| import step.core.objectenricher.ObjectPredicate; | ||
| import step.core.plans.Plan; | ||
| import step.core.plans.PlanAccessor; | ||
| import step.functions.Function; | ||
| import step.functions.accessor.FunctionAccessor; | ||
| import step.resources.Resource; | ||
|
|
||
| import jakarta.annotation.PostConstruct; | ||
| import jakarta.inject.Singleton; | ||
| import jakarta.ws.rs.*; | ||
| import jakarta.ws.rs.core.MediaType; | ||
|
|
||
| import java.util.*; | ||
| import java.util.stream.Collectors; | ||
| import java.util.stream.Stream; | ||
|
|
||
| @Singleton | ||
| @Path("references") | ||
| @Tag(name = "References") | ||
| public class ReferenceFinderServices extends AbstractStepServices { | ||
|
|
||
| private EntityManager entityManager; | ||
| private ReferenceFinder referenceFinder; | ||
|
|
||
| @PostConstruct | ||
| public void init() throws Exception { | ||
| super.init(); | ||
| entityManager = getContext().getEntityManager(); | ||
| referenceFinder = new ReferenceFinder(getContext().getEntityManager(), getContext().require(ObjectHookRegistry.class)); | ||
| } | ||
|
|
||
|
|
||
| // Uncomment for easier debugging (poor man's Unit Test), URL will be http://localhost:8080/rest/references/findReferencesDebug | ||
| /* | ||
| @GET | ||
| @Path("/findReferencesDebug") | ||
| @Produces(MediaType.APPLICATION_JSON) | ||
| public List<FindReferencesResponse> findReferencesTest() { | ||
| List<FindReferencesResponse> result = new ArrayList<>(); | ||
| result.addAll(findReferences(new FindReferencesRequest(PLAN_NAME, "TestXXX"))); | ||
| // result.addAll(findReferences(new FindReferencesRequest(PLAN_ID, "6195001c0a98d92da8a57830"))); | ||
| result.addAll(findReferences(new FindReferencesRequest(KEYWORD_NAME, "UnitTest"))); | ||
| // result.addAll(findReferences(new FindReferencesRequest(KEYWORD_ID, "60cca3488b81b227a5fe92d9"))); | ||
| return result; | ||
| } | ||
| //*/ | ||
|
|
||
| @Path("/findReferences") | ||
| @POST | ||
| @Secured | ||
| @Produces(MediaType.APPLICATION_JSON) | ||
| @Consumes(MediaType.APPLICATION_JSON) | ||
| public List<FindReferencesResponse> findReferences(FindReferencesRequest request) { | ||
| if (request.searchType == null) { | ||
| throw new IllegalArgumentException("A valid searchType must be provided"); | ||
| } | ||
| if (request.searchValue == null || request.searchValue.trim().isEmpty()) { | ||
| throw new IllegalArgumentException("A non-empty searchValue must be provided"); | ||
| } | ||
|
|
||
| List<FindReferencesResponse> results = new ArrayList<>(); | ||
|
|
||
| PlanAccessor planAccessor = (PlanAccessor) entityManager.getEntityByName(EntityConstants.plans).getAccessor(); | ||
|
|
||
| // Find composite keywords containing requested usages; composite KWs are really just plans in disguise :-) | ||
| FunctionAccessor functionAccessor = (FunctionAccessor) entityManager.getEntityByName(EntityConstants.functions).getAccessor(); | ||
|
|
||
| try (Stream<Function> functionStream = functionAccessor.streamLazy()) { | ||
| functionStream.forEach(function -> { | ||
| List<Object> matchingObjects = getReferencedObjectsMatchingRequest(EntityConstants.functions, function, request); | ||
| if (!matchingObjects.isEmpty()) { | ||
| results.add(new FindReferencesResponse(function)); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| // Find plans containing usages | ||
| try (Stream<Plan> stream = (request.includeHiddenPlans) ? planAccessor.streamLazy() : planAccessor.getVisiblePlans()) { | ||
| stream.forEach(plan -> { | ||
| List<Object> matchingObjects = getReferencedObjectsMatchingRequest(EntityConstants.plans, plan, request); | ||
| if (!matchingObjects.isEmpty()) { | ||
| results.add(new FindReferencesResponse(plan)); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| // Sort the results by name | ||
| results.sort(Comparator.comparing(f -> f.name)); | ||
| return results; | ||
| } | ||
|
|
||
| private List<Object> getReferencedObjectsMatchingRequest(String entityType, AbstractOrganizableObject object, FindReferencesRequest request) { | ||
| List<Object> referencedObjects = getReferencedObjects(entityType, object).stream().filter(o -> (o != null && !o.equals(object))).collect(Collectors.toList()); | ||
| //System.err.println("objects referenced from plan: " + planToString(plan) + ": "+ referencedObjects.stream().map(ReferenceFinderServices::objectToString).collect(Collectors.toList())); | ||
| return referencedObjects.stream().filter(o -> doesRequestMatch(request, o)).collect(Collectors.toList()); | ||
| return referenceFinder.findReferences(request); | ||
| } | ||
|
|
||
| // returns a (generic) set of objects referenced by a plan | ||
| private Set<Object> getReferencedObjects(String entityType, AbstractOrganizableObject object) { | ||
| Set<Object> referencedObjects = new HashSet<>(); | ||
|
|
||
| // The references can be filled in three different ways due to the implementation: | ||
| // 1. through the predicate (just below) | ||
| // 2. by (actual object) reference in the tree visitor (onResolvedEntity) | ||
| // 3. by object ID in the tree visitor (onResolvedEntityId) | ||
|
|
||
| ObjectPredicate visitedObjectPredicate = visitedObject -> { | ||
| referencedObjects.add(visitedObject); | ||
| return true; | ||
| }; | ||
|
|
||
| EntityDependencyTreeVisitor entityDependencyTreeVisitor = new EntityDependencyTreeVisitor(entityManager, visitedObjectPredicate); | ||
| FindReferencesTreeVisitor entityTreeVisitor = new FindReferencesTreeVisitor(entityManager, referencedObjects); | ||
| entityDependencyTreeVisitor.visitEntityDependencyTree(entityType, object.getId().toString(), entityTreeVisitor, false); | ||
|
|
||
| return referencedObjects; | ||
| } | ||
|
|
||
| private boolean doesRequestMatch(FindReferencesRequest req, Object o) { | ||
| if (o instanceof Plan) { | ||
| Plan p = (Plan) o; | ||
| switch (req.searchType) { | ||
| case PLAN_NAME: | ||
| return p.getAttribute(AbstractOrganizableObject.NAME).equals(req.searchValue); | ||
| case PLAN_ID: | ||
| return p.getId().toString().equals(req.searchValue); | ||
| default: | ||
| return false; | ||
| } | ||
| } else if (o instanceof Function) { | ||
| Function f = (Function) o; | ||
| switch (req.searchType) { | ||
| case KEYWORD_NAME: | ||
| return f.getAttribute(AbstractOrganizableObject.NAME).equals(req.searchValue); | ||
| case KEYWORD_ID: | ||
| return f.getId().toString().equals(req.searchValue); | ||
| default: | ||
| return false; | ||
| } | ||
| } else if (o instanceof Resource) { | ||
| Resource r = (Resource) o; | ||
| switch (req.searchType) { | ||
| case RESOURCE_NAME: | ||
| return r.getAttribute(AbstractOrganizableObject.NAME).equals(req.searchValue); | ||
| case RESOURCE_ID: | ||
| return r.getId().toString().equals(req.searchValue); | ||
| default: | ||
| return false; | ||
| } | ||
| } else { | ||
| return false; | ||
| } | ||
| } | ||
|
|
||
| // the following functions are only needed for debugging | ||
| private static String objectToString(Object o) { | ||
| if (o instanceof Plan) { | ||
| return planToString((Plan) o); | ||
| } else if (o instanceof Function) { | ||
| return functionToString((Function) o); | ||
| } else { | ||
| return o.getClass() + " " + o.toString(); | ||
| } | ||
| } | ||
|
|
||
| private static String planToString(Plan plan) { | ||
| return "PLAN: " + plan.getAttributes().toString() + " id=" + plan.getId().toString(); | ||
| } | ||
|
|
||
| private static String functionToString(Function function) { | ||
| return "FUNCTION: " + function.getAttributes().toString() + " id=" + function.getId().toString(); | ||
| } | ||
|
|
||
|
|
||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.