feat(studio): add email-security-analyst sample; agents list UX - #1060
Open
nv-odrulea wants to merge 5 commits into
Open
feat(studio): add email-security-analyst sample; agents list UX#1060nv-odrulea wants to merge 5 commits into
nv-odrulea wants to merge 5 commits into
Conversation
…ests - CardSelect: new common component for card-style option selection used by the eval config picker in SubmitEvaluationModal - StudioDataViewToolbar: adds support for row actions - entityName.test.ts: removes stale secrets SDK assertions after the secrets schema dropped named regex/max exports Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Octavian Drulea <odrulea@nvidia.com>
Studio can now submit and display both evaluation paradigms against a deployed agent. Previously only task-driven (/agent-evaluate/jobs) was reachable; this adds dataset-driven (/evaluate/jobs). - SubmitEvaluationModal: detects eval config shape (tasks[] vs dataset+metrics) and routes to the correct evaluator endpoint. Fileset creation is now atomic (409 → user-facing collision error) with upload rollback on failure. - EvaluationResultDetailsRoute: new route for dataset-driven results showing aggregate scores and per-row results panels. - AgentEvaluationDetailRoute: Task-Driven chip on header; single cancel path. - DatasetEvalRowResultsPanel, DatasetEvalScoresPanel, useDatasetEvalResults: new components for dataset eval result rendering with failure/pending states. - eval-config-fileset.ts, evaluator-jobs.ts: unified job fetcher and hardened fileset helpers shared by both paradigms. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Octavian Drulea <odrulea@nvidia.com>
…uation SubmitEvaluationModal, AgentEvalTaskResultsPanel, and its test were moved to src/components/evaluation/; delete the stale copies under routes/agents/AgentEvaluationsRoute/components/. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Octavian Drulea <odrulea@nvidia.com>
…ation Test was renamed to submitEvaluationJob.test.ts and moved to components/evaluation/; delete the copy at the old route path. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Octavian Drulea <odrulea@nvidia.com>
…vements - email-security-analyst: Studio sample agent with task-driven and dataset-driven eval configs, eval README files, and a 40-row phishing dataset - email-phishing-analyzer: converts to dataset-driven eval config with placeholder dataset reference; removes stale task-driven files - AgentsDataView: adds Run Evaluation row action; renames Test Models to Compare Models; converts to StudioDataView - AgentsListRoute: wires Run Evaluation into SubmitEvaluationModal - CreateExampleAgentModal: removes useEffect/seededRef smell; key-based remount with single seed effect - loadSampleAgentModelName: moved to util/sampleAgents.ts (not API layer) - Removes stale POC tests and mock handlers for sample agents Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Octavian Drulea <odrulea@nvidia.com>
nv-odrulea
marked this pull request as ready for review
August 4, 2026 18:34
nv-odrulea
force-pushed
the
od/studio-dataset-eval-feature
branch
from
August 4, 2026 20:57
89e54c2 to
ba750d7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
<workspace>/<fileset>#dataset.jsonlplaceholder; removes stale task-driven filesuseEffect/seededRefsmell; key-based remount with single seed effectloadSampleAgentModelNamemoved toutil/sampleAgents.ts(not API layer)Test plan
pnpm --filter nemo-studio-ui typecheckpnpm --filter nemo-studio-ui test🤖 Generated with Claude Code