Skip to content

Pipeline changes for better handling of failures#22

Merged
aditya0by0 merged 7 commits into
mainfrom
fix/validator_changes
May 8, 2026
Merged

Pipeline changes for better handling of failures#22
aditya0by0 merged 7 commits into
mainfrom
fix/validator_changes

Conversation

@aditya0by0
Copy link
Copy Markdown
Member

@aditya0by0 aditya0by0 commented May 1, 2026

  • Add a set-learn-false CLI subcommand to mark a learned/additional definition as learn_success=False in the learned-definitions pickle.
  • Make compare_with_c3p.py robust to missing learned definitions and provide a default --c3p-json path.
  • Adjust validator behavior to continue past per-class validation errors, and refine recursive learning behavior by having _learn() return a boolean.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates several inference/utility scripts to better handle validation failures, improve c3p comparison ergonomics, and add tooling for inspecting/adjusting learned-definition pickles.

Changes:

  • Add a set-learn-false CLI subcommand to mark a learned/additional definition as learn_success=False in the learned-definitions pickle.
  • Make compare_with_c3p.py robust to missing learned definitions and provide a default --c3p-json path.
  • Adjust validator behavior to continue past per-class validation errors, and refine recursive learning behavior by having _learn() return a boolean.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
nl_2_fol/inference/utils/show_learned_content.py Adds a new pickle-editing command (set-learn-false) for flipping learn_success flags.
nl_2_fol/inference/utils/compare_with_c3p.py Avoids KeyErrors when learned definitions are missing and adds a default c3p JSON path.
nl_2_fol/inference/utils/calculate_c3p_stats.py Introduces a new script to compute and print F1 distribution statistics from c3p_trust.json.
nl_2_fol/inference/learner/validator.py Continues validation despite exceptions and summarizes the number of failures.
nl_2_fol/inference/learner/learn_definitions.py Makes _learn() return a success boolean and uses it when handling missing predicates.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread nl_2_fol/inference/utils/show_learned_content.py Outdated
Comment thread nl_2_fol/inference/utils/calculate_c3p_stats.py
Comment thread nl_2_fol/inference/learner/learn_definitions.py Outdated
Comment thread nl_2_fol/inference/learner/validator.py Outdated
@aditya0by0 aditya0by0 changed the title Fix/validator changes Pipeline changes for better handling of failures May 8, 2026
@aditya0by0 aditya0by0 marked this pull request as ready for review May 8, 2026 08:52
@aditya0by0 aditya0by0 merged commit 1b43117 into main May 8, 2026
4 checks passed
@aditya0by0 aditya0by0 deleted the fix/validator_changes branch May 8, 2026 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants