Optimization effort

[mohammedahmed18]

User description

fixes CF-938

---

PR Type

Enhancement

---

Description

• Introduce effort-based optimization levels

• Replace static candidate/test counts

• Add CLI flag for effort selection

• Simplify git worktree utilities

---

Diagram Walkthrough

flowchart LR
  CLI["CLI --effort flag"] -- "passes effort" --> Optimizer["Function Optimizer"]
  Optimizer -- "queries" --> Effort["Effort config (counts)"]
  Effort -- "n_candidates / tests" --> Optimizer
  Optimizer -- "payload with n_candidates" --> AISvc["AI Service Client"]

Loading

File Walkthrough

	Relevant files
Enhancement	aiservice.py Align API payloads to effort-based candidate keys                codeflash/api/aiservice.py Use n_candidates in payload Use n_candidates_lp for line profiler payload Remove static effective constants from payloads Set default LP candidates to 8 +3/-6      cli.py Add effort level CLI flag                                                                codeflash/cli_cmds/cli.py Add --effort CLI option Choices: low, medium, high; default medium +3/-0      config_consts.py Introduce effort-based configuration utilities                      codeflash/code_utils/config_consts.py Remove static candidate/test constants Add EffortLevel enum and Effort helper Provide methods for candidates and tests by effort Keep effective total looping time logic +37/-12  function_optimizer.py Use effort-driven counts in optimization flow                        codeflash/optimization/function_optimizer.py Replace constants with Effort-based counts Dynamic tests and candidates using CLI effort Update line-profiler candidate count usage +6/-8
Cleanup	git_utils.py Simplify git utils by removing worktree helpers                    codeflash/code_utils/git_utils.py Remove unused worktree creation/removal helpers Drop dependency on candidate constants +0/-34

---

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review API Contract Change The request payload keys changed from 'num_variants' and constants to 'n_candidates'/'n_candidates_lp' driven by CLI effort. Verify the server accepts these new fields and that removing the previous fields does not break backward compatibility. payload = { "source_code": source_code, "dependency_code": dependency_code, "n_candidates": num_candidates, "trace_id": trace_id, "python_version": platform.python_version(), "experiment_metadata": experiment_metadata, "codeflash_version": codeflash_version, "current_username": get_last_commit_author_if_pr_exists(None), "repo_owner": git_repo_owner, "repo_name": git_repo_name, "is_async": is_async, } Input Validation Effort parsing relies on string equality; invalid values raise ValueError. Ensure all call sites pass normalized effort (from CLI choices) and that programmatic uses cannot bypass validation or crash user flows. class EffortLevel(str, Enum): LOW = "low" MEDIUM = "medium" HIGH = "high" class Effort: @staticmethod def get_number_of_optimizer_candidates(effort: str) -> int: if effort == EffortLevel.LOW.value: return 3 if effort == EffortLevel.MEDIUM.value: return 4 if effort == EffortLevel.HIGH.value: return 5 msg = f"Invalid effort level: {effort}" raise ValueError(msg) @staticmethod def get_number_of_optimizer_lp_candidates(effort: str) -> int: if effort == EffortLevel.LOW.value: return 3 if effort == EffortLevel.MEDIUM.value: return 5 if effort == EffortLevel.HIGH.value: return 6 msg = f"Invalid effort level: {effort}" raise ValueError(msg) @staticmethod def get_number_of_generated_tests(effort: str) -> int: # noqa: ARG004 # we don't use effort with generated tests for now return 2 Concurrency Sizing ThreadPoolExecutor max_workers depends on effort-derived test count. Confirm this does not reduce parallelism unexpectedly for higher/lower effort and that queueing behavior remains acceptable under typical workloads. n_tests = Effort.get_number_of_generated_tests(args.effort) self.executor = concurrent.futures.ThreadPoolExecutor( max_workers=n_tests + 3 if self.experiment_id is None else n_tests + 4 ) self.optimization_review = ""

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Default missing effort safely Accessing args.effort assumes the attribute is always present; older code paths or programmatic invocations may omit it, causing AttributeError. Default to "medium" when missing or falsy to ensure stable behavior. codeflash/optimization/function_optimizer.py [240-243] -n_tests = Effort.get_number_of_generated_tests(args.effort) +effort_level = getattr(args, "effort", None) or "medium" +n_tests = Effort.get_number_of_generated_tests(effort_level) Suggestion importance[1-10]: 7 __ Why: Using args.effort directly can raise AttributeError in older or programmatic paths; defaulting to "medium" improves robustness with minimal risk and aligns with CLI default.	Medium
Possible issue	Preserve API request compatibility Ensure the request keys match the backend API contract. The key was changed from num_variants to n_candidates; if the server still expects num_variants, this will break requests. Include both keys mapped to the same value to maintain backward compatibility. codeflash/api/aiservice.py [129-141] payload = { "source_code": source_code, "dependency_code": dependency_code, "n_candidates": num_candidates, + "num_variants": num_candidates, # backward compatibility "trace_id": trace_id, "python_version": platform.python_version(), "experiment_metadata": experiment_metadata, "codeflash_version": codeflash_version, "current_username": get_last_commit_author_if_pr_exists(None), "repo_owner": git_repo_owner, "repo_name": git_repo_name, "is_async": is_async, } Suggestion importance[1-10]: 6 __ Why: The suggestion is contextually valid: the PR changes the payload key from num_variants to n_candidates. Proposing to send both keys can prevent backend mismatches during rollout, though it assumes server behavior and adds redundancy, so impact is moderate.	Low
	Avoid API field mismatch The request key was changed from num_variants to n_candidates_lp. If the backend still accepts num_variants (or n_candidates_lp is not recognized), requests will fail. Send both keys with the same value for safe rollout. codeflash/api/aiservice.py [191-201] payload = { "source_code": source_code, "dependency_code": dependency_code, "n_candidates_lp": num_candidates, + "num_variants": num_candidates, # backward compatibility "line_profiler_results": line_profiler_results, "trace_id": trace_id, "python_version": platform.python_version(), "experiment_metadata": experiment_metadata, "codeflash_version": codeflash_version, "lsp_mode": is_LSP_enabled(), } Suggestion importance[1-10]: 6 __ Why: Similar to suggestion 1, this guards against backend expecting num_variants when the PR sends n_candidates_lp. It’s reasonable for backward compatibility, but speculative about server requirements and adds duplicate data.	Low