Express indicators requirements using DSL rules

[ndrhzn]

Context & Motivation

Currently, field requirements for usability indicators and red flags are defined as a mix of code logic (e.g. when we choose which set of fields to use as a proxy for buyer or award value), and a dictionary with a set of required fields for each indicator. See component_setup_usability for a reference.

This might be hard to maintain and follow, as conditions for choosing proxy fields are separated from indicator definitions.

What we can do instead is define requirements for indicators using DSL rules.

Proposed Solution

For each indicator (and red flag), define a rule using nested structure where:

• {"all": [ ... ]} means all sub-requirements are mandatory (logical AND)
• {"any": [ ... ]} means at least one sub-requirement is needed (logical OR)

Combine rules into one and rewrite the 'usability checks' function to take that dictionary as an input.

Example

U066 'Percent of contracts that exceed budget' can be calculated using either planning or tender values.

Currently it is expressed in two parts

Selection of a proxy fields for 'budget':

 # U066, U067
    if "planning/budget/amount/amount" in fields_list and "planning/budget/amount/currency":
        planning = ["planning/budget/amount/amount", "planning/budget/amount/currency"]
    elif "tender/value/amount" in fields_list and "tender/value/currency":
        planning = ["tender/value/amount", "tender/value/currency"]
    else:
        planning = ["planning/budget/amount/amount", "planning/budget/amount/currency"]

Construction of an indicator requirements:

{
  "Percent of contracts that exceed budget": [
    [
      "U066"
    ],
    [
      "ocid",
      "contracts/status",
      "contracts/value/amount",
      "contracts/value/currency",
      "*planning"
    ]
  ]
}

Alternatively it can be expressed as a rule:

U066_REQUIREMENT = {
  "all": [
    "ocid",
    "contracts/status",
    "contracts/value/amount",
    "contracts/value/currency",
    {"any": [
      {"all": ["planning/budget/amount/amount", "planning/budget/amount/currency"]},
      {"all": ["tender/value/amount", "tender/value/currency"]}
    ]}
  ]
}

Benefits

Defined this way, rules should be more readable and maintainable. It should be easy to add new requirements and conditions if needed.

[ndrhzn]

@jpmckinney I can see you had some fun with Copilot over the weekend!

So did I. From what I managed to see, my implementation is a bit different from yours. Adding here for a reference.

I steered Copilot into including full rules each time and not optimizing much (meaning not putting repeated elements into variables) - just for readability’s sake & to ensure that each indicator rule is fully encapsulated.

Additionally, when there are a few options for calculation, a decision can be made regarding what to consider as missing and required. And in this implementation, there are some opinionated choices regarding that (but I'm not sure if everyone would agree with that).

For {"any": [...]} (OR) nodes, it selects the branch that is "best" given the available fields:
  1) prefer a branch with 0 missing fields (fully satisfied)
  2) otherwise prefer the branch with the fewest missing fields
  3) tie-breaker: keep the first branch

This was done primarily to test if the proposed approach would work at all, so in the actual rewrite design choices could be different of course.

[jpmckinney]

Do you have a preferred approach? (I haven't read the code in #99, as it'll be easier to review if I do #100 first.)

[jpmckinney]

@ndrhzn Now that I've done #100, I now asked Claude Code to work on this issue, and it came up with this: https://github.com/open-contracting/notebooks-ocds/blob/98-indicators/component_setup_usability.py

I need to review:

• the indicators table
• _evaluate_rule
• _get_required_fields
• _get_missing_fields
• usability_checks
• get_coverage
• most_common_fields_to_calculate_indicators

I'm not familiar with the old code, so please have a look and let me know what you think.

I just asked whether the behavior was still the same:

Calculation result (whether indicator is calculable): Same behavior

• Old: Pre-selected one alternative based on availability, then checked if all those fields exist
• New: evaluate_rule checks if ANY alternative has all its fields present

Both mark an indicator as calculable if at least one valid alternative exists.

"fields needed" column: Slight difference

• Old: Showed whichever alternative was selected (based on field availability)
• New: Always shows the first alternative (from get_required_fields)

"missing fields" column: Slight difference

• Old: Showed missing fields from the pre-selected alternative
• New: Shows missing fields from the alternative with fewest missing (from get_missing_fields)

So the core logic (can we calculate this indicator?) is the same, but the display of which fields are needed/missing may differ. The new approach arguably shows more useful information for "missing
fields" since it picks the closest alternative.

Do you want me to adjust the behavior, for example making get_required_fields also pick the best available alternative?

[jpmckinney]

In your version:

• Not sure why _dedupe_preserve_order is needed. I'm not aware that rules repeat fields (there was one accidental repetition).
• _flatten_all and _missing have a if isinstance(req, list) branch, but I don't think this can occur.

From earlier, Claude asked:

Do you want me to adjust the behavior, for example making get_required_fields also pick the best available alternative?

I answered yes (like in your code).

I've familiarized myself with the code now, so I'll merge the branch.

We can test whether the new notebooks generate the same results as existing notebooks.

Edit: Also, I find having named "subrules" can be helpful, e.g. repeating the BUYER_RULE verbatim can lead to error if we decide to update that rule, but forget some occurrences. In the original code, the variables were poorly named, but with good naming the variable becomes useful again.