⚡️ Speed up method ObjectDetectionEvalProcessor._compute_page_detection_matching by 5%

[codeflash-ai]

📄 5% (0.05x) speedup for ObjectDetectionEvalProcessor._compute_page_detection_matching in unstructured/metrics/object_detection.py

⏱️ Runtime : 1.21 seconds → 1.15 seconds (best of 8 runs)

📝 Explanation and details

The optimized code achieves a 5% speedup through two key optimizations:

1. Numba-accelerated IoU computation: The most significant optimization is replacing the PyTorch _box_iou implementation with a Numba JIT-compiled version (_box_iou_numba). When running on CPU (which is common for object detection evaluation), this Numba implementation provides substantial performance gains by:

• Eliminating PyTorch's tensor operation overhead for simple arithmetic
• Using compiled native code instead of interpreted Python loops
• Operating directly on NumPy arrays with efficient memory access patterns

2. Numba-accelerated bounding box clipping: The _change_bbox_bounds_for_image_size function now uses a Numba-compiled helper (_change_bbox_bounds_for_image_size_numba) that:

• Performs in-place modifications to avoid memory allocations
• Uses explicit loops with simple conditional logic that compiles efficiently
• Replaces PyTorch's clip operations with faster native code

Performance characteristics from tests:

• Small datasets (single predictions): 10-15% speedups due to reduced overhead
• Medium datasets (hundreds of objects): 5-7% speedups from more efficient computations
• Large datasets (500+ objects): 3-5% speedups, where the core matching algorithm dominates

The optimizations are most effective for CPU-based evaluation workloads where object detection metrics are computed post-training. Since evaluation typically processes many images with moderate numbers of detections, the cumulative effect of these micro-optimizations provides meaningful performance gains. The code maintains a fallback to the original PyTorch implementation for GPU tensors, ensuring compatibility across different execution environments.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 60 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Generated Regression Tests and Runtime

import pytest  # used for our unit tests
import torch

from unstructured.metrics.object_detection import ObjectDetectionEvalProcessor

# function to test
# (Paste the function code here, as you provided above.)

# ----------- Unit Tests for ObjectDetectionEvalProcessor._compute_page_detection_matching -----------

class_labels = ["cat", "dog", "bird"]


def make_preds(preds_list):
    """
    Helper to create torch.Tensor for predictions.
    Each entry: (x1, y1, x2, y2, confidence, class_label)
    """
    return torch.tensor(preds_list, dtype=torch.float32)


def make_targets(targets_list):
    """
    Helper to create torch.Tensor for targets.
    Each entry: (label, x1, y1, x2, y2)
    """
    return torch.tensor(targets_list, dtype=torch.float32)


@pytest.fixture
def processor():
    # Minimal processor for single-page tests
    return ObjectDetectionEvalProcessor(
        document_preds=[],
        document_targets=[],
        pages_height=[100],
        pages_width=[100],
        class_labels=class_labels,
        device="cpu",
    )


# ------------------- BASIC TEST CASES -------------------


def test_single_pred_single_target_match(processor):
    # One prediction matches one target
    preds = make_preds([[10, 10, 20, 20, 0.9, 0]])  # class 0
    targets = make_targets([[0, 10, 10, 20, 20]])  # class 0
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 226μs -> 223μs (1.70% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_single_pred_single_target_no_match_due_to_class(processor):
    # One prediction, one target, different class
    preds = make_preds([[10, 10, 20, 20, 0.9, 1]])  # class 1
    targets = make_targets([[0, 10, 10, 20, 20]])  # class 0
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 129μs -> 113μs (13.9% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_multiple_preds_and_targets_simple(processor):
    # Two predictions, two targets, each matches exactly one
    preds = make_preds(
        [
            [10, 10, 20, 20, 0.9, 0],  # class 0
            [30, 30, 40, 40, 0.8, 1],  # class 1
        ]
    )
    targets = make_targets(
        [
            [0, 10, 10, 20, 20],  # class 0
            [1, 30, 30, 40, 40],  # class 1
        ]
    )
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 172μs -> 155μs (10.8% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_preds_with_low_confidence_are_ignored(processor):
    # Only predictions above score_threshold should be considered
    preds = make_preds(
        [
            [10, 10, 20, 20, 0.05, 0],  # below threshold
            [30, 30, 40, 40, 0.8, 1],  # above threshold
        ]
    )
    targets = make_targets(
        [
            [0, 10, 10, 20, 20],
            [1, 30, 30, 40, 40],
        ]
    )
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 163μs -> 146μs (11.4% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


# ------------------- EDGE TEST CASES -------------------


def test_pred_outside_image_bounds_is_clipped(processor):
    # Prediction partially outside image, should be clipped
    preds = make_preds([[90, 90, 110, 110, 0.9, 0]])
    targets = make_targets([[0, 90, 90, 100, 100]])
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 139μs -> 122μs (13.7% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_pred_and_target_overlap_below_iou_threshold(processor):
    # Overlap is below minimum IoU threshold
    preds = make_preds([[0, 0, 10, 10, 0.9, 0]])
    targets = make_targets([[0, 9, 9, 19, 19]])
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 117μs -> 102μs (14.4% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_multiple_preds_for_one_target(processor):
    # Two predictions for one target, only the highest confidence should match
    preds = make_preds(
        [
            [10, 10, 20, 20, 0.7, 0],  # lower confidence
            [10, 10, 20, 20, 0.9, 0],  # higher confidence
        ]
    )
    targets = make_targets([[0, 10, 10, 20, 20]])
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 141μs -> 126μs (12.3% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_multiple_targets_for_one_pred(processor):
    # One prediction overlaps two targets, only one should be matched due to priority
    preds = make_preds([[10, 10, 20, 20, 0.9, 0]])
    targets = make_targets(
        [
            [0, 10, 10, 20, 20],
            [0, 10, 10, 20, 20],
        ]
    )
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100)
    out = codeflash_output  # 156μs -> 139μs (12.5% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_top_k_limit(processor):
    # More than top_k preds per class, only top_k used for matching
    preds = make_preds(
        [
            [10, 10, 20, 20, 0.9, 0],  # class 0
            [11, 11, 21, 21, 0.8, 0],  # class 0
            [12, 12, 22, 22, 0.7, 0],  # class 0
            [13, 13, 23, 23, 0.6, 0],  # class 0
        ]
    )
    targets = make_targets(
        [
            [0, 10, 10, 20, 20],
            [0, 11, 11, 21, 21],
            [0, 12, 12, 22, 22],
            [0, 13, 13, 23, 23],
        ]
    )
    # Set top_k=2
    codeflash_output = processor._compute_page_detection_matching(preds, targets, 100, 100, top_k=2)
    out = codeflash_output  # 212μs -> 192μs (10.3% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


# ------------------- LARGE SCALE TEST CASES -------------------


def test_large_number_of_preds_and_targets(processor):
    # 500 preds, 500 targets, all matching, class 0
    N = 500
    preds = torch.zeros((N, 6), dtype=torch.float32)
    targets = torch.zeros((N, 5), dtype=torch.float32)
    for i in range(N):
        preds[i, :4] = torch.tensor([i, i, i + 1, i + 1])
        preds[i, 4] = 0.9
        preds[i, 5] = 0
        targets[i, 0] = 0
        targets[i, 1:5] = torch.tensor([i, i, i + 1, i + 1])
    codeflash_output = processor._compute_page_detection_matching(preds, targets, N + 1, N + 1)
    out = codeflash_output  # 3.16ms -> 3.05ms (3.72% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_large_number_of_preds_and_targets_no_matches(processor):
    # 500 preds, 500 targets, no overlap
    N = 500
    preds = torch.zeros((N, 6), dtype=torch.float32)
    targets = torch.zeros((N, 5), dtype=torch.float32)
    for i in range(N):
        preds[i, :4] = torch.tensor([i, i, i + 1, i + 1])
        preds[i, 4] = 0.9
        preds[i, 5] = 0
        targets[i, 0] = 0
        targets[i, 1:5] = torch.tensor([i + 2, i + 2, i + 3, i + 3])  # no overlap
    codeflash_output = processor._compute_page_detection_matching(preds, targets, N + 3, N + 3)
    out = codeflash_output  # 3.12ms -> 3.01ms (3.54% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_large_number_of_classes(processor):
    # 10 classes, 50 preds per class, 50 targets per class, all match
    num_classes = 10
    per_class = 50
    N = num_classes * per_class
    preds = torch.zeros((N, 6), dtype=torch.float32)
    targets = torch.zeros((N, 5), dtype=torch.float32)
    for c in range(num_classes):
        for i in range(per_class):
            idx = c * per_class + i
            preds[idx, :4] = torch.tensor([i, i, i + 1, i + 1])
            preds[idx, 4] = 0.9
            preds[idx, 5] = c
            targets[idx, 0] = c
            targets[idx, 1:5] = torch.tensor([i, i, i + 1, i + 1])
    codeflash_output = processor._compute_page_detection_matching(
        preds, targets, per_class + 1, per_class + 1
    )
    out = codeflash_output  # 13.3ms -> 12.7ms (4.36% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


def test_large_top_k_limit(processor):
    # 1000 preds, top_k=500, only first 500 are used for matching
    N = 1000
    preds = torch.zeros((N, 6), dtype=torch.float32)
    targets = torch.zeros((500, 5), dtype=torch.float32)
    for i in range(N):
        preds[i, :4] = torch.tensor([i, i, i + 1, i + 1])
        preds[i, 4] = 1.0 - (i / N)  # descending confidence
        preds[i, 5] = 0
    for i in range(500):
        targets[i, 0] = 0
        targets[i, 1:5] = torch.tensor([i, i, i + 1, i + 1])
    codeflash_output = processor._compute_page_detection_matching(
        preds, targets, N + 1, N + 1, top_k=500
    )
    out = codeflash_output  # 13.4ms -> 12.7ms (5.17% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = out


# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

import pytest
import torch

from unstructured.metrics.object_detection import ObjectDetectionEvalProcessor

# function to test
# (Paste the full ObjectDetectionEvalProcessor class definition here as provided above)

# unit tests

class_labels = ["cat", "dog", "bird"]  # Example class labels for tests


@pytest.fixture
def processor():
    # Setup a default processor for testing
    return ObjectDetectionEvalProcessor(
        document_preds=[],
        document_targets=[],
        pages_height=[],
        pages_width=[],
        class_labels=class_labels,
        device="cpu",
    )


# ---------------------- BASIC TEST CASES ----------------------


def test_no_predictions_no_targets(processor):
    # No predictions and no targets
    preds = torch.empty((0, 6))
    targets = torch.empty((0, 5))
    height, width = 100, 100
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 26.2μs -> 20.2μs (30.2% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_single_prediction_single_target_match(processor):
    # One prediction, one target, same class, perfect overlap
    preds = torch.tensor([[10, 10, 20, 20, 0.9, 1]])  # class_label=1 ("dog")
    targets = torch.tensor([[1, 10, 10, 20, 20]])  # class_label=1
    height, width = 30, 30
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 188μs -> 168μs (11.6% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_single_prediction_single_target_no_match_due_to_class(processor):
    # One prediction, one target, different class
    preds = torch.tensor([[10, 10, 20, 20, 0.8, 2]])  # class_label=2 ("bird")
    targets = torch.tensor([[1, 10, 10, 20, 20]])  # class_label=1 ("dog")
    height, width = 30, 30
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 124μs -> 113μs (9.96% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_multiple_predictions_multiple_targets(processor):
    # Two predictions, two targets, same class, one matches, one doesn't
    preds = torch.tensor(
        [
            [0, 0, 10, 10, 0.7, 0],  # "cat", overlaps with target 1
            [20, 20, 30, 30, 0.6, 1],  # "dog", no overlap
        ]
    )
    targets = torch.tensor(
        [
            [0, 0, 10, 10, 0],  # "cat"
            [1, 50, 50, 60, 60],  # "dog"
        ]
    )
    height, width = 100, 100
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 129μs -> 118μs (9.37% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_top_k_limit(processor):
    # More predictions than top_k, only top_k per class should be considered
    preds = torch.tensor(
        [
            [0, 0, 10, 10, 0.9, 0],  # "cat"
            [0, 0, 10, 10, 0.8, 0],  # "cat"
            [0, 0, 10, 10, 0.7, 0],  # "cat"
            [0, 0, 10, 10, 0.6, 0],  # "cat"
        ]
    )
    targets = torch.tensor(
        [
            [0, 0, 10, 10, 0],  # "cat"
        ]
    )
    height, width = 20, 20
    codeflash_output = processor._compute_page_detection_matching(
        preds, targets, height, width, top_k=2
    )
    result = codeflash_output  # 124μs -> 113μs (9.71% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


# ---------------------- EDGE TEST CASES ----------------------


def test_prediction_box_outside_image(processor):
    # Prediction box outside image bounds, should be clipped
    preds = torch.tensor([[110, 110, 120, 120, 0.5, 1]])  # class_label=1 ("dog")
    targets = torch.tensor([[1, 110, 110, 120, 120]])
    height, width = 100, 100
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 120μs -> 107μs (12.1% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_target_box_outside_image(processor):
    # Target box outside image bounds, should be clipped
    preds = torch.tensor([[10, 10, 20, 20, 0.7, 2]])  # class_label=2 ("bird")
    targets = torch.tensor([[2, 110, 110, 120, 120]])
    height, width = 100, 100
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 118μs -> 105μs (12.1% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_prediction_confidence_below_threshold(processor):
    # Prediction confidence below score threshold, but function does not filter by score
    preds = torch.tensor([[10, 10, 20, 20, 0.05, 1]])  # class_label=1 ("dog"), low confidence
    targets = torch.tensor([[1, 10, 10, 20, 20]])
    height, width = 30, 30
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 139μs -> 125μs (10.8% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_multiple_classes(processor):
    # Multiple predictions and targets, multiple classes, some match, some not
    preds = torch.tensor(
        [
            [0, 0, 10, 10, 0.9, 0],  # "cat"
            [20, 20, 30, 30, 0.8, 1],  # "dog"
            [40, 40, 50, 50, 0.7, 2],  # "bird"
        ]
    )
    targets = torch.tensor(
        [
            [0, 0, 10, 10, 0],  # "cat"
            [1, 20, 20, 30, 30],  # "dog"
            [2, 40, 40, 50, 50],  # "bird"
        ]
    )
    height, width = 60, 60
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 167μs -> 150μs (11.2% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result
    for i in range(3):
        pass


def test_empty_targets(processor):
    # No targets, predictions should not match anything
    preds = torch.tensor([[10, 10, 20, 20, 0.7, 1]])
    targets = torch.empty((0, 5))
    height, width = 30, 30
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 47.7μs -> 47.5μs (0.263% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_empty_predictions(processor):
    # No predictions, targets present
    preds = torch.empty((0, 6))
    targets = torch.tensor([[1, 10, 10, 20, 20]])
    height, width = 30, 30
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 12.3μs -> 12.2μs (0.343% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_non_overlapping_boxes_same_class(processor):
    # Same class, but boxes do not overlap
    preds = torch.tensor([[0, 0, 10, 10, 0.7, 1]])
    targets = torch.tensor([[1, 20, 20, 30, 30]])
    height, width = 40, 40
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 120μs -> 107μs (11.8% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


# ---------------------- LARGE SCALE TEST CASES ----------------------


def test_large_number_of_predictions_and_targets(processor):
    # Many predictions and targets, all same class, all overlap perfectly
    N = 500
    preds = torch.zeros((N, 6))
    preds[:, 0:4] = torch.tensor([5, 5, 15, 15])  # All same box
    preds[:, 4] = torch.linspace(0.5, 1.0, N)  # Confidence varies
    preds[:, 5] = 1  # class_label=1 ("dog")
    targets = torch.zeros((N, 5))
    targets[:, 0] = 1  # class_label=1
    targets[:, 1:5] = torch.tensor([5, 5, 15, 15])
    height, width = 20, 20
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 1.16s -> 1.10s (5.13% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_large_number_of_classes(processor):
    # Many classes, each with one prediction and one target, all match
    N = 50
    preds = torch.zeros((N, 6))
    targets = torch.zeros((N, 5))
    for i in range(N):
        preds[i, 0:4] = torch.tensor([i, i, i + 10, i + 10])
        preds[i, 4] = 0.8
        preds[i, 5] = i
        targets[i, 0] = i
        targets[i, 1:5] = torch.tensor([i, i, i + 10, i + 10])
    height, width = N + 10, N + 10
    # Extend class labels for processor
    processor.class_labels = [str(i) for i in range(N)]
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 1.18ms -> 1.13ms (5.26% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result
    for i in range(N):
        pass


def test_large_number_of_predictions_one_target(processor):
    # Many predictions, one target, only one should match
    N = 200
    preds = torch.zeros((N, 6))
    preds[:, 0:4] = torch.tensor([5, 5, 15, 15])
    preds[:, 4] = torch.linspace(0.1, 1.0, N)
    preds[:, 5] = 1
    targets = torch.tensor([[1, 5, 5, 15, 15]])
    height, width = 20, 20
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 211μs -> 183μs (15.5% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_large_number_of_targets_one_prediction(processor):
    # One prediction, many targets, only one should match
    N = 300
    preds = torch.tensor([[5, 5, 15, 15, 0.9, 1]])
    targets = torch.zeros((N, 5))
    targets[:, 0] = 1
    targets[:, 1:5] = torch.tensor([5, 5, 15, 15])
    height, width = 20, 20
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 6.33ms -> 5.90ms (7.20% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


def test_large_scale_non_overlapping(processor):
    # Large number of predictions and targets, none overlap
    N = 400
    preds = torch.zeros((N, 6))
    targets = torch.zeros((N, 5))
    for i in range(N):
        preds[i, 0:4] = torch.tensor([i * 10, i * 10, i * 10 + 5, i * 10 + 5])
        preds[i, 4] = 0.5
        preds[i, 5] = 0
        targets[i, 0] = 0
        targets[i, 1:5] = torch.tensor([i * 10 + 6, i * 10 + 6, i * 10 + 10, i * 10 + 10])
    height, width = N * 10 + 10, N * 10 + 10
    codeflash_output = processor._compute_page_detection_matching(preds, targets, height, width)
    result = codeflash_output  # 739μs -> 694μs (6.37% faster)
    preds_matched, preds_to_ignore, preds_scores, preds_cls, targets_cls = result


# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-ObjectDetectionEvalProcessor._compute_page_detection_matching-mjceyvsc and push.