[Bug]: Custom metric not correctly aligned on metrics chart

[hermesdt]

What component(s) are affected?

• Opik Python SDK
• Opik Typescript SDK
• Opik Agent Optimizer SDK
• Opik UI
• Opik Server
• Documentation

Opik version

• Opik version: 1.9.24

Describe the problem

I'm tracking multiple metrics on a project, and they are misaligned on the metrics chart. I suspect might be because I created them at different days. As you can see on the screenshot the metric "Tips chart count" should end on the right side, but looks like it stopped working.

However it's running just fine and everything go up to the last date.

Tips chart count is a custom metric generated with the custom metric dialog.

Reproduction steps and code snippets

No response

Error logs or stack trace

No response

Healthcheck results

No response

[dsblank]

@hermesdt thank you for reporting this issue. We'll take a look.

[andrescrz]

Hi @hermesdt

Thank you for sharing these details and your careful observations.

Your suspicion is correct—recently, we updated how metrics charts are handled in the Opik UI. Specifically, our charts now align data based on the id field (which is a UUID v7 and time-based), rather than on the start_time. This change was made as part of our rollout of the new data selector in the frontend, to make date-based filtering more scalable and efficient.

For the majority of users and production scenarios (99% of our data), the metric id very closely matches the start_time of the data (often just a few milliseconds apart). However, we have received some reports, including yours, where misalignment occurs. This typically happens when a custom start date is set, and there’s a significant gap between the chosen start time and the moment the data was actually generated (and the id created).

Since user experience is a top priority for us, we’ve already started an internal discussion to explore ways to improve this behavior. This could include reverting back to using the start_time for alignment on the charts, at least for these specific cases.

Just to confirm and help us assess the best approach—can you verify if you set the custom start time for your metric on a different day than when the metric was actually created? This information will help us to better diagnose and address the issue.

Thank you again for your feedback and patience!

[hermesdt]

Hi @andrescrz ,

I did not set any start time, I just created the rule through the online evaluation section of a running project.

I used a custom code rule:

from typing import Any
from opik.evaluation.metrics import base_metric, score_result

METRIC_NAME = "Tips char count"

def error_result(message: str) -> score_result.ScoreResult:
    return score_result.ScoreResult(
        value=0,
        name=METRIC_NAME,
        reason=message,
        scoring_failed=True,
    )

class MyCustomMetric(base_metric.BaseMetric):
    def __init__(self, name: str = METRIC_NAME):
        self.name = name

    def score(self, input: str, output: str, **ignored_kwargs: Any):
        import json

        data = json.loads(output)
        if not isinstance(data, dict):
            return error_result("Output is not a valid JSON object.")
        
        tips = data.get("tips", [])
        if not tips:
            return error_result("No tips found in the output.")
        
        max_size = max((len(tip) for tip in tips), default=0)

        return score_result.ScoreResult(
            value=max_size / 100,
            name=self.name,
        )

Thank you!

[andrescrz]

Thanks @hermesdt for the clarification. We'll use this information for the debugging process. We'll keep you posted!

[thiagohora]

Hi @hermesdt,

Thanks for reporting it.

Can you provide the project name and the workspace (assuming you are using Opik Cloud)?

The scores are attached to the trace, so you should see the feedback scores aligned with the ID's creation time, which is usually almost the same as start_time.