Evaluation: Investigate long-running process

[AkhileshNegi]

Is your feature request related to a problem?
The Educate Girls team has reported that the evaluation process has been running for the last 23 hours. This delay can hinder timely project progress and decision-making.

• Investigate the cause of long-running evaluations
• Explore ways to optimize the evaluation run if possible for quicker results

Screenshots

[AkhileshNegi]

None of the evaluation runs are currently processing.

A few runs failed. We received this message from OpenAI:

Batch expired (error_file_id: file-1rte74QPMX4zZfvBdXy6k6)

Looking into the error file, this is what it shows:

An error occurred while processing your request. You can retry your request, or contact us through our help center at help.openai.com if the error persists. Please include the request ID 2e83bcff-075d-411a-83c7-3fa4b1182864 in your message

Based on this, there’s nothing actionable on our side. Best next step is to rerun the evaluations.

[PritamSGB]

@AkhileshNegi Now nothing may be in process, but it was in process for longer time as it was seen in first screenshot.
But any idea, why Batch expired error came? was it any issue at our end ?

[AkhileshNegi]

We use OpenAI's Batch API for evaluations, which runs jobs asynchronously with a completion window of up to 24 hours — we don't control when a batch actually finishes processing on their end.

The expired batch in this case was an issue on OpenAI's side (the error file confirms it was a server-side error with a request ID for their support). Per their docs, when a batch expires, any completed requests are still returned, and the rest are cancelled — there's no action available to us beyond rerunning the failed evaluations, which we've done