Users/mcampbell/yield and throughput

[markacamps]

Adding example for Yield and Throughput which includes 2 notebooks, 1 dashboard, and 1 README that details how to setup and configure step-by-step.

[Copilot]

Pull request overview

Adds a new “Yield and Throughput” example under examples/dashboards/ that combines two SystemLink notebooks, a Grafana dashboard JSON, and a README to guide setup of yield/throughput reporting.

Changes:

• Added a Grafana dashboard (Yield-and-Throughput.json) wired to a SystemLink notebook datasource.
• Added two notebooks: one to backfill/write a “Test Iteration” property (UpdateTestIteration.ipynb) and one to compute yield metrics (FirstPassYield.ipynb).
• Added installation/setup documentation (README.md) for publishing notebooks and importing the dashboard.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
examples/dashboards/Yield and Throughput/Yield-and-Throughput.json	New Grafana dashboard definition for yield/throughput visualization.
examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb	New notebook to compute and write “Test Iteration” onto Test Monitor results.
examples/dashboards/Yield and Throughput/FirstPassYield.ipynb	New notebook to query Test Monitor results and compute yield + counts for Grafana.
examples/dashboards/Yield and Throughput/README.md	New step-by-step instructions to publish notebooks, schedule routine, and import dashboard.

Comments suppressed due to low confidence (14)

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:167

• perform_batched_query never appends the final page (or any page when continuation_token is initially empty), so it can return an empty/incomplete result set even when the API returns results. Update the loop to always append the current response page before checking/advancing the continuation token.

    "#Define batched query function\n",
    "async def perform_batched_query(query_function, query, response_field):\n",
    "    results = []\n",
    "    response = await query_function(post_body=query)\n",
    "    while response.continuation_token:\n",
    "        results = results + getattr(response, response_field)\n",
    "        query.continuation_token = response.continuation_token\n",
    "        response = await query_function(post_body=query)\n",
    "    return results"

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:65

• The parameters-cell metadata lists inputs like group_by, step_filter, etc., but the code in this cell defines/uses results_filter and iteration_filter. Update the papermill.parameters / systemlink.parameters metadata to reflect the actual parameters this notebook expects so the execution UI exposes the right inputs.

    "papermill": {
     "parameters": {
      "group_by": "Day",
      "products_filter": "",
      "result_status_filter": "",
      "results_filter": "",
      "station_filter": "",
      "step_filter": "",
      "step_status_filter": ""
     }

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:76

• The systemlink.outputs metadata advertises an output detailedsteps_df, but this notebook does not create/return that output. This will confuse consumers and can break dashboard panels expecting specific output IDs—please align the declared outputs with what the notebook actually returns (or remove the unused output declaration).

     "outputs": [
      {
       "display_name": "Detailed Steps",
       "id": "detailedsteps_df",
       "type": "data_frame"
      }

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:36

• Several imports in this notebook are unused (time, datetime, copy, numpy, exit). Removing unused imports reduces notebook startup time and avoids implying dependencies that aren't required.

    "import pandas as pd\n",
    "import scrapbook as sb\n",
    "import time\n",
    "import datetime\n",
    "import copy\n",
    "import numpy as np\n",
    "from sys import exit\n",

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:49

• Spelling/grammar issues in this parameters description (e.g., “theTest”, “doesn't exists”, “Staut Type”) make the filter definition unclear. Please proofread and correct to avoid confusion about the intended query conditions.

    "### Parameters\n",
    "- `results_filter`: Number of hours the automation should look for new results (DD.HH:mm:ss)\n",
    "  Default: `30 min`\n",
    "- `iteration_filter`: Set the conditions to create theTest Iteration property: properties ->\"Test Iteration\" is empty or doesn't exists, Staut Type is not \"running\", and serialNumber is not empty  "

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:180

• This notebook is committed with executed cell outputs / non-null execution_count (e.g., the stdout output here). Most notebooks in this repo keep outputs: [] and execution_count: null (see examples/Asset Utilization/Delete Multiple Tags.ipynb), which avoids noisy diffs and stale example output. Please clear outputs before committing.

   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:248

• This markdown cell contains what looks like a Python snippet, but it’s incorrect (for results in results2 then indexing results2[...], and grouping is undefined). Either remove it or replace with a correct, fenced code example so the documentation doesn’t mislead users.

    "group_names = []\n",
    "for results in results2:\n",
    "    if grouping in results2:\n",
    "        group_names.append(results2[grouping])\n",
    "#print(results_list2)\n"

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:256

• Typo in section title: “Datafame” should be “Dataframe”.

    "## Create Pandas Datafame"
   ]

examples/dashboards/Yield and Throughput/UpdateTestIteration.ipynb:407

• update_results_v2 is called inside the loop while new_result keeps growing, so each iteration re-sends all prior updates again (O(n²) payload) and makes one API call per row. This is inefficient and can trigger rate limits. Build the full list first (or batch it) and call update_results_v2 once per batch.

    "new_result = []\n",
    "for ind in df_results.index:\n",
    "    test_props = {}\n",
    "    result_id = str(df_results['id'][ind])\n",
    "    test_props[\"Test Iteration\"] = str(df_results['Iteration'][ind])\n",
    "    new_result.append(ind)\n",
    "    new_result[len(new_result)-1] = tm_models.test_result_update_request_object.TestResultUpdateRequestObject(id=result_id, properties=test_props)  \n",
    "    request_body = tm_models.update_test_results_request.UpdateTestResultsRequest(results=new_result, replace=False)\n",
    "\n",
    "ret_val = await results_api.update_results_v2(request_body)\n",

examples/dashboards/Yield and Throughput/FirstPassYield.ipynb:275

• The pagination loop drops results: when continuation_token is empty, the while loop never runs so the first page is never appended; when it’s non-empty, the final page is never appended. This can make results_list incomplete/empty. Append response.results for the initial and final response pages (e.g., loop that always appends current page, then breaks when no continuation token).

    "results_query = testmon.ResultsAdvancedQuery(\n",
    "    results_filter, product_filter=products_filter, order_by=testmon.ResultField.STARTED_AT)\n",
    "\n",
    "\n",
    "results = []\n",
    "\n",
    "response = await results_api.query_results_v2(post_body=results_query)\n",
    "while response.continuation_token:\n",
    "    results = results + response.results\n",
    "    results_query.continuation_token = response.continuation_token\n",
    "    response = await results_api.query_results_v2(post_body=results_query)\n",
    "\n",
    "results_list = [result.to_dict() for result in results]\n",
    "#results_list"

examples/dashboards/Yield and Throughput/FirstPassYield.ipynb:542

• The yield denominator (and Total calculation) adds ERRORED twice: ... + df_grouped['ERRORED'] + ... + df_grouped['ERRORED']. This will understate yield and overstate totals. Sum each status exactly once (and consider whether RUNNING should be included/excluded consistently).

    "df_first_pass_yield = []\n",
    "print(df_first_pass_yield)\n",
    "df_first_pass_yield = pd.DataFrame(100 * df_grouped['PASSED'] / (df_grouped['FAILED'] + df_grouped['ERRORED'] + df_grouped['PASSED'] + df_grouped['TERMINATED']+ df_grouped['TIMED_OUT']+ df_grouped['ERRORED']))\n",
    "df_first_pass_yield['1'] = pd.DataFrame((df_grouped['FAILED'] + df_grouped['ERRORED'] + df_grouped['PASSED'] + df_grouped['TERMINATED']+ df_grouped['TIMED_OUT']+ df_grouped['ERRORED']))\n",
    "df_first_pass_yield['2'] = pd.DataFrame(df_grouped['PASSED'])\n",

examples/dashboards/Yield and Throughput/FirstPassYield.ipynb:235

• This notebook is committed with executed outputs (e.g., the printed results_filter output here) and populated execution_count values. Most notebooks in this repo keep outputs: [] and execution_count: null to avoid stale data and noisy diffs—please clear outputs before committing.

   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "startedWithin <= \"28.0:0:0\" && workspace = \"a67109d9-0ba8-4c47-9612-a7863bf08f07\" && partNumber = \"TCU2\" && programName = \"TCU2-Main.seq\" \n"
     ]
    }

examples/dashboards/Yield and Throughput/Yield-and-Throughput.json:114

• Notebook targets set a hard-coded workspace ID (e2897cf7-...) even though the dashboard defines a $Workspace variable and passes workspace_filter: "$Workspace". This makes the dashboard non-portable and can point at the wrong SystemLink workspace. Prefer using the $Workspace variable (or omitting this field if workspace_filter is the intended selector).

            "testProgram_filter": "$TestProgram",
            "workspace_filter": "$Workspace"
          },
          "refId": "A",
          "workspace": "e2897cf7-6332-433c-8284-65b7b628f3f6"
        }

examples/dashboards/Yield and Throughput/Yield-and-Throughput.json:660

• Field name casing is inconsistent: stat panels filter for "Yield", but this timeseries panel filters for "yield". The notebook code currently renames the column to "Yield" (FirstPassYield.ipynb), so this panel likely won’t find the field. Standardize on one field name/case across the notebook output and all dashboard transformations.

          "id": "filterFieldsByName",
          "options": {
            "include": {
              "names": [
                "started_at",
                "yield",
                "Total"
              ]

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.