APOSMM accepts an initial sample, via ingest #1616

jlnav · 2025-11-07T22:22:13Z

Based on concepts and keyword argument names from shuds' #1615 .

New do_not_produce_sample_points field, specifying to APOSMM that user will ingest first.
Fix various circumstances and conditions where specifying sim_id shouldn't be necessary.
Within gen_f: if new field is specified, keep receiving until initial_sample_size points received. Then update local_H as though "actual" evaluations were received.
Calling ingest first on an interfacer generator starts the background process.
Unit test for the above
[11/14] Try to account for a handful of error conditions
[11/21]

Warnings about APOSMM not supporting constraints/constants in VOCS
Much more error handling in APOSMM. RuntimeError s and ValueError s instead of hanging or failing silently
APOSMM enables "generate_sample_points" option based on whether suggest or ingest is called first
Initial sample ingests don't need sim_ids
LibensembleGenerator class maps _id to sim_id
Some Renaming of attributes to be internal (leading underscore)
Additional functionality test using UniformSample + executor
Additional regression test using APOSMM + executor
Unit test for APOSMM's error handling.
Unit test for APOSMM ingesting first
Unit test for consecutive ingests and suggests during APOSMM's sample phase
Fix an issue where sim_id 's default dtype was interpreted as a float

… an attempt to make the arg perform an extra receive so the user can pass in those values. add the boilerplate for an extra test

…an happen as expected

…ust pass the data as-is into the gen_f

…ition where aposmm receives until enough have been received, then local_H is updated with the method used when libE has returned points. Then fix something_sent condition. Also the update_local_H_after_receiving resizes local_H in this initialization routine. Then in generators.py, ensure additionally that Work can ingest points without sim_id needing to be in results. Assume that an np.arange is good enough

codecov · 2025-11-13T16:34:02Z

Codecov Report

❌ Patch coverage is 2.77778% with 70 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.66%. Comparing base (90fe2dc) to head (ba1e2f3).
⚠️ Report is 30 commits behind head on experimental/jlnav_plus_shuds_asktell.

Files with missing lines	Patch %	Lines
libensemble/gen_classes/aposmm.py	0.00%	32 Missing ⚠️
libensemble/generators.py	5.00%	19 Missing ⚠️
libensemble/gen_funcs/persistent_aposmm.py	7.14%	13 Missing ⚠️
libensemble/utils/misc.py	0.00%	3 Missing and 1 partial ⚠️
libensemble/utils/runners.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@                            Coverage Diff                            @@
##           experimental/jlnav_plus_shuds_asktell    #1616      +/-   ##
=========================================================================
- Coverage                                  47.95%   47.66%   -0.29%     
=========================================================================
  Files                                         80       80              
  Lines                                       8079     8125      +46     
  Branches                                    1222     1234      +12     
=========================================================================
- Hits                                        3874     3873       -1     
- Misses                                      3985     4032      +47     
  Partials                                     220      220

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…, primarily displaying a series of warnings about the characteristics of vocs

…interacting with the aposmm class may fail. add supported globus_compute_sdk version to dev env

shuds13 · 2025-11-18T18:45:42Z

This should work

import numpy as np
from xopt import Xopt
from libensemble.gen_classes.aposmm import APOSMM
from xopt import Evaluator
from gest_api.vocs import VOCS
import pdb_si

def six_hump_camel(H, _, sim_specs):
    """Six-Hump Camel sim_f."""

    batch = len(H["x"])  # Num evaluations each sim_f call.
    H_o = np.zeros(batch, dtype=sim_specs["out"])  # Define output array H

    for i, x in enumerate(H["x"]):
        H_o["f"][i] = six_hump_camel_func(x)  # Function evaluations placed into H

    return H_o


def six_hump_camel_func(x):
    """Six-Hump Camel function definition"""
    x1 = x[0]
    x2 = x[1]
    term1 = (4 - 2.1 * x1**2 + (x1**4) / 3) * x1**2
    term2 = x1 * x2
    term3 = (-4 + 4 * x2**2) * x2**2

    return term1 + term2 + term3

def evaluator_function(inputs):
    return {"f": six_hump_camel_func([inputs["x0"], inputs["x1"]])}
evaluator = Evaluator(function=evaluator_function)

vocs = VOCS(
    variables={
        "x0": [-2.0,2.0],
        "x1": [-1.0,1.0],
        "x0_on_cube": [0.0,1.0],
        "x1_on_cube": [0.0,1.0],
        },
    objectives={"f": "MINIMIZE"},
)

variables_mapping = {
    "x": ["x0", "x1"],
    "x_on_cube": ["x0_on_cube", "x1_on_cube"],
}

max_active_runs = 1
initial_sample_size = 20
gen = APOSMM(vocs=vocs, max_active_runs=max_active_runs, initial_sample_size=initial_sample_size, variables_mapping=variables_mapping)

X = Xopt(vocs=vocs, evaluator=evaluator, generator=gen)

X.random_evaluate(10)
X.step()

for i in range(30):
    print(i)
    X.step()

Also should work if use suggest instead of ingest, unless we decided not.

We can put a X.generator.finalize() at the end to prevent hang

shuds13 · 2025-11-18T18:46:11Z

libensemble/gen_funcs/persistent_aposmm.py

+            initial_sample_size=6,
+            variables_mapping=variables_mapping,
+            ...
+            do_not_produce_sample_points=True,


i dont actually want to call it this

And when using aposmm.py i think it should default to True.

Or maybe call generate_sample and True default in persistent_aposmm but sets to false by default in aposm.py

shuds13 · 2025-11-18T18:53:55Z

libensemble/gen_classes/aposmm.py

+
+    do_not_produce_sample_points: bool = False
+
+        If `True`, APOSMM can ingest sample points (with matching objective values)


what about suggest?

Do we support them starting with suggest.

shuds13 · 2025-11-18T18:54:39Z

libensemble/gen_classes/aposmm.py

    """

+    def _validate_vocs(self, vocs: VOCS):
+        if len(vocs.constraints):


does aposmm support constraints

shuds13 · 2025-11-18T18:55:01Z

libensemble/gen_classes/aposmm.py

+    def _validate_vocs(self, vocs: VOCS):
+        if len(vocs.constraints):
+            warnings.warn("APOSMM's constraints are provided as keyword arguments on initialization.")
+        if len(vocs.constants):


same for constants.

shuds13 · 2025-11-18T18:55:48Z

libensemble/gen_funcs/persistent_aposmm.py

+UNEXPECTED_SIMID_ERR = """APOSMM received unexpected input data.
+
+APOSMM *typically* expects to provide sample points itself following initialization.
+If you wish to provide sample points with matching objective values to APOSMM,


I would say evaluated points, "matching objective values" is a bit confusing. Matching with what...

shuds13 · 2025-11-18T18:58:12Z

Does it support the rule, once a suggest() is called, you cannot give more pre-evaluated points (points with no _id).

shuds13 · 2025-11-18T19:00:46Z

How is "_id" not overwritten still in calling structure?

misc.py still has

    for entry in list_dicts:
        if "_id" in entry:
            entry["sim_id"] = entry.pop("_id")

which overwrites the structure in user space.

Did you look at using

        if "sim_id" not in self.variables_mapping:
            self.variables_mapping["sim_id"] = ["_id"]

in generators.py?

shuds13 · 2025-11-18T19:39:43Z

"Fix various circumstances and conditions where specifying sim_id shouldn't be necessary."

I know of only one - giving pre-evaluated points.

…ing message. appends _id to sim_id mapping in generators.py and removes the associated pop in misc.py

shuds13 · 2025-11-19T17:57:44Z

Given our rule that sample is either all ingest or all suggest.

I dont think the generate_points option is needed to gest-api APOSMM. Given that the backend process is started on either the first call to suggest or ingest. If its started on the suggest(), then it can let APOSMM generate points, if started on ingest it goes the other path. Can this be done?

…inverse

… using as indexes later). need to accomodate circumstance where we have set up a sim_id <- _id mapping, but no input _id data is appearing, since this is acceptable for an initial sample

…djust tests to match expected behavior

…t or ingest is called first

…eError on unexpected usage instead of failing/returning silently. Additional tests for coverage

…larifying comments

…ecutor test

shuds13 · 2025-11-24T17:39:41Z

why set in not first call???

    if not self._first_call:
        self._first_call = "suggest"
        self.gen_specs["user"]["generate_sample_points"] = True

shuds13 · 2025-11-26T20:39:03Z

libensemble/gen_classes/aposmm.py

+            warnings.warn("APOSMM does not support constraints in VOCS. Ignoring.")
+        if len(vocs.constants):
+            warnings.warn("APOSMM does not support constants in VOCS. Ignoring.")
+        if len(vocs.observables):


aposmm DOES support observables, it may not do anything with them, but we use them in multiple examples, including critically in IBCDFO.

Or actually we do use them as fvec is returned as observables.

And gradients can be also.

shuds13 · 2025-11-26T20:51:05Z

libensemble/gen_classes/aposmm.py

    def suggest_numpy(self, num_points: int = 0) -> npt.NDArray:
        """Request the next set of points to evaluate, as a NumPy array."""
+
+        if not self._first_call:


name should be _first_call_type or _initial_call_type and test should be that its None.

Also I think longer term this should be a option for all libE gens like this so would go in generators.py just before setup is called.

shuds13 · 2025-12-02T20:01:37Z

libensemble/gen_classes/aposmm.py

+    retrieved via `.suggest()`, updated with objective values, and ingested via `.ingest()`.
+
+    ```python
+    gen = APOSMM(vocs, max_active_runs=2, initial_sample_size=10, generate_sample_points=True)


I thought we dont need this option now.

…s (like the macOS jobs)

jlnav · 2025-12-03T19:16:09Z

~~I think I'd prefer to experiment with the APOSMM-in-xopt use-case in another branch off of ask-tell develop once this one is pulled in, since this one is getting large~~

Nevermind I fixed it!

…o the gen_f. add a initialize_dists_and_inds call upon the initial sample being ingested in, inside update_local_H_after_receiving

shuds13 · 2025-12-09T17:14:23Z

I think we may have to change using variables_mapping. It maybe that we always convert _id to sim_id and vice versa as this can happen in either direction.

E.g. listdicts_2_np

xopt ingests on libE gen:
Ingest with _id, becomes sim_id in generator.

libE calls suggest() on gest-api gen:
Suggest produces _id, we want that to set sim_id....

So conversion always needs to happen.

We could address that on another branch

shuds13

Will need to resolve conflicts

…_aposmm_refactor

jlnav added 9 commits November 6, 2025 10:34

also start running thread upon initial ingest

3e41b4b

better docstrings. new "do_not_produce_sample_points" arg from shuds.…

2303512

… an attempt to make the arg perform an extra receive so the user can pass in those values. add the boilerplate for an extra test

add kwarg where necessary, plus more test glue

802de9c

unpack "x"s passed in into dtype-less array. so _on_cube conversion c…

fa254a0

…an happen as expected

TODOs, plus more

1606ef3

mostly just trying the current aposmm.py with Xopt in this commit...

473487d

fix initial sample (within aposmm *class*) requiring sim_ids. we'll j…

b42e1ba

…ust pass the data as-is into the gen_f

add unit test where APOSMM class ingests first

5dc5155

jlnav marked this pull request as ready for review November 13, 2025 16:24

jlnav requested review from jmlarson1 and shuds13 November 13, 2025 16:24

jlnav force-pushed the asktell/nov_aposmm_refactor branch from 0dfae0a to 5dc5155 Compare November 14, 2025 15:22

jlnav mentioned this pull request Nov 14, 2025

CommFinishedException from APOSMM after initial evaluations are done #1613

Open

jlnav added 2 commits November 14, 2025 11:01

make an internal method internal. implement _validate_vocs for aposmm…

ed03ac1

…, primarily displaying a series of warnings about the characteristics of vocs

trying to account for, then test a large variety of conditions where …

b7a79c3

…interacting with the aposmm class may fail. add supported globus_compute_sdk version to dev env

shuds13 reviewed Nov 18, 2025

View reviewed changes

jlnav added 2 commits November 19, 2025 11:14

adjusts send-conditions upon full sample being ingested. adjusts warn…

1d335c8

…ing message. appends _id to sim_id mapping in generators.py and removes the associated pop in misc.py

additional docs

b23d13c

tentative refactor of do_not_produce_sample_points keyword to be the …

42f772a

…inverse

jlnav added 8 commits November 19, 2025 16:45

doing "generate_sample_points". ensuring sim_id gets cast as int (for…

95bc6a0

… using as indexes later). need to accomodate circumstance where we have set up a sim_id <- _id mapping, but no input _id data is appearing, since this is acceptable for an initial sample

add onto n_s with the len of received presumptive_user_sample

f399750

fix conditions for something_sent after initial_sample is ingested. a…

4ca2153

…djust tests to match expected behavior

internally, generate_sample_points is now set based on whether sugges…

635439d

…t or ingest is called first

various fixes; sim_id warning no longer needed in gen_f. raise RunTim…

854a336

…eError on unexpected usage instead of failing/returning silently. Additional tests for coverage

always map _id to sim_id. add simple executor + UniformSample test. c…

4dbfc42

…larifying comments

put the executor.comm adjustment back - and add a asktell aposmm + ex…

f4f6b96

…ecutor test

other MPI processes attempting to finalize their gen is problematic

2c28939

jlnav requested a review from shuds13 November 21, 2025 21:26

shuds13 reviewed Nov 26, 2025

View reviewed changes

tweaks based on PR feedback

4fda921

shuds13 reviewed Dec 2, 2025

View reviewed changes

jlnav added 2 commits December 3, 2025 10:14

doc/docstring additional tweaks and clarifications

548f580

bump polling_loop interval for (perhaps) slower containers/VMs/system…

aaac936

…s (like the macOS jobs)

jlnav requested a review from shuds13 December 3, 2025 19:15

set the sim_ids of the ingest_buf right when we're about to send it t…

e9fa694

…o the gen_f. add a initialize_dists_and_inds call upon the initial sample being ingested in, inside update_local_H_after_receiving

shuds13 approved these changes Dec 9, 2025

View reviewed changes

jlnav added 3 commits December 9, 2025 14:40

Merge branch 'develop' into asktell/nov_aposmm_refactor

d5fbe14

Merge branch 'experimental/jlnav_plus_shuds_asktell' into asktell/nov…

68f1d4e

…_aposmm_refactor

Merge branch 'experimental/jlnav_plus_shuds_asktell' into asktell/nov…

ba1e2f3

…_aposmm_refactor

shuds13 merged commit 3daf0c1 into experimental/jlnav_plus_shuds_asktell Dec 9, 2025
12 of 14 checks passed

shuds13 deleted the asktell/nov_aposmm_refactor branch December 9, 2025 22:34

jlnav mentioned this pull request Dec 10, 2025

Required sim_id key in APOSMM #1612

Closed


		do_not_produce_sample_points: bool = False

		If `True`, APOSMM can ingest sample points (with matching objective values)

APOSMM accepts an initial sample, via ingest #1616

APOSMM accepts an initial sample, via ingest #1616

Uh oh!

Conversation

jlnav commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

shuds13 commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shuds13 Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shuds13 commented Nov 18, 2025

Uh oh!

shuds13 commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shuds13 commented Nov 18, 2025

Uh oh!

shuds13 commented Nov 19, 2025

Uh oh!

shuds13 commented Nov 24, 2025

Uh oh!

shuds13 Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shuds13 Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jlnav commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shuds13 commented Dec 9, 2025

Uh oh!

shuds13 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jlnav commented Nov 7, 2025 •

edited

Loading

codecov bot commented Nov 13, 2025 •

edited

Loading

shuds13 commented Nov 18, 2025 •

edited

Loading

shuds13 Nov 18, 2025 •

edited

Loading

shuds13 commented Nov 18, 2025 •

edited

Loading

shuds13 Nov 26, 2025 •

edited

Loading

shuds13 Nov 26, 2025 •

edited

Loading

jlnav commented Dec 3, 2025 •

edited

Loading