Anytopo by ziyuezzy · Pull Request #2598 · sstsimulator/sst-elements

ziyuezzy · 2025-11-04T12:40:44Z

Hi Devs,

This is a pull request, previously mentioned in issue #2584

There are documentations of this work in
"sst-elements-src/src/sst/elements/merlin/interfaces/endpointNIC/documentations/README.md", and
"sst-elements-src/src/sst/elements/merlin/topology/anytopo_ultility/README.md"

In a few words, this branch extends the Merlin module in SST-element, such that any network topology can be built with an input networkx graph. Another key feature is to support source routing (the endpoint NIC determines the routed paths). And a few popular HPC network topologies (dragonfly, slimfly, polarfly, jellyfish) are already defined in
"sst-elements-src/src/sst/elements/merlin/topology/anytopo_ultility/" dir, together with some tests in "sst-elements-src/src/sst/elements/merlin/topology/anytopo_ultility/tests".

If you would like to accept this pull requests, maybe we can move these tests to the main merlin test dir?

It worth mention that the reorderedlinkcontrol has been modified to fit in the new 'ExtendedRequest' framework, this has been included in tests as well, see EndpointNIC(use_reorderLinkControl=True ... in the tests.

I believe that this pull request will have the following contribution to Merlin:

supporting more network topologies (any topology from input graph)
supporting source routing, with is increasingly interesting in HPC network traffic engineering
Adds a framework of EndpointNIC that allows different NIC/smartNIC functionalities to be plugged in the packet-processing pipeline. For now source routing is implemented through this framework.

Please let me know if there is any feedback/comments/questions for the code.

Best regards,
Z.

Automatically Merged using SST Master Branch Merger

…into anytopo

sst-autotester · 2026-01-05T11:47:45Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

feldergast · 2026-01-05T17:36:09Z

Sorry for the delay, things got a bit hectic during December. I will be looking at this PR this week, though it may take me a bit to get through since I'll have to remind myself of the details in the merlin code.

feldergast · 2026-01-05T19:05:08Z

src/sst/elements/merlin/interfaces/ExtendedRequest.h

+class ExtendedRequest : public SST::Interfaces::SimpleNetwork::Request {
+private:
+    // Map from plugin name/key to metadata
+    std::map<std::string, std::shared_ptr<PluginMetadata>> metadata;


Using shard_ptr in classes that end up in events is not supported since std::shared_ptr is not serializable for events used in synchronization. This is because pointers are not tracked during event serialization, so nothing will end up being shared outside of the rank it was created on. I know this will cause some memory increase, but, in general, elements in the merlin library are assumed to be able to run across multiple ranks. I'm going to continue to review the code, but I think this will need to be changed before we can merge it.

src/sst/elements/merlin/interfaces/endpointNIC/NICPlugin.h

feldergast · 2026-01-05T19:38:12Z

src/sst/elements/merlin/interfaces/endpointNIC/SourceRouting.h

+{
+public:
+    // Static shared data accessible by both SourceRoutingPlugin and topology classes
+    static std::map<int, int> endpoint_to_router_map;  // Shared endpoint-to-router mapping


Generally, static variables shouldn't be used in SubComponents. They should use the SharedObjects provided by the core. Since both of the classes that need access are derived from SubComponent, they will have access to those APIs.

Ok, a minor question:
these SharedObjects support array of common data types (int, bool, ...) right?
Can we also share arbitrary objects like mutex of iostream?

The SharedObjects pretty much support any Object that is serializable, which means that a mutex would not be supported. If you need to have a shared Mutex, then that will still need to be a static variable. I/O generally doesn't fall under the same restrictions as variables used during the simulation.

ziyuezzy · 2026-01-06T16:31:09Z

Hi, no worries, we are not in any hurry.
And thanks for giving feedbacks on the code, I will try to fix the points you raised.

Recently I have been thinking about the design of this branch.
Of course importing network topology from graph, and making the routers do source-routing, is probably necessary to be implemented in Merlin.
But this whole concept of 'EndpointNIC with plugins', feels suitable to be an individual sst-element, similar to the 'rdmaNic' element?
Implementing these in Merlin, may benefit its extensibility. For example, I recently implemented another small plugin (not in this branch) that dumps traffic traces from Merlin, it was really clean and efficient. I wonder whether it is even feasible to implement circuit-switching within this framework....
But on the other hand, with a lot of plugins, this framework might also make Merlin complicated/confusing to read for people just started to use this simulator... I don't know if that is a concern or not.

sst-autotester · 2026-01-07T10:17:48Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

sst-autotester · 2026-01-16T11:17:49Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

Automatically Merged using SST Master Branch Merger

…ding ugal routing with the source routing mechanisms.

sst-autotester · 2026-01-16T18:17:45Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

feldergast · 2026-01-20T18:33:33Z

src/sst/elements/merlin/interfaces/ExtendedRequest.h

+    void serialize_order(SST::Core::Serialization::serializer &ser) override {
+        Request::serialize_order(ser);
+
+        // Manually serialize the metadata map since std::variant isn't directly supported


std::variant is supported by the serializer now, but it was developed primarily for checkpointing and may not work properly with pointer tracking turned off. I'd have to look through that code to know if pointer tracking matters. It may be worth trying to see if you can just serialize the vector directly.

Automatically Merged using SST Master Branch Merger

sst-autotester · 2026-01-26T14:48:17Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

Automatically Merged using SST Master Branch Merger

…ation

sst-autotester · 2026-02-02T10:47:52Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

feldergast · 2026-02-02T21:11:31Z

Just an update. I've worked through about a third of the code changes, but have a deadline this week and then have a week of vacation. I will continue to review code when I get back.

sst-autotester and others added 30 commits August 17, 2023 07:57

Merge pull request sstsimulator#2204 from sstsimulator/devel

1e8894c

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2206 from sstsimulator/devel

72f5867

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2212 from sstsimulator/devel

c83f878

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2215 from sstsimulator/devel

9b2e8aa

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2217 from sstsimulator/devel

334ce4e

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2222 from sstsimulator/devel

f5c9109

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2228 from sstsimulator/devel

21a495d

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2231 from sstsimulator/devel

5cc3503

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2237 from sstsimulator/devel

35d6ec7

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2245 from sstsimulator/devel

2bffcee

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2246 from sstsimulator/devel

d8a9ca8

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2254 from sstsimulator/devel

22276fe

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2259 from sstsimulator/devel

4c43f48

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2262 from sstsimulator/devel

ebee81a

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2267 from sstsimulator/devel

a4561e3

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2272 from sstsimulator/devel

64ba3f8

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2279 from sstsimulator/devel

7e3c94a

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2281 from sstsimulator/devel

03a13c0

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2292 from sstsimulator/devel

bce52e3

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2295 from sstsimulator/devel

0011cc7

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2298 from sstsimulator/devel

a0d0980

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2301 from sstsimulator/devel

aa9d85a

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2303 from sstsimulator/devel

7e67f8f

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2305 from sstsimulator/devel

e3d68e7

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2307 from sstsimulator/devel

d46db5f

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2309 from sstsimulator/devel

0a9aaa0

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2312 from sstsimulator/devel

eb16f08

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2316 from sstsimulator/devel

8e32145

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2322 from sstsimulator/devel

54843c2

Automatically Merged using SST Master Branch Merger

Merge pull request sstsimulator#2332 from sstsimulator/devel

50b8b83

Automatically Merged using SST Master Branch Merger

ziyzhang and others added 2 commits January 5, 2026 12:07

Merge branch 'devel' of https://github.com/sstsimulator/sst-elements …

7e5bea2

…into anytopo

adding library checks for slimfly and polarfly

f074514

feldergast reviewed Jan 5, 2026

View reviewed changes

src/sst/elements/merlin/interfaces/endpointNIC/NICPlugin.h Show resolved Hide resolved

feldergast reviewed Jan 5, 2026

View reviewed changes

ziyuezzy added 2 commits January 7, 2026 09:17

fixing metadata serialization

a645afc

adding plugin_complete

c171090

fixing static variables, using shared array instead

7bf9a31

ziyuezzy force-pushed the anytopo branch from 3b28033 to 7bf9a31 Compare January 7, 2026 12:19

ziyuezzy requested a review from feldergast January 14, 2026 10:16

sst-autotester and others added 2 commits January 16, 2026 08:10

Merge pull request sstsimulator#2623 from sstsimulator/devel

036b481

Automatically Merged using SST Master Branch Merger

fixing a small bug in source routing (num_hops did not increment), ad…

3e4fe83

…ding ugal routing with the source routing mechanisms.

ziyuezzy force-pushed the anytopo branch from 064192c to 3e4fe83 Compare January 16, 2026 17:54

feldergast reviewed Jan 20, 2026

View reviewed changes

sst-autotester and others added 3 commits January 21, 2026 08:19

Merge pull request sstsimulator#2626 from sstsimulator/devel

0fb52ff

Automatically Merged using SST Master Branch Merger

adding verbose debug

6dc76a6

fixing serlization

85c231e

ziyuezzy requested a review from feldergast January 27, 2026 09:11

sst-autotester and others added 2 commits January 30, 2026 08:11

Merge pull request sstsimulator#2630 from sstsimulator/devel

a5a1bf1

Automatically Merged using SST Master Branch Merger

removing assertions, replace with fatal calls; fix ugal weight calcul…

2f0fa67

…ation

Merge branch 'sstsimulator:master' into anytopo

820f7f2

Conversation

ziyuezzy commented Nov 4, 2025

Uh oh!

sst-autotester commented Jan 5, 2026

Uh oh!

feldergast commented Jan 5, 2026

Uh oh!

feldergast Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

feldergast Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

ziyuezzy Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

feldergast Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

ziyuezzy commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sst-autotester commented Jan 7, 2026

Uh oh!

sst-autotester commented Jan 16, 2026

Uh oh!

sst-autotester commented Jan 16, 2026

Uh oh!

feldergast Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

sst-autotester commented Jan 26, 2026

Uh oh!

sst-autotester commented Feb 2, 2026

Uh oh!

feldergast commented Feb 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ziyuezzy commented Jan 6, 2026 •

edited

Loading