Skip to content

Fix Missing PQC KEM Verbose Tests#53

Open
Guiliano99 wants to merge 6 commits into
siemens:mainfrom
Guiliano99:FixPQCKEMVerboseTests
Open

Fix Missing PQC KEM Verbose Tests#53
Guiliano99 wants to merge 6 commits into
siemens:mainfrom
Guiliano99:FixPQCKEMVerboseTests

Conversation

@Guiliano99

Copy link
Copy Markdown
Contributor

Fixed missing verbose test for the PQC KEM algorithms, which were completely missing.

Description

  • Added missing PQC KEM tests to the kem_alg.robot file, generated with scripts/generate_alg_test_cases.py.
  • Refactored test tag structure for better consistency across PQC and hybrid KEM tests.
  • Simplified kem_alg.robot keywords sections to allow easier modification of test cases.
  • Fixed inconsistencies in test tags within the kem_alg.robot file.

TODO

  • Decide to add a separated file for PQC- and Hybrid-KEMs (with also a new URL).

Motivation and Context

  • Improves test completeness by including previously missing PQC-KEM tests and enhances test execution through better tag consistency.

How Has This Been Tested?

MockCA

@Guiliano99
Guiliano99 requested a review from ralienpp as a code owner January 28, 2026 14:03
@ralienpp

Copy link
Copy Markdown
Collaborator

I'm not sure I understand the reasoning behind the automatic generation of test cases by writing to files. A better way to accomplish this is to leverage test templates.

The same technique must be applied to simplify how existing tests are written. For example, there are tests like CA MUST Reject SLH-DSA-SHAKE-128S with Shake128 IR with Invalid POP and CA MUST Reject SLH-DSA-SHA2-128F with Sha256 IR with Invalid POP and the other ones that are essentially copies of each other. They can be rewritten in a more compact way.

Comment on lines 229 to +231
tags.append(test_type)
tags.append(hybrid_type)
tags.extend(["hybrid", "hybrid-kem"])

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A somewhat peculiar choice to use extend here, and append there, no?

@Guiliano99

Guiliano99 commented Jan 28, 2026

Copy link
Copy Markdown
Contributor Author

I'm not sure I understand the reasoning behind the automatic generation of test cases by writing to files. A better way to accomplish this is to leverage test templates.

The same technique must be applied to simplify how existing tests are written. For example, there are tests like CA MUST Reject SLH-DSA-SHAKE-128S with Shake128 IR with Invalid POP and CA MUST Reject SLH-DSA-SHA2-128F with Sha256 IR with Invalid POP and the other ones that are essentially copies of each other. They can be rewritten in a more compact way.

My Idea was to have the test cases static and generate the test cases with a script, so that they can be easily changed, if new combinations or algorithms are added. And for the test case itself are template test cases added.

And additionally we have tags and can ignore algorithms or combinations with the tags and do not use a config variable for that.

To explain it a little bit more, currently I write the test cases into a text file, so that I manually have to copy them into the RF files, to be static and not created during run time.

@Guiliano99

Copy link
Copy Markdown
Contributor Author

But this could also be done via Set Tags keyword.

@Guiliano99

Copy link
Copy Markdown
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants