Skip to content

Support parallel execution and fix issue with testcase stats not available #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kobenguyent merged 2 commits into from
Jan 7, 2022
Merged

Support parallel execution and fix issue with testcase stats not available #8

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3d8264f
Support for parallel execution
ilangv Oct 15, 2021
aef6f97
fix issue with test case widgets not working
ilangv Nov 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 14 additions & 8 deletions index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ module.exports = (config) => {

event.dispatcher.on(event.suite.before, (suite) => {
recorder.add(async () => {
suiteObj = startTestItem(suite.title, rp_SUITE);
suiteObj = startTestItem(suite.title, rp_TEST);

@AmsterGet AmsterGet Jan 11, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After that changes the structure TEST -> STEP -> NESTED STEP (if applicable) will be presented.
Is it a good idea to avoid suites in the reporting structure? I suggest to use the following structure: SUITE -> TEST (if applicable) -> STEP -> NESTED STEP (if applicable), as from reportportal side we expect the top level suite as a container of tests.

@AmsterGet AmsterGet Jan 11, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@m3t4n & @ilangv guys, could you please look at my notes, will it suit you?

@AmsterGet AmsterGet Jan 11, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what do you think about @DavertMik work on this for Codecept 3.0+
#3
May be better to implement it on the integration side to avoid additional work with bootstrapAll and teardownAll?

@m3t4n m3t4n Jan 12, 2022

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @AmsterGet thank you for your attention.
I agree with the suite > test > step > nestedStep structure. However, on the RP side it's represented as LaunchName > Suite > Test > Step > nestedStep and hence I believe @ilangv 's change (i might be wrong). Whereas I agree with you but maybe an improvement point on this is that after Suite the tests can be expanded instead of navigating to a new page (like nested steps do). Personally, I find it hard to reach the steps.

On the other hand @DavertMik's change seems to be the stable choice. Currently, it is not possible to properly display the parallel executions without any update on this plugin side. The suites are randomly placed under launches and this makes it really hard to see which ones are under which launch. Another option might be appending the suite name to launch id (thinking out loud).

@DavertMik: any chance on finalizing the PR? CodeceptJS is already 3.2.3.

debug(`${suiteObj.tempId}: The suiteId '${suite.title}' is started.`);
suite.tempId = suiteObj.tempId;
suiteStatus = rp_PASSED;
Expand All @@ -113,7 +113,7 @@ module.exports = (config) => {
recorder.add(async () => {
currentMetaSteps = [];
stepObj = null;
testObj = startTestItem(test.title, rp_TEST, suiteObj.tempId);
testObj = startTestItem(test.title, rp_STEP, suiteObj.tempId, true);
test.tempId = testObj.tempId;
failedStep = null;
debug(`${testObj.tempId}: The testId '${test.title}' is started.`);
Expand Down Expand Up @@ -211,9 +211,9 @@ module.exports = (config) => {
});
});

function startTestItem(testTitle, method, parentId = null) {
function startTestItem(testTitle, method, parentId = null, stats = null) {
try {
const hasStats = method !== rp_STEP;
const hasStats = stats || (method !== rp_STEP);
return rpClient.startTestItem({
name: testTitle,
type: method,
Expand All @@ -233,7 +233,7 @@ module.exports = (config) => {
status: suiteStatus,
}).promise;
}
await finishLaunch();
if (!process.env.RP_LAUNCH_ID) await finishLaunch();
});

function startLaunch(suiteTitle) {
Expand All @@ -244,13 +244,19 @@ module.exports = (config) => {
debug: config.debug,
});

return rpClient.startLaunch({
const options = {
name: config.launchName || suiteTitle,
description: config.launchDescription,
attributes: config.launchAttributes,
rerun: config.rerun,
rerunOf: config.rerunOf,
});
}

if (process.env.RP_LAUNCH_ID) {
options.id = process.env.RP_LAUNCH_ID
}

return rpClient.startLaunch(options);
}

async function attachScreenshot() {
Expand Down Expand Up @@ -316,7 +322,7 @@ module.exports = (config) => {
metaStepObj = currentMetaSteps[i-1] || metaStepObj;

const isNested = !!metaStepObj.tempId;
metaStepObj = startTestItem(metaStep.toString(), rp_STEP, metaStepObj.tempId || testObj.tempId);
metaStepObj = startTestItem(metaStep.toString(), rp_STEP , metaStepObj.tempId || testObj.tempId, false);
metaStep.tempId = metaStepObj.tempId;
debug(`${metaStep.tempId}: The stepId '${metaStep.toString()}' is started. Nested: ${isNested}`);
}
Expand Down