Want to disable step execution logging, want to disable PASSED logs

[kazurayam]

In one application of VisualTestingInKatalonStudio, my Test Suite executes opening/verifyElementPresent/delay for 300 URLs. The test suite emitted 5000 lines of step execution log. This volume of log is huge. When I clicked the LogView for this, I had to wait for 10 minutes until the 5000 lines of logs are loaded into the GUI. It's silly, because those 5000 lines are not informative at all.

I want to disable those step execution logging, and want to disable PASSED logs.

[kazurayam]

Prior to ver 7.3.1 the option of enbling/disabling step execution log was available, but eventually that option turned to be only available in Katalon Studio Professional. It means, the option is not free now.

https://forum.katalon.com/t/project-settings-executions-enable-disable-logging-executed-steps-is-it-deprecated/41799

[kazurayam]

I can read the source of WebUI.verifyElementPresent keyword at https://github.com/katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/webui/keyword/builtin/VerifyElementPresentKeyword.groovy

At Line#82 I found this:

if (foundElement != null) {
                        logger.logPassed(MessageFormat.format(StringConstants.KW_LOG_PASSED_OBJ_X_IS_PRESENT, to.getObjectId()))
                    }

This logger.logPassed() call emits a verbose PASSED log.

I can turn logPassed() method do nothing, then I can reduce the amount of execution log. So I read the source more and found that com.kms.katalon.core.logging.KeywordLogger class is most interesting.

https://github.com/katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/logging/KeywordLogger.java

[kazurayam]

In KeywordLogger I found the implementation of logPassed method, which is as simple as this:

public void logPassed(String message) {
        logPassed(message, null);
    }


    public void logPassed(String message, Map<String, String> attributes) {
        logger.debug("{} {}", PASSED, message);
        xmlKeywordLogger.logPassed(message, attributes);
    }

I would be able to modify these implementation by Groovy Metaprogramming as follows

public void logPassed(String message) {
        // does nothing
    }


    public void logPassed(String message, Map<String, String> attributes) {
        // does nothing
    }

then the amount of log messages will be much reduced.

[kazurayam]

I found START, END logs are also verbose. I thought these are not informative, and wanted to disable them.
I read the source of KeywordLogger and found the following code:

public void startKeyword(String name, Map<String, String> attributes, int nestedLevel) {
    	
    	if (shouldLogTestSteps()) {
	        logStartKeyword(name, attributes);
	        xmlKeywordLogger.startKeyword(name, attributes, nestedLevel);
    	}
    }

This shouldLogTestSteps() method is interesting. If I could modify this method to return false, then the amout of log messages will be much reduced.

I believe that the version 7.3.1 of Katalon Studio did something about this shouldLogTestSteps().

How to modify the method? --- It looks easy. I found the following code:

private boolean shouldLogTestSteps() {
    	return shouldLogTestSteps;
    }

then why not to modify this to

private boolean shouldLogTestSteps() {
    	return false
    }

[kazurayam]

I tried a bit and soon found it is difficult to achieve this: modifying the behavior of logger.

The logger is instantiated by each caller classes. The logger instances are identified by the caller class name and cached as singleton managed as private inside the KeywordLogger class. It is impossible for outsiders to modify the cached /private instances.

It seems there is no way

[kazurayam]

Look at Line#71 of
https://github.com/katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/logging/KeywordLogger.java

private void initShouldLogTestSteps() {
		Map<String, Object> executionProperties = RunConfiguration.getExecutionProperties();
		if (executionProperties == null) {
			shouldLogTestSteps = true;
		} else {
			shouldLogTestSteps = (boolean) Optional
	                .ofNullable(executionProperties.get(RunConfiguration.LOG_TEST_STEPS))
	                .orElse(false);
		}
	}

If I modify RunConfiguration.LOG_TEST_STEPS to be false, then I guess many of STRT/END log would be disabled. Let me try it.

[kazurayam]

I wrote a report titled "Log Viewer slows down your tests, how to prevent it" and published it in the Katalon Forum.
https://forum.katalon.com/t/log-viewer-slows-down-your-tests-how-to-prevent-it/60252/4

I insisted that "START" and "END" logs makes the stream of log message superfluous, makes the Log Viewer heavy, and slows the test down significantly. Therefore now I want to find a way somehow to stop "START" and "END" logs completely.

I can achieve it. I can change the KeywordLogger, modify the startKeyword() and the endKeyword() so that they do not call logging API at at. As simple as this.

[kazurayam]

I studied a bit and found it is difficult to stop "START" and "END" logs completely.

I tried to modify com.kms.katalon.core.logging.KeywordLogger#startKeyword() method by Groovy's meta prgramming.
I could write the code.
But it had no effect.
Why?

The KeywordLogger#startKeyword is not a static method.
The KeywordLogger#startKeyword is an instance method.
Katalon Studio creates an instance of the KeywordLogger object during its start-up process, before my Test Suite/Test Case starts.
Therefore, even if my Test Listener modifies the startKeyword method implementation, it has no effect to the already-created & cached instance.

[kazurayam]

See https://github.com/katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/logging/KeywordLogger.java

KeywordLogger class has a static property where instances of KeywordLogger are cached. Katalon Studio creates many instances of KeywordLogger in its startup process, and these instances are used through out. Yes, I can write a Groovy script that override methods that write "START" and "END" logs, but when a Test Listener calls the tool, it is too late. Cached instances will be used.

KeywordLogger class is written in Java, and KeywordLogger does not implement a public method that enable me to clear the cache.

[kazurayam]

I have written a Test Case TC1 in Katalon Studio

import com.kms.katalon.core.logging.KeywordLogger
import java.lang.reflect.Field
import java.lang.reflect.Modifier

// https://qiita.com/5at00001040/items/83bd7ea85d0f545ae7c3
// https://stackoverflow.com/questions/3301635/change-private-static-final-field-using-java-reflection

KeywordLogger instance = new KeywordLogger();

// obtain a Field object throuch which I update the target
Field targetField = instance.getClass().getDeclaredField("shouldLogTestSteps")
assert targetField != null

println "before: ${getPrivateBooleanFieldValue(instance, targetField)}"

setPrivateBooleanFieldWithValue(instance, targetField, Boolean.FALSE)

println "after : ${getPrivateBooleanFieldValue(instance, targetField)}"

def setPrivateBooleanFieldWithValue(Object obj, Field targetField, Object newValue) {
	Objects.requireNonNull(targetField)
	Objects.requireNonNull(newValue)
	//
	targetField.setAccessible(true)
	Field modifiers = Field.class.getDeclaredField("modifiers")
	modifiers.setAccessible(true)
	modifiers.setInt(targetField,
		targetField.getModifiers() & ~Modifier.PRIVATE & ~Modifier.FINAL)
	//
	targetField.set(obj, newValue)
}

Boolean getPrivateBooleanFieldValue(Object obj, Field targetField) {
	Objects.requireNonNull(obj)
	Objects.requireNonNull(targetField)
	targetField.setAccessible(true)
	return targetField.getBoolean(obj)
}

When I run this, I got:

2021-11-25 09:31:10.835 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2021-11-25 09:31:10.839 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC1
2021-11-25 09:31:11.602 WARN  c.k.katalon.core.logging.KeywordLogger   - Please use "KeywordUtil.logInfo()" instead of "new KeywordLogger()" constructor. "KeywordLogger" is an internal API and might be changed in the future.
before: true
after : false
2021-11-25 09:31:11.770 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/TC1

I could modify an instance of com.kms.katalon.core.logging.KeywordLogger.
I could update the value of its property private static boolean shouldLogTestSteps from true to false
Attention please! The property is declared as private static.