Experimental reproduction

[YancyLii]

Hello, I am currently using this project for experimentation and encountered three issues during the process:

Firstly, make.yml seems to be incomplete and inaccurate. I have found that the configuration of the filter is incomplete, and the nodes, testact, and filter cannot be executed. Do you have a more complete execution version?

Secondly, how is the coverage information calculated after execution? I am a beginner in fuzzy testing and not very familiar with the use of related tools. I hope to receive your help?

Finally, in the paper, it is mentioned that 25+30 projects were tested together, but the table only includes the results of 25 programs. May I ask if it is convenient to provide results for the other 30 projects? Thank you very much.

I'm very sorry to disturb you.

[yuweol]

Hello.

1. Did you mean python3 -m helper.make {project_name} is failed for node and tesseract? And what is the filter?
2. UTopia creates two types of fuzzers for the same unittest. One fuzzer is built with a sanitizer, and the other is constructed for a coverage profile. First, the fuzzer built with the sanitizer is run to perform fuzzing, and it collects a corpus in a queue directory provided via command-line options. After the fuzzing is complete, provides this queue directory to the fuzzer built for the coverage profile along with the option runs=1. This fuzzer then executes the program once for each corpus in the queue directory, collecting coverage information. By performing this process for all fuzzers created within the project and aggregating the coverage, the overall project coverage can be measured.
3. 30 projects are not public, thus only 25 open-source projects are available.

[YancyLii]

1.Sorry, I typed the wrong letter. What I mean is that in make. yml, the part about filling is incomplete, not "fillter". Indeed, I am unable to complete the compilation of node、tesseract、mediapipe using '''Python 3- m helper. make {projectname}'''
2.May I ask what is your tool for calculating coverage? I first use llvm-profdata merge to merge all the profraw files to obtain profdata, and then use llvm-cov show to analyze the profdata files, which take the form of "llvm-cov show -format=html -output-dir=coverage_report -instr-profile=default.profdata libxxxx_profile.a". I often encounter the situation of "libxxxx_profile.a: Failed to load coverage: No coverage data found".

[yuweol]

First, I could reproduce the problems when we make node and tesseract. I'll fix the problem, but it might take some time.
Second, it's weird, we used same tool tool but that is new issue to me. Did you use the same version of llvm-cov and llvm-profdata as the llvm that is installed on the docker?

[YancyLii]

Thank you for your reply. Will you not encounter errors when compiling “filament” and "mediapipe"? I also encountered issues with these two libraries. I will double check the coverage rate. Thank you again for your help.

[yuweol]

I remembered that "filament" and "mediapipe" are android libraries that requires slightly different environment to build them. I'm not sure i can rebuild that environment but i'll try to find it. It would be great to lower the priority.

[YancyLii]

Okay, thank you for your reply