Add support for advanced SQL features and improvements #1397

jgaleotti · 2025-12-01T18:48:43Z

### Summary

This pull request introduces support for a range of advanced SQL features and general heuristic improvements. The updates ensure better handling and support for complex SQL structures, making the SQL processing logic more robust and comprehensive.

### Changelog

- **Multischema Handling**: Utilized `nullSafeEndsWithIgnoreCase` to retrieve `DataRow` values.
- **SQL Heuristics Improvements**:
  - Added handling for parenthesized select items.
  - Introduced support for Common Table Expressions (CTEs), including alias resolution.
  - Enhanced detection of tables and columns in DELETE statements with subqueries.
  - Improved evaluation of `ORDER BY` expressions.
  - Specific logic added for `SELECT` queries with `LIMIT` clauses.
- **New SQL Function Support**:
  - Added support for functions: `LOWER()`, `NOW()`, `DATE_TRUNC()`, and `COALESCE()`.
  - Enhancements for `TIME()` function.

…IT clause.

… and test cases for SQL non-aggregation functions

… Java <8 and Java 8+ time API

…risticsCalculator

…nd columns involved in a DELETE statement having a subquery.

…per()

…uristics, table alias and table resolution

…ding test

…le multischema

…g-features

arcuri82 · 2025-12-02T08:01:41Z

hi @jgaleotti you requested a review, but then made a new push that broke CI.

jgaleotti · 2025-12-02T15:51:32Z

my bad. I am fixing it. JP

…

On Tue, Dec 2, 2025 at 5:02 AM Andrea Arcuri ***@***.***> wrote: *arcuri82* left a comment (WebFuzzing/EvoMaster#1397) <#1397 (comment)> hi @jgaleotti <https://github.com/jgaleotti> you requested a review, but then made a new push that broke CI. — Reply to this email directly, view it on GitHub <#1397 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTDTPWWB6NVWDDMPJS2HXL37VBPZAVCNFSM6AAAAACNWUTB32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMBQG4ZDOMRYGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

…database to compute the heuristic score

…g-features

jgaleotti · 2025-12-03T20:57:34Z

CI is now passing.

…g-features

…instead of throwing an exception.

…sBooleanExpression and IsNullExpression in subclass

…g-features

...ava/sql/src/main/java/org/evomaster/client/java/sql/heuristic/function/CoalesceFunction.java

...va/sql/src/main/java/org/evomaster/client/java/sql/heuristic/function/DateTruncFunction.java

...t-java/sql/src/main/java/org/evomaster/client/java/sql/heuristic/function/LowerFunction.java

...nt-java/sql/src/main/java/org/evomaster/client/java/sql/heuristic/function/TimeFunction.java

client-java/sql/src/main/java/org/evomaster/client/java/sql/heuristic/TableAliasResolver.java

client-java/sql/src/main/java/org/evomaster/client/java/sql/heuristic/TableNameMatcher.java

client-java/sql/src/main/java/org/evomaster/client/java/sql/internal/SqlSelectBuilder.java

client-java/sql/src/main/java/org/evomaster/client/java/sql/internal/SqlTableId.java

client-java/sql/src/main/java/org/evomaster/client/java/sql/internal/SqlTableIdParser.java

…g-features

…e arguments than expected.

…in TimeFunction / added a test suite for LowerFunction.

…o its own class and added unit tests.

… differentiation, added new test cases for MySQL and MariaDB, and refactored qualified table name building logic.

…g-features

jgaleotti · 2026-01-15T15:21:46Z

CI is now passing. The PR is ready for your revision @arcuri82

jgaleotti added 15 commits November 17, 2025 20:27

introduces specific logic to handle SELECT queries that include a LIM…

1cb0b0a

…IT clause.

support for SQL COALESCE() function, enhancing of SQL TIME() function…

6b7c3cc

… and test cases for SQL non-aggregation functions

support for ORDER BY (ascending and descending)

de4806d

replace size()>0 with !isEmpty()

ea6e025

initial implementation of DateTrunc

44573bc

support for SQL NOW() as canonical value / DATE_TRUNC() supports both…

a7e7f64

… Java <8 and Java 8+ time API

ORDER BY expressions are evaluated as any other expressions in SqlHeu…

057b062

…risticsCalculator

test testDeleteFromWithSubquery validates the detection of tables a…

2d0dde5

…nd columns involved in a DELETE statement having a subquery.

Merge branch 'master' into advanced-sql-missing-features

e84be1b

support for SQL LOWER() function / test cases for both lower() and up…

c61b154

…per()

support for Common Table Expressions (CTE)

f72d733

add support for Common Table Expressions (CTE) with aliases in SQL he…

7ffd2e7

…uristics, table alias and table resolution

handle parenthesized select items in SQL heuristics and add correspon…

01494f2

…ding test

use nullSafeEndsWithIgnoreCase when retrieving DataRow values to hand…

dc4a956

…le multischema

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

a3e0530

…g-features

jgaleotti requested a review from arcuri82 December 1, 2025 18:48

Add multi-schema support in SQL table and column resolution

c1fe77f

jgaleotti added 4 commits December 2, 2025 15:06

Refactor SQL table and column resolution

a93816d

Add support for catalog and schema names when querying data from the …

b8de407

…database to compute the heuristic score

support for catalog/schema/table name

dcc1e68

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

e87b94a

…g-features

jgaleotti added 6 commits December 3, 2025 17:58

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

1031b00

…g-features

support for SELECT DISTINCT and SELECT DISTINCT ON

610d800

If an SQL aggregation returns multiple rows, SQL picks the first one …

fbbb362

…instead of throwing an exception.

Fixing bug in TablesNameFinder by adding missing implementation for I…

e572dbc

…sBooleanExpression and IsNullExpression in subclass

SQL DATE() function

f2ac00e

SQL UUID comparisons

69b82a8

jgaleotti added 5 commits December 5, 2025 10:03

bugfix: invalid database names

fc49ad0

bugfix: handle MYSQL select queries with both database and schema.

36f0506

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

79cf4fd

…g-features

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

f544aa5

…g-features

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

bafec04

…g-features

arcuri82 requested changes Dec 19, 2025

View reviewed changes

jgaleotti added 12 commits December 30, 2025 14:27

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

e25a404

…g-features

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

e6b1bbf

…g-features

Add some explanantion and tests for CoalesceFunction

c69573a

fixed error message plus added test cases for empty arguments and mor…

3767698

…e arguments than expected.

added number of arguments to message errors / improved message error …

89e46d5

…in TimeFunction / added a test suite for LowerFunction.

Refactored TableAliasResolver by extracting TableAliasContext int…

71cae15

…o its own class and added unit tests.

added TODO comments to TableNameMatcher.matches()

b48b596

added logic to detect illegal catalog and schema names

1653850

added some comment to sqlTableId constructor

fc40c87

Updated SqlTableId and SqlTableIdParser to support DatabaseType…

8971efb

… differentiation, added new test cases for MySQL and MariaDB, and refactored qualified table name building logic.

Merge remote-tracking branch 'origin/master' into advanced-sql-missin…

feee946

…g-features

testfix: adding database type to schema

300ccf5

jgaleotti requested a review from arcuri82 January 15, 2026 15:21

arcuri82 approved these changes Jan 16, 2026

View reviewed changes

arcuri82 merged commit fc9f787 into master Jan 16, 2026
15 of 25 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add support for advanced SQL features and improvements #1397

Add support for advanced SQL features and improvements #1397

Uh oh!

jgaleotti commented Dec 1, 2025

Uh oh!

arcuri82 commented Dec 2, 2025

Uh oh!

jgaleotti commented Dec 2, 2025 via email

Uh oh!

jgaleotti commented Dec 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jgaleotti commented Jan 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add support for advanced SQL features and improvements #1397

Add support for advanced SQL features and improvements #1397

Uh oh!

Conversation

jgaleotti commented Dec 1, 2025

Uh oh!

arcuri82 commented Dec 2, 2025

Uh oh!

jgaleotti commented Dec 2, 2025 via email

Uh oh!

jgaleotti commented Dec 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jgaleotti commented Jan 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants