Introduce at high level which other params such as , etc.. follow it. It validates the shape to send a valid query type to the ES.

Author: mashhurs

docs/index.asciidoc

@@ -231,23 +231,23 @@ The next scheduled run:
 * updates the value of the field at the end of the pagination.
 
 [id="plugins-{type}s-{plugin}-esql"]
-==== ES|QL support
-{es} Query Language (ES|QL) provides a SQL-like interface for querying your {es} data.
+==== {esql} support
+{es} Query Language ({esql}) provides a SQL-like interface for querying your {es} data.
 
 To use {esql}, this plugin needs to be installed in {ls} 8.17.4 or newer, and must be connected to {es} 8.11 or newer.
 
-To configure {esql} query in the plugin, set the `response_type` to `esql` and provide your {esql} query in the `query` parameter.
+To configure {esql} query in the plugin, set the `query_type` to `esql` and provide your {esql} query in the `query` parameter.
 
 IMPORTANT: {esql} is evolving and may still have limitations with regard to result size or supported field types. We recommend understanding https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-limitations.html[ES|QL current limitations] before using it in production environments.
 
-The following is a basic scheduled ES|QL query that runs hourly:
+The following is a basic scheduled {esql} query that runs hourly:
 [source, ruby]
     input {
       elasticsearch {
         id => hourly_cron_job
         hosts => [ 'https://..']
         api_key => '....'
-        response_type => 'esql'
+        query_type => 'esql'
         query => '
           FROM food-index
             | WHERE spicy_level = "hot" AND @timestamp > NOW() - 1 hour
@@ -259,11 +259,11 @@ The following is a basic scheduled ES|QL query that runs hourly:
 
 Set `config.support_escapes: true` in `logstash.yml` if you need to escape special chars in the query.
 
-NOTE: With ES|QL query, {ls} doesn't generate `event.original`.
+NOTE: With {esql} query, {ls} doesn't generate `event.original`.
 
 [id="plugins-{type}s-{plugin}-esql-event-mapping"]
-===== Mapping ES|QL result to {ls} event
-ES|QL returns query results in a structured tabular format, where data is organized into _columns_ (fields) and _values_ (entries).
+===== Mapping {esql} result to {ls} event
+{esql} returns query results in a structured tabular format, where data is organized into _columns_ (fields) and _values_ (entries).
 The plugin maps each value entry to an event, populating corresponding fields.
 For example, a query might produce a table like:
 
@@ -303,9 +303,9 @@ NOTE: If your index has a mapping with sub-objects where `status.code` and `stat
 [id="plugins-{type}s-{plugin}-esql-multifields"]
 ===== Conflict on multi-fields
 
-ES|QL query fetches all parent and sub-fields fields if your {es} index has https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/multi-fields[multi-fields] or https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/subobjects[subobjects].
+{esql} query fetches all parent and sub-fields fields if your {es} index has https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/multi-fields[multi-fields] or https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/subobjects[subobjects].
 Since {ls} events cannot contain parent field's concrete value and sub-field values together, the plugin ignores sub-fields with warning and includes parent.
-We recommend using the `RENAME` (or `DROP` to avoid warnings) keyword in your ES|QL query explicitly rename the fields to include sub-fields into the event.
+We recommend using the `RENAME` (or `DROP` to avoid warnings) keyword in your {esql} query explicitly rename the fields to include sub-fields into the event.
 
 This a common occurrence if your template or mapping follows the pattern of always indexing strings as "text" (`field`) + " keyword" (`field.keyword`) multi-field.
 In this case it's recommended to do `KEEP field` if the string is identical and there is only one subfield as the engine will optimize and retrieve the keyword, otherwise you can do `KEEP field.keyword | RENAME field.keyword as field`.
@@ -318,14 +318,14 @@ To illustrate the situation with example, assuming your mapping has a time `time
         "time.max": { "type": "long" }
     }
 
-The ES|QL result will contain all three fields but the plugin cannot map them into {ls} event.
+The {esql} result will contain all three fields but the plugin cannot map them into {ls} event.
 To avoid this, you can use the `RENAME` keyword to rename the `time` parent field to get all three fields with unique fields.
 [source, ruby]
     ...
     query => 'FROM my-index | RENAME time AS time.current'
     ...
 
-For comprehensive ES|QL syntax reference and best practices, see the https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-syntax.html[{es} ES|QL documentation].
+For comprehensive {esql} syntax reference and best practices, see the https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-syntax.html[{esql} documentation].
 
 [id="plugins-{type}s-{plugin}-options"]
 ==== Elasticsearch Input configuration options
@@ -354,7 +354,8 @@ Please check out <<plugins-{type}s-{plugin}-obsolete-options>> for details.
 | <<plugins-{type}s-{plugin}-password>> |<<password,password>>|No
 | <<plugins-{type}s-{plugin}-proxy>> |<<uri,uri>>|No
 | <<plugins-{type}s-{plugin}-query>> |<<string,string>>|No
-| <<plugins-{type}s-{plugin}-response_type>> |<<string,string>>, one of `["hits","aggregations","esql"]`|No
+| <<plugins-{type}s-{plugin}-query_type>> |<<string,string>>, one of `["dsl","esql"]`|No
+| <<plugins-{type}s-{plugin}-response_type>> |<<string,string>>, one of `["hits","aggregations"]`|No
 | <<plugins-{type}s-{plugin}-request_timeout_seconds>> | <<number,number>>|No
 | <<plugins-{type}s-{plugin}-schedule>> |<<string,string>>|No
 | <<plugins-{type}s-{plugin}-schedule_overlap>> |<<boolean,boolean>>|No
@@ -596,12 +597,22 @@ environment variables e.g. `proxy => '${LS_PROXY:}'`.
   * Default value is `'{ "sort": [ "_doc" ] }'`
 
 The query to be executed.
-Accepted query shape is DSL or ES|QL (when `response_type => 'esql'`).
-Read the {ref}/query-dsl.html[{es} query DSL documentation] or {ref}/esql.html[{es} ES|QL documentation] for more information.
+Accepted query shape is DSL or {esql} (when `query_type => 'esql'`).
+Read the {ref}/query-dsl.html[{es} query DSL documentation] or {ref}/esql.html[{esql} documentation] for more information.
 
 When <<plugins-{type}s-{plugin}-search_api>> resolves to `search_after` and the query does not specify `sort`,
 the default sort `'{ "sort": { "_shard_doc": "asc" } }'` will be added to the query. Please refer to the {ref}/paginate-search-results.html#search-after[Elasticsearch search_after] parameter to know more.
 
+[id="plugins-{type}s-{plugin}-query_type"]
+===== `query_type`
+
+* Value can be `dsl` or `esql`
+* Default value is `dsl`
+
+Defines the <<plugins-{type}s-{plugin}-query>> shape.
+When `dsl`, the query shape must be valid {es} JSON-style string.
+When `esql`, the query shape must be a valid {esql} string and `index`, `size`, `slices`, `search_api`, `docinfo`, `docinfo_target`, `docinfo_fields`, `response_type` and `tracking_field` parameters are not allowed.
+
 [id="plugins-{type}s-{plugin}-response_type"]
 ===== `response_type`
 
@@ -613,14 +624,11 @@ response from the query.
 
 The default `hits` will generate one event per returned document (i.e. "hit").
 
-When set to `aggregations`, a single Logstash event will be generated with the
+When set to `aggregations`, a single {ls} event will be generated with the
 contents of the `aggregations` object of the query's response. In this case the
 `hits` object will be ignored. The parameter `size` will be always be set to
 0 regardless of the default or user-defined value set in this plugin.
 
-When using the `esql` setting, the query must be a valid ES|QL string.
-When this setting is active, `index`, `size`, `slices`, `search_api`, `docinfo`, `docinfo_target` and `docinfo_fields` parameters are not allowed.
-
 [id="plugins-{type}s-{plugin}-request_timeout_seconds"]
 ===== `request_timeout_seconds`
 

lib/logstash/inputs/elasticsearch.rb

@@ -97,18 +97,21 @@ class LogStash::Inputs::Elasticsearch < LogStash::Inputs::Base
   # The index or alias to search.
   config :index, :validate => :string, :default => "logstash-*"
 
-  # The query to be executed. DSL or ES|QL (when `response_type => 'esql'`) query shape is accepted.
+  # A type of Elasticsearch query, provided by @query. This will validate query shape and other params.
+  config :query_type, :validate => %w[dsl esql], :default => 'dsl'
+
+  # The query to be executed. DSL or ES|QL (when `query_type => 'esql'`) query shape is accepted.
   # Read the following documentations for more info
   # Query DSL: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html
   # ES|QL: https://www.elastic.co/guide/en/elasticsearch/reference/current/esql.html
   config :query, :validate => :string, :default => '{ "sort": [ "_doc" ] }'
 
-  # This allows you to speccify the response type: one of [hits, aggregations, esql]
+  # This allows you to specify the DSL response type: one of [hits, aggregations]
   # where
   #   hits: normal search request
   #   aggregations: aggregation request
-  #   esql: ES|QL request
-  config :response_type, :validate => %w[hits aggregations esql], :default => 'hits'
+  # Note that this param is invalid when `query_type => 'esql'`, ES|QL response shape is always a tabular format
+  config :response_type, :validate => %w[hits aggregations], :default => 'hits'
 
   # This allows you to set the maximum number of hits returned per scroll.
   config :size, :validate => :number, :default => 1000
@@ -309,10 +312,10 @@ def register
     fill_hosts_from_cloud_id
     setup_ssl_params!
 
-    if @response_type == 'esql'
+    if @query_type == 'esql'
       validate_ls_version_for_esql_support!
       validate_esql_query!
-      not_allowed_options = original_params.keys & %w(index size slices search_api, docinfo, docinfo_target, docinfo_fields)
+      not_allowed_options = original_params.keys & %w(index size slices search_api docinfo docinfo_target docinfo_fields response_type tracking_field)
       raise(LogStash::ConfigurationError, "Configured #{not_allowed_options} params are not allowed while using ES|QL query") if not_allowed_options&.size > 1
     else
       @base_query = LogStash::Json.load(@query)
@@ -361,7 +364,7 @@ def register
 
     setup_search_api
 
-    setup_query_executor
+    @query_executor = create_query_executor
 
     setup_cursor_tracker
 
@@ -396,7 +399,7 @@ def decorate_event(event)
   private
 
   def get_query_object
-    return @query if @response_type == 'esql'
+    return @query if @query_type == 'esql'
     if @cursor_tracker
       query = @cursor_tracker.inject_cursor(@query)
       @logger.debug("new query is #{query}")
@@ -685,20 +688,16 @@ def setup_search_api
 
   end
 
-  def setup_query_executor
-    @query_executor = case @response_type
-                      when 'hits'
-                        if @resolved_search_api == "search_after"
-                          LogStash::Inputs::Elasticsearch::SearchAfter.new(@client, self)
-                        else
-                          logger.warn("scroll API is no longer recommended for pagination. Consider using search_after instead.") if es_major_version >= 8
-                          LogStash::Inputs::Elasticsearch::Scroll.new(@client, self)
-                        end
-                      when 'aggregations'
-                        LogStash::Inputs::Elasticsearch::Aggregation.new(@client, self)
-                      when 'esql'
-                        LogStash::Inputs::Elasticsearch::Esql.new(@client, self)
-                      end
+  def create_query_executor
+    return LogStash::Inputs::Elasticsearch::Esql.new(@client, self) if @query_type == 'esql'
+
+    # DSL query executor
+    return LogStash::Inputs::Elasticsearch::Aggregation.new(@client, self) if @response_type == 'aggregations'
+    # response_type is hits, executor can be search_after or scroll type
+    return LogStash::Inputs::Elasticsearch::SearchAfter.new(@client, self) if @resolved_search_api == "search_after"
+
+    logger.warn("scroll API is no longer recommended for pagination. Consider using search_after instead.") if es_major_version >= 8
+    LogStash::Inputs::Elasticsearch::Scroll.new(@client, self)
   end
 
   def setup_cursor_tracker
@@ -751,7 +750,7 @@ def validate_esql_query!
   end
 
   def validate_es_for_esql_support!
-    return unless @response_type == 'esql'
+    return unless @query_type == 'esql'
     # make sure connected ES supports ES|QL (8.11+)
     es_supports_esql = Gem::Version.create(es_version) >= Gem::Version.create(ES_ESQL_SUPPORT_VERSION)
     fail("Connected Elasticsearch #{es_version} version does not supports ES|QL. ES|QL feature requires at least Elasticsearch #{ES_ESQL_SUPPORT_VERSION} version.") unless es_supports_esql

spec/inputs/elasticsearch_spec.rb

@@ -1374,7 +1374,7 @@ def extract_transport(client) # on 7.x client.transport is a ES::Transport::Clie
     let(:config) do
       {
         "query" => "FROM test-index | STATS count() BY field",
-        "response_type" => "esql",
+        "query_type" => "esql",
         "retries" => 3
       }
     end
@@ -1387,8 +1387,8 @@ def extract_transport(client) # on 7.x client.transport is a ES::Transport::Clie
 
     describe "#initialize" do
       it "sets up the ESQL client with correct parameters" do
+        expect(plugin.instance_variable_get(:@query_type)).to eq(config["query_type"])
         expect(plugin.instance_variable_get(:@query)).to eq(config["query"])
-        expect(plugin.instance_variable_get(:@response_type)).to eq(config["response_type"])
         expect(plugin.instance_variable_get(:@retries)).to eq(config["retries"])
       end
     end
@@ -1449,10 +1449,12 @@ def extract_transport(client) # on 7.x client.transport is a ES::Transport::Clie
             "docinfo" => true,
             "docinfo_target" => "[@metadata][docinfo]",
             "docinfo_fields" => ["_index"],
+            "response_type" => "hits",
+            "tracking_field" => "[@metadata][tracking]"
           })}
 
         it "raises a config error" do
-          mixed_fields = %w[index size slices docinfo_fields]
+          mixed_fields = %w[index size slices docinfo_fields response_type tracking_field]
           expect { plugin.register }.to raise_error(LogStash::ConfigurationError, /Configured #{mixed_fields} params are not allowed while using ES|QL query/)
         end
       end

spec/inputs/integration/elasticsearch_esql_spec.rb

@@ -23,7 +23,7 @@
   let(:config) do
     {
       "hosts" => ES_HOSTS,
-      "response_type" => "esql"
+      "query_type" => "esql"
     }
   end
   let(:es_client) do