Rework Query-DSL command. Add new Macro builtin command. Rework error handling. Updates Docs. Testcases

CHANGELOG.md

@@ -1,3 +1,13 @@
+# 1.0.4
+ADDED commando.commands.builtin/commando-macro-spec. The new type of command that allow to group instructions by its functionality and use it as a single command. Added Readme information about the macro.
+ADDED documentation for commando.commands.builtin commands. Now each built-in command have explanation of its behavior and examples of usage.
+UPDATED upgrade commando.commands.query-dsl. Function `resolve-query` was removed and replaced by `resolve-fn`, `resolve-instruction`, `resolve-instruction-qe` function called a **resolvers**. Explanations about the resolvers added to _docs/query-dsl.md_ file.
+UPDATED error serialization. `commando.impl.utils` contains new way to serialize errors for CLJ/CLJS. Now all errors are serialized to map with keys: `:type`, `:class`, `:message`, `:data` (if exists) and `:stacktrace` (if exists), `:cause` (if exists). See `commando.impl.utils/serialize-exception` for more information. You can expand the error handlers using `serialize-exception-fn` multimethod (but for CLJ only).
+ADDED tests for macro-spec, errors and query-dsl changes.
+UPDATED README.md 'Debugging section' was replaced on 'Configuring Execution Behavior' which contains more detailed information how to modify execution behavior.
+UPDATED dynamic variable *debug-mode* replaced by the `*execute-config*` which is a map that can contain multiple configuration options.
+FIXED Removed `detach-instruction-commands` call from `commando.core/build-compiler`. In `commando.core/build-compiler`, the line that detached instruction commands from the registry was removed. This means the compiled registry now includes internal commands (_map, _value, _vector).
+
 # 1.0.3
 UPDATED behavior `:validate-params-fn`. If the function return anything except `true` it ment validation failure. If the function return data, they will be attached to returned error inside status map. Added tests.
 FIXED align serialization of exeption for CLJ/CLJS

README.md

@@ -4,7 +4,7 @@
 
 [![Clojars Project](https://img.shields.io/clojars/v/org.clojars.funkcjonariusze/commando.svg)](https://clojars.org/org.clojars.funkcjonariusze/commando)
 [![Run tests](https://github.com/funkcjonariusze/commando/actions/workflows/unit_test.yml/badge.svg)](https://github.com/funkcjonariusze/commando/actions/workflows/unit_test.yml)
-[![cljdoc badge](https://cljdoc.org/badge/org.clojars.funkcjonariusze/commando)](https://cljdoc.org/d/org.clojars.funkcjonariusze/commando/1.0.3)
+[![cljdoc badge](https://cljdoc.org/badge/org.clojars.funkcjonariusze/commando)](https://cljdoc.org/d/org.clojars.funkcjonariusze/commando/1.0.4)
 
 **Commando** is a flexible Clojure library for managing, extracting, and transforming data inside nested map structures aimed to build your own Data DSL.
 
@@ -19,9 +19,12 @@
 	- [command-fn-spec](#command-fn-spec)
 	- [command-apply-spec](#command-apply-spec)
 	- [command-mutation-spec](#command-mutation-spec)
+	- [command-macro-spec](#command-macro-spec)
   - [Adding New Commands](#adding-new-commands)
 - [Status-Map and Internals](#status-map-and-internals)
-  - [Debugging commando](#debugging-commando)
+  - [Configuring Execution Behavior](#configuring-execution-behavior)
+	- [`:debug-result`](#debug-result)
+	- [`:error-data-string`](#error-data-string)
 - [Integrations](#integrations)
 - [Versioning](#versioning)
 - [License](#license)
@@ -30,10 +33,10 @@
 
 ```clojure
 ;; deps.edn with git
-{org.clojars.funkcjonariusze/commando {:mvn/version "1.0.3"}}
+{org.clojars.funkcjonariusze/commando {:mvn/version "1.0.4"}}
 
 ;; leiningen
-[org.clojars.funkcjonariusze/commando "1.0.3"]
+[org.clojars.funkcjonariusze/commando "1.0.4"]
 ```
 
 ## Quick Start
@@ -161,9 +164,9 @@ A convenient wrapper over `apply`.
    "v2" 2
    "sum="
    {:commando/fn +
-    :args [{:commando/from ["v1"]}
-           {:commando/from ["v2"]}
-	       3]}})
+	:args [{:commando/from ["v1"]}
+		   {:commando/from ["v2"]}
+		   3]}})
 ;; => {"v1" 1 "v2" 2 "sum=" 6}
 ```
 
@@ -175,13 +178,13 @@ A wrapper similar to `commando/fn`, but conceptually closer to `commando/from`,
 (commando/execute
   [commands-builtin/command-apply-spec]
   {"0" {:commando/apply
-	{"1" {:commando/apply
-	      {"2" {:commando/apply
-		    {"3" {:commando/apply {"4" {:final "5"}}
-			  := #(get % "4")}}
-		    := #(get % "3")}}
-	      := #(get % "2")}}
-	:= #(get % "1")}})
+		{"1" {:commando/apply
+			  {"2" {:commando/apply
+					{"3" {:commando/apply {"4" {:final "5"}}
+						  := #(get % "4")}}
+					:= #(get % "3")}}
+			  := #(get % "2")}}
+		:= #(get % "1")}})
 ;; => {"0" {:final "5"}}
 ```
 
@@ -194,19 +197,19 @@ Imagine the following instruction is your initial database migration, adding use
   [commands-builtin/command-from-spec
    commands-builtin/command-mutation-spec]
   {"add-new-user-01" {:commando/mutation :add-user :name "Bob Anderson"
-		              :permissions [{:commando/from ["perm_send_mail"] := :id}
-                      {:commando/from ["perm_recv_mail"] := :id }]}
+					  :permissions [{:commando/from ["perm_send_mail"] := :id}
+					  {:commando/from ["perm_recv_mail"] := :id }]}
    "add-new-user-02" {:commando/mutation :add-user :name "Damian Nowak"
-                      :permissions [{:commando/from ["perm_recv_mail"] := :id}]}
+					  :permissions [{:commando/from ["perm_recv_mail"] := :id}]}
    "perm_recv_mail" {:commando/mutation :add-permission
-                     :name "receive-email-notification"}
+					 :name "receive-email-notification"}
    "perm_send_mail" {:commando/mutation :add-permission
-	                 :name "send-email-notification"}})
+					 :name "send-email-notification"}})
 ```
 
 You can see that you need both :add-permission and :add-user commands. In most cases, such patterns can be abstracted and reused, simplifying your migrations and business logic.
 
-`commando-mutation-spec` uses `defmethod commando/command-mutation` underneath, making it easy to wrap business logic into commands and integrate them into your instructions/migrations:
+`commando-mutation-spec` uses `defmethod commando.commands.builtin/command-mutation` underneath, making it easy to wrap business logic into commands and integrate them into your instructions/migrations:
 
 ```clojure
 (defmethod commands-builtin/command-mutation :add-user [_ {:keys [name permissions]}]
@@ -224,8 +227,64 @@ You can see that you need both :add-permission and :add-user commands. In most c
 
 This approach enables you to quickly encapsulate business logic into reusable commands, which can then be easily composed in your instructions or migrations.
 
+#### command-macro-spec
 
-### Adding new commands 
+Allows describing reusable command templates that are expanded into regular Commando commands at runtime. This is useful when you want to describe a pattern for building a complex command or a set of related commands without duplicating the same structure throughout an instruction
+
+Asume we have a Instruction what calculates mean.
+```clojure
+(commando/execute
+  [commands-builtin/command-from-spec
+   commands-builtin/command-apply-spec
+   commands-builtin/command-fn-spec]
+  {:= :result
+   :commando/apply
+   {:vector-of-numbers [1, 2, 3, 4, 5]
+	:result
+	{:fn (fn [& [vector-of-numbers]]
+		   (/ (reduce + 0 vector-of-numbers)
+			 (count vector-of-numbers)))
+	 :args [{:commando/from [:commando/apply :vector-of-numbers]}]}}})
+;; => 3
+```
+
+This works, but the structure is not very easy to read when repeated. When you need the same mean calculation many times, the instruction quickly grows and becomes hard to follow. A macro can help by encapsulating the pattern into a readable reusable shortcut.
+
+Define a macro
+
+```clojure
+(defmethod commands-builtin/command-macro :mean-calc [{vector-of-numbers :vector-of-numbers}]
+  {:= :result
+   :commando/apply
+   {:vector-of-numbers vector-of-numbers
+	:result
+	{:fn (fn [& [vector-of-numbers]]
+		   (/ (reduce + 0 vector-of-numbers)
+			 (count vector-of-numbers)))
+	 :args [{:commando/from [:commando/apply :vector-of-numbers]}]}}})
+
+
+(commando/execute
+  [commands-builtin/command-macro-spec
+   commands-builtin/command-from-spec
+   commands-builtin/command-apply-spec
+   commands-builtin/command-fn-spec]
+  {:v1 {:commando/macro :mean-calc :vector-of-numbers [1, 2, 3, 4, 5]}
+   :v2 {:commando/macro :mean-calc :vector-of-numbers [10, 22, 33]}
+   :v3 {:commando/macro :mean-calc :vector-of-numbers [7, 8, 1000, 1]}})
+;; =>
+;; {:v1 3
+;;  :v2 21.666
+;;  :v3 254}
+```
+
+command-macro-spec detects entries with `:commando/macro` and calls the multimethod `(defmethod) commands-builtin/command-macro` using the macro identifier (e.g. `:mean-calc`) and the parameter map from the instruction.
+
+The defmethod should return a Instruction. Commando will then treat that returned map as a fully separate instruction: dependencies (like `:commando/from`) are discovered inside the macro hierarchy.
+
+Use these macro handlers to hide repeated command structure and keep your instructions shorter and easier to read.
+
+### Adding new commands
 
 As you start using commando, you will start writing your own command specs to match your data needs.
 
@@ -256,9 +315,9 @@ Let's create a new command using a CommandMapSpec configuration map:
 {:type :CALC=
  :recognize-fn #(and (map? %) (contains? % :CALC=))
  :validate-params-fn (fn [m]
-				       (and
-					     (fn? (:CALC= m))
-    					 (not-empty (:ARGS m))))
+					   (and
+						 (fn? (:CALC= m))
+						 (not-empty (:ARGS m))))
  :apply (fn [_instruction _command m]
 			 (apply (:CALC= m) (:ARGS m)))
  :dependencies {:mode :all-inside}}
@@ -270,27 +329,27 @@ Let's create a new command using a CommandMapSpec configuration map:
 - `:apply` - the function that directly executes the command as params it receives whole instruction, command spec and as a last argument what was recognized by :cm/recognize
 - `:dependencies` - describes the type of dependency this command has. Commando supports three modes:
   - `{:mode :all-inside}` - the command scans itself for dependencies on other commands within its body.
-  - `{:mode :none}` - the command has no dependencies and can be evaluated whenever. 
+  - `{:mode :none}` - the command has no dependencies and can be evaluated whenever.
   - `{:mode :point :point-key :commando/from}` - allowing to be dependent anywhere in the instructions. Expects point-key which tells where is the dependency (commando/from as an example uses this)
 
 Now you can use it for more expressive operations like "summ=" and "multiply=" as shown below:
 
 ```clojure
 (def command-registry
   (commando/create-registry
-    [;; Add `:commando/from`
-     commands-builtin/command-from-spec
-     ;; Add `:CALC=` command to be handled
-     ;; inside instruction
-     {:type :CALC=
-      :recognize-fn #(and (map? %) (contains? % :CALC=))
-      :validate-params-fn (fn [m]
-                            (and
-                              (fn? (:CALC= m))
-                              (not-empty (:ARGS m))))
-      :apply (fn [_instruction _command m]
-                (apply (:CALC= m) (:ARGS m)))
-      :dependencies {:mode :all-inside}}]))
+	[;; Add `:commando/from`
+	 commands-builtin/command-from-spec
+	 ;; Add `:CALC=` command to be handled
+	 ;; inside instruction
+	 {:type :CALC=
+	  :recognize-fn #(and (map? %) (contains? % :CALC=))
+	  :validate-params-fn (fn [m]
+							(and
+							  (fn? (:CALC= m))
+							  (not-empty (:ARGS m))))
+	  :apply (fn [_instruction _command m]
+				(apply (:CALC= m) (:ARGS m)))
+	  :dependencies {:mode :all-inside}}]))
 
 (commando/execute
   command-registry
@@ -322,7 +381,7 @@ The concept of a **command** is not limited to map structures it is basically an
   [{:type :custom/json
 	:recognize-fn #(and (string? %) (clojure.string/starts-with? % "json"))
 	:apply (fn [_instruction _command-map string-value]
-              (clojure.data.json/read-str (apply str (drop 4 string-value))
+			  (clojure.data.json/read-str (apply str (drop 4 string-value))
 				:key-fn keyword))
 	:dependencies {:mode :none}}]
   {:json-command-1 "json{\"some-json-value-1\": 123}"
@@ -413,23 +472,32 @@ On unsuccessful execution (`:failed`), you get:
   #<CommandMapPath "root,1[_value]">]}
 ```
 
-### Debugging commando
+### Configuring Execution Behavior
+
+The `commando.impl.utils/*execute-config*` dynamic variable allows for fine-grained control over `commando/execute`'s behavior. You can bind this variable to a map containing the following configuration keys:
+
+- `:debug-result` (boolean)
+- `:error-data-string` (boolean)
 
-You can set dynamic variable `commando.impl.utils/*debug-mode* true` to see more details on how execution went.
+#### `:debug-result`
+
+When set to `true`, the returned status-map will include additional execution information, such as `:internal/cm-list`, `:internal/cm-dependency`, and `:internal/cm-running-order`. This helps in analyzing the instruction's execution flow.
+
+Here's an example of how to use `:debug-result`:
 
 ```clojure
 (require '[commando.core :as commando])
 (require '[commando.commands.builtin :as commands-builtin])
 (require '[commando.impl.utils :as commando-utils])
 
-(binding [commando-utils/*debug-mode* true]
-  (execute
-    [commands-builtin/command-from-spec]
-    {"1" 1
-     "2" {:commando/from ["1"]}
-     "3" {:commando/from ["2"]}}))
-	 
-;; RETURN => 
+(binding [commando-utils/*execute-config* {:debug-result true}]
+  (commando/execute
+	[commands-builtin/command-from-spec]
+	{"1" 1
+	 "2" {:commando/from ["1"]}
+	 "3" {:commando/from ["2"]}}))
+
+;; RETURN =>
 {:status :ok,
  :instruction {"1" 1, "2" 1, "3" 1}
  :registry
@@ -460,12 +528,55 @@ You can set dynamic variable `commando.impl.utils/*debug-mode* true` to see more
   "root,3[from]"   #{"root,2[from]"}}}
 ```
 
-`:internal/cm-list` - a list of all recognized commands in an instruction. This list also contains the `_map`, `_value`, and the unmentioned `_vector` commands, which are not included in the registry. Commando includes several internal built-in commands that describe the _instruction's structure_. An _instruction_ is a composition of maps, their values, and vectors that represent its structure and help build a clear dependency graph. These commands are removed from the final output after this step.
+`:internal/cm-list` - a list of all recognized commands in an instruction. This list also contains the `_map`, `_value`, and the unmentioned `_vector` commands. Commando includes several internal built-in commands that describe the _instruction's structure_. An _instruction_ is a composition of maps, their values, and vectors that represent its structure and help build a clear dependency graph. These commands are removed from the final output after this step, but included in the compiled registry.
 
 `:internal/cm-dependency` - describes how parts of an _instruction_ depend on each other.
 
 `:internal/cm-running-order` - the correct order in which to execute commands.
 
+
+#### `:error-data-string`
+
+When `:error-data-string` is `true`, the `:data` key within serialized `ExceptionInfo` objects (processed by `commando.impl.utils/serialize-exception`) will contain a string representation of the exception's data. Conversely, if `false`, the `:data` key will hold the raw data structure (map). This setting is particularly useful for controlling the verbosity of error details, in example when examining Malli validation explanations etc.
+
+```clojure
+(def value
+  (commando/execute [commands-builtin/command-from-spec]
+    {"a" 10
+     "ref" {:commando/from "BROKEN"}}))
+(get-in value [:errors 0 :error])
+;; =>
+;; {:type "exception-info",
+;;  :class "clojure.lang.ExceptionInfo",
+;;  :message "Failed while validating params for :commando/from ...",
+;;  :stack-trace
+;;  [["commando.impl.finding_commands$instruction_command_spec$fn__14401" "invoke" "finding_commands.cljc" 65]
+;;   ["clojure.core$some" "invokeStatic" "core.clj" 2718]
+;;   ...
+;;   ...],
+;;  :cause nil,
+;;  :data "{:command-type :commando/from, :reason #:commando{:from [\"commando/from should be a sequence path to value in Instruction: [:some 2 \\\"value\\\"]\"]}, :path [\"ref\"], :value #:commando{:from \"BROKEN\"}}"}
+
+
+(def value
+  (binding [sut/*execute-config* {:error-data-string false}]
+    (commando/execute [commands-builtin/command-from-spec]
+      {"a" 10
+       "ref" {:commando/from "BROKEN"}})))
+(get-in value [:errors 0 :error])
+;; =>
+;; {:type "exception-info",
+;;  :class "clojure.lang.ExceptionInfo",
+;;  ...
+;;  ...
+;;  :data
+;;  {:command-type :commando/from,
+;;   :reason {:commando/from
+;;            ["commando/from should be a sequence path to value in Instruction: [:some 2 \"value\"]"]},
+;;   :path ["ref"],
+;;   :value {:commando/from "BROKEN"}}}
+```
+
 # Integrations
 
 - [Work with JSON](./doc/json.md)
@@ -474,7 +585,7 @@ You can set dynamic variable `commando.impl.utils/*debug-mode* true` to see more
 - [Commando QueryDSL](./doc/query_dsl.md)
 - [Example Http commando transit handler + Reitit](./doc/example_reitit.clj)
 
-# Versioning 
+# Versioning
 
 We comply with: [Break Versioning](https://www.taoensso.com/break-versioning)
 `<major>.<minor>.<non-breaking>[-<optional-qualifier>]`