diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..24a8e87 --- /dev/null +++ b/.gitattributes @@ -0,0 +1 @@ +*.png filter=lfs diff=lfs merge=lfs -text diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f80d0b..206aa70 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,17 +1,45 @@ +# 1.0.5 +ADDED new keys to `commando.impl.utils/*execute-config*`. Added hooks keys + - `:hook-execute-start` if not nil, call procedure at the start of `commando.core/execute` function. + - `:hook-execute-end` if not nil, call procedure at the end of `commando.core/execute` function. + +ADDED microsecs time measurement. All steps inside the `commando.core/execute` measure time. Every measurement adding to _status-map_ structure under `:stats` key. + +UPDATED `status-map` structure. Was added two keys + - `:uuid` autogenerated unique invocation identifier gotted for each `commando.core/execute` call + - `:stats` contains vector of tuples like `["execute", 1085471, "1.085471ms"]` where `[, , ]`. Counts of steps depended from `*execute-config*` key `:debug-mode`. + +UPDATED sort-commands-by-deps. Straightforward sets joining in base Kahn's algorithm was rewrited with in-degree counting optimization. + +UPDATED build-deps-tree. Instead of searching dependency using the list of commandmaps by iterating across the list(O(n^2) in worst case), before starting to build a dependency graph, we quickly building a path-trie structure efficiently. This gave as fast way to resolve point/all-inside dependency only in O(n) time. + +FIXED find-commands. StackOverflowException in case of long lists of dependencies in the one level. + + # 1.0.4 ADDED commando.commands.builtin/commando-macro-spec. The new type of command that allow to group instructions by its functionality and use it as a single command. Added Readme information about the macro. + ADDED documentation for commando.commands.builtin commands. Now each built-in command have explanation of its behavior and examples of usage. + UPDATED upgrade commando.commands.query-dsl. Function `resolve-query` was removed and replaced by `resolve-fn`, `resolve-instruction`, `resolve-instruction-qe` function called a **resolvers**. Explanations about the resolvers added to _docs/query-dsl.md_ file. + UPDATED error serialization. `commando.impl.utils` contains new way to serialize errors for CLJ/CLJS. Now all errors are serialized to map with keys: `:type`, `:class`, `:message`, `:data` (if exists) and `:stacktrace` (if exists), `:cause` (if exists). See `commando.impl.utils/serialize-exception` for more information. You can expand the error handlers using `serialize-exception-fn` multimethod (but for CLJ only). + ADDED tests for macro-spec, errors and query-dsl changes. + UPDATED README.md 'Debugging section' was replaced on 'Configuring Execution Behavior' which contains more detailed information how to modify execution behavior. + UPDATED dynamic variable *debug-mode* replaced by the `*execute-config*` which is a map that can contain multiple configuration options. + FIXED Removed `detach-instruction-commands` call from `commando.core/build-compiler`. In `commando.core/build-compiler`, the line that detached instruction commands from the registry was removed. This means the compiled registry now includes internal commands (_map, _value, _vector). # 1.0.3 UPDATED behavior `:validate-params-fn`. If the function return anything except `true` it ment validation failure. If the function return data, they will be attached to returned error inside status map. Added tests. + FIXED align serialization of exeption for CLJ/CLJS + ADDED function normalization for :commando/fn, :commando/apply, :commando/from commands. In CLJ it will acept the symbols,vars,functions,keywords. In CLJS acceptable is only function and keywords. + FIXED QueryDSL. QueryExpression passing by :keys and :strs(for string Instruction keys) # 1.0.2 diff --git a/README.md b/README.md index e6727cd..a030f49 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ - [Configuring Execution Behavior](#configuring-execution-behavior) - [`:debug-result`](#debug-result) - [`:error-data-string`](#error-data-string) + - [Performance](#performance) - [Integrations](#integrations) - [Versioning](#versioning) - [License](#license) @@ -33,10 +34,10 @@ ```clojure ;; deps.edn with git -{org.clojars.funkcjonariusze/commando {:mvn/version "1.0.4"}} +{org.clojars.funkcjonariusze/commando {:mvn/version "1.0.5"}} ;; leiningen -[org.clojars.funkcjonariusze/commando "1.0.4"] +[org.clojars.funkcjonariusze/commando "1.0.5"] ``` ## Quick Start @@ -413,6 +414,9 @@ On successful execution (`:ok`), you get: ;; RETURN => {:status :ok, :instruction {"1" 1, "2" 1, "3" 1} + :stats + [["execute-commands!" 95838 "95.838µs"] + ["execute" 1085471 "1.085471ms"]] :successes [{:message "Commando. parse-instruction-map. Entities was successfully collected"} @@ -500,6 +504,13 @@ Here's an example of how to use `:debug-result`: ;; RETURN => {:status :ok, :instruction {"1" 1, "2" 1, "3" 1} + :stats + [["use-registry" 111876 "111.876µs"] + ["find-commands" 303062 "303.062µs"] + ["build-deps-tree" 134049 "134.049µs"] + ["sort-commands-by-deps" 292206 "292.206µs"] + ["execute-commands!" 53762 "53.762µs"] + ["execute" 1074110 "1.07411ms"]] :registry [{:type :commando/from, :recognize-fn #function[commando.commands.builtin/fn], @@ -577,6 +588,48 @@ When `:error-data-string` is `true`, the `:data` key within serialized `Exceptio ;; :value {:commando/from "BROKEN"}}} ``` + +### Performance + +Commando is designed for high performance, using efficient algorithms for dependency resolution and command execution to process complex instructions swiftly. + +All benchmarks were conducted on an **Intel Core i9-13980HX**. The primary metric for performance is the number of dependencies within an instruction. + +#### Total Execution Time (Typical Workloads) + +The graph below illustrates the total execution time for instructions with a typical number of dependencies, ranging from 1,250 to 80,000. As you can see, the execution time scales linearly and remains in the low millisecond range, demonstrating excellent performance for common use cases. + +
+ +
+ +#### Execution Step Analysis + +To provide deeper insight, we've broken down the execution into five distinct steps: +1. **use-registry**: Builds the command registry from the provided specs. +2. **find-commands**: Scans the instruction map to identify all command instances. +3. **build-deps-tree**: Constructs a Directed Acyclic Graph (DAG) of dependencies between commands. +4. **sort-commands-by-deps**: Sorts the commands based on the dependency graph to determine the correct execution order. +5. **execute-commands!**: Executes the commands in the resolved order. + +The following graphs show the performance of each step under both normal and extreme load conditions. + +**Normal Workloads (up to 80,000 dependencies)** + +Under normal conditions, each execution step completes in just a few milliseconds. The overhead of parsing, dependency resolution, and execution is minimal, ensuring a fast and responsive system. + +
+ +
+ +**Massive Workloads (up to 5,000,000 dependencies)** + +To test the limits of the library, we benchmarked it with instructions containing up to 5 million dependencies. The graph below shows that while the system scales, the `find-commands` (parsing) and `build-deps-tree` (dependency graph construction) phases become the primary bottlenecks. This demonstrates that the core execution remains fast, but performance at extreme scales is dominated by the initial analysis steps. + +
+ +
+ # Integrations - [Work with JSON](./doc/json.md) diff --git a/deps.edn b/deps.edn index 7f0fa80..dd4b66b 100644 --- a/deps.edn +++ b/deps.edn @@ -9,8 +9,10 @@ :main-opts ["-m" "cljs-test-runner.main" "-d" "test/unit"] :patterns [".*-test$"]} ;;For local performance regression tests - :dev {:extra-deps {criterium/criterium {:mvn/version "0.4.6"}} - :extra-paths ["test/perf"]} + :performance + {:extra-deps {cljfreechart/cljfreechart {:mvn/version "0.2.0"}} + :main-opts ["-m" "commando.core-perf-test"] + :extra-paths ["test/perf"]} ;;Build jar :build {:deps {io.github.clojure/tools.build {:mvn/version "0.10.10"}} :ns-default build diff --git a/logo/Commando(bottom outlined).png b/logo/Commando(bottom outlined).png index 9b830c2..31bb05e 100644 Binary files a/logo/Commando(bottom outlined).png and b/logo/Commando(bottom outlined).png differ diff --git a/logo/Commando(bottom).png b/logo/Commando(bottom).png index 9d0cb48..e2a476e 100644 Binary files a/logo/Commando(bottom).png and b/logo/Commando(bottom).png differ diff --git a/logo/Commando(outlined).png b/logo/Commando(outlined).png index c34b2ea..4ab5e98 100644 Binary files a/logo/Commando(outlined).png and b/logo/Commando(outlined).png differ diff --git a/logo/Commando(top outlined).png b/logo/Commando(top outlined).png index e505d48..285f386 100644 Binary files a/logo/Commando(top outlined).png and b/logo/Commando(top outlined).png differ diff --git a/logo/Commando.png b/logo/Commando.png index e28aee2..52c47fc 100644 Binary files a/logo/Commando.png and b/logo/Commando.png differ diff --git a/pom.xml b/pom.xml index f4acc17..307c683 100644 --- a/pom.xml +++ b/pom.xml @@ -4,7 +4,7 @@ jar org.clojars.funkcjonariusze commando - 1.0.4 + 1.0.5 commando @@ -42,6 +42,6 @@ https://github.com/funkcjonariusze/commando scm:git:git://github.com/funkcjonariusze/commando.git scm:git:ssh://git@github.com:funkcjonariusze/commando.git - 1.0.4 + 1.0.5 diff --git a/src/commando/commands/builtin.cljc b/src/commando/commands/builtin.cljc index 9279ab6..b7296c8 100644 --- a/src/commando/commands/builtin.cljc +++ b/src/commando/commands/builtin.cljc @@ -438,7 +438,7 @@ `commando.commands.builtin/command-macro` `commando.commands.builtin/command-macro-spec`"} command-macro-json-spec - {:type :commando/macro + {:type :commando/macro-json :recognize-fn #(and (map? %) (contains? % "commando-macro")) diff --git a/src/commando/core.cljc b/src/commando/core.cljc index 77e3351..8abf93f 100644 --- a/src/commando/core.cljc +++ b/src/commando/core.cljc @@ -48,17 +48,19 @@ (defn ^:private use-registry [status-map registry] - (case (:status status-map) - :failed (-> status-map - (smap/status-map-handle-warning {:message "Skip step with registry check"})) - :ok (try (-> status-map - (assoc :registry (if (registry/built? registry) registry (create-registry registry)))) - (catch #?(:clj Exception - :cljs :default) - e - (-> status-map - (smap/status-map-handle-error {:message "Invalid registry specification" - :error (utils/serialize-exception e)})))))) + (let [start-time (utils/now) + result (case (:status status-map) + :failed (-> status-map + (smap/status-map-handle-warning {:message "Skip step with registry check"})) + :ok (try (-> status-map + (assoc :registry (if (registry/built? registry) registry (create-registry registry)))) + (catch #?(:clj Exception + :cljs :default) + e + (-> status-map + (smap/status-map-handle-error {:message "Invalid registry specification" + :error (utils/serialize-exception e)})))))] + (smap/status-map-add-measurement result "use-registry" start-time (utils/now)))) (defn ^:private find-commands "Searches the instruction map for commands defined in the registry. @@ -70,20 +72,22 @@ - Commands are validated using their :validate-params-fn if present" [{:keys [instruction registry] :as status-map}] - (case (:status status-map) - :failed (-> status-map - (smap/status-map-handle-warning {:message "Skipping search for commands due to :failed status"})) - :ok (try (-> status-map - (assoc :internal/cm-list (finding-commands/find-commands instruction registry)) - (smap/status-map-handle-success {:message "Commands were successfully collected"})) - (catch #?(:clj Exception - :cljs :default) - e - (-> status-map - (smap/status-map-handle-error {:message - "Failed during commands search in instruction. See error for details." - :error (utils/serialize-exception e)})))) - (smap/status-map-undefined-status status-map))) + (let [start-time (utils/now) + result (case (:status status-map) + :failed (-> status-map + (smap/status-map-handle-warning {:message "Skipping search for commands due to :failed status"})) + :ok (try (-> status-map + (assoc :internal/cm-list (finding-commands/find-commands instruction registry)) + (smap/status-map-handle-success {:message "Commands were successfully collected"})) + (catch #?(:clj Exception + :cljs :default) + e + (-> status-map + (smap/status-map-handle-error {:message + "Failed during commands search in instruction. See error for details." + :error (utils/serialize-exception e)})))) + (smap/status-map-undefined-status status-map))] + (smap/status-map-add-measurement result "find-commands" start-time (utils/now)))) (defn ^:private build-deps-tree "Builds a dependency tree by resolving ':commando/from' references in commands. @@ -91,37 +95,42 @@ [{:keys [instruction] :internal/keys [cm-list] :as status-map}] - (case (:status status-map) - :failed (-> status-map - (smap/status-map-handle-warning {:message "Skipping dependency resolution due to :failed status"})) - :ok (try (-> status-map - (assoc :internal/cm-dependency (deps/build-dependency-graph instruction cm-list)) - (smap/status-map-handle-success {:message "Dependency map was successfully built"})) - (catch #?(:clj clojure.lang.ExceptionInfo - :cljs :default) - e - (-> status-map - (smap/status-map-handle-error (ex-data e))))) - (smap/status-map-undefined-status status-map))) + (let [start-time (utils/now) + result (case (:status status-map) + :failed (-> status-map + (smap/status-map-handle-warning {:message "Skipping dependency resolution due to :failed status"})) + :ok (try (-> status-map + (assoc :internal/cm-dependency (deps/build-dependency-graph instruction cm-list)) + (smap/status-map-handle-success {:message "Dependency map was successfully built"})) + (catch #?(:clj clojure.lang.ExceptionInfo + :cljs :default) + e + (-> status-map + (smap/status-map-handle-error (ex-data e))))) + (smap/status-map-undefined-status status-map))] + (smap/status-map-add-measurement result "build-deps-tree" start-time (utils/now)))) (defn ^:private sort-commands-by-deps [status-map] - (case (:status status-map) - :failed (-> status-map - (smap/status-map-handle-warning {:message (str utils/exception-message-header - "sort-entities-by-deps. Skipping mandatory step")})) - :ok (let [sort-result (graph/topological-sort (:internal/cm-dependency status-map)) - status-map (assoc status-map :internal/cm-running-order (vec (reverse (:sorted sort-result))))] - (if (not-empty (:cyclic sort-result)) - (smap/status-map-handle-error status-map - {:message (str utils/exception-message-header - "sort-entities-by-deps. Detected cyclic dependency") - :cyclic (:cyclic sort-result)}) - (smap/status-map-handle-success - status-map - {:message (str utils/exception-message-header - "sort-entities-by-deps. Entities was sorted and prepare for evaluating")}))) - (smap/status-map-undefined-status status-map))) + (let [start-time (utils/now) + result (case (:status status-map) + :failed (-> status-map + (smap/status-map-handle-warning {:message (str utils/exception-message-header + "sort-entities-by-deps. Skipping mandatory step")})) + :ok (let [sort-result (graph/topological-sort (:internal/cm-dependency status-map)) + status-map (assoc status-map :internal/cm-running-order (vec ;; (reverse (:sorted sort-result)) + (:sorted sort-result)))] + (if (not-empty (:cyclic sort-result)) + (smap/status-map-handle-error status-map + {:message (str utils/exception-message-header + "sort-entities-by-deps. Detected cyclic dependency") + :cyclic (:cyclic sort-result)}) + (smap/status-map-handle-success + status-map + {:message (str utils/exception-message-header + "sort-entities-by-deps. Entities was sorted and prepare for evaluating")}))) + (smap/status-map-undefined-status status-map))] + (smap/status-map-add-measurement result "sort-commands-by-deps" start-time (utils/now)))) (defn ^:private execute-commands! "Execute commands based on `:internal/cm-running-order`, transforming the instruction map. @@ -130,37 +139,46 @@ [{:keys [instruction registry] :internal/keys [cm-running-order] :as status-map}] - (binding [utils/*command-map-spec-registry* registry] - (cond - (smap/failed? status-map) - (smap/status-map-handle-warning status-map {:message "Skipping command evaluation due to failed status"}) - (empty? cm-running-order) (smap/status-map-handle-success status-map {:message "No commands to execute"}) - :else (let [[updated-instruction error-info] (executing/execute-commands instruction cm-running-order)] - (if error-info - (-> status-map - (assoc :instruction updated-instruction) - (smap/status-map-handle-error {:message "Command execution failed during evaluation" - :error (utils/serialize-exception (:original-error error-info)) - :command-path (:command-path error-info) - :command-type (:command-type error-info)})) - (-> status-map - (assoc :instruction updated-instruction) - (smap/status-map-handle-success {:message "All commands executed successfully"}))))))) + (let [start-time (utils/now) + result (binding [utils/*command-map-spec-registry* registry] + (cond + (smap/failed? status-map) + (smap/status-map-handle-warning status-map {:message "Skipping command evaluation due to failed status"}) + (empty? cm-running-order) (smap/status-map-handle-success status-map {:message "No commands to execute"}) + :else (let [[updated-instruction error-info] (executing/execute-commands instruction cm-running-order)] + (if error-info + (-> status-map + (assoc :instruction updated-instruction) + (smap/status-map-handle-error {:message "Command execution failed during evaluation" + :error (utils/serialize-exception (:original-error error-info)) + :command-path (:command-path error-info) + :command-type (:command-type error-info)})) + (-> status-map + (assoc :instruction updated-instruction) + (smap/status-map-handle-success {:message "All commands executed successfully"}))))))] + (smap/status-map-add-measurement result "execute-commands!" start-time (utils/now)))) (defn build-compiler [registry instruction] (let [status-map (-> (smap/status-map-pure {:instruction instruction}) - (use-registry registry) - (find-commands) - (build-deps-tree) - (sort-commands-by-deps))] + (utils/hook-process (:hook-execute-start (utils/execute-config))) + (use-registry registry) + (find-commands) + (build-deps-tree) + (sort-commands-by-deps))] (case (:status status-map) :failed (-> status-map - (smap/status-map-handle-warning {:message (str utils/exception-message-header - "build-compiler. Error building compiler")})) + (smap/status-map-handle-warning {:message (str utils/exception-message-header + "build-compiler. Error building compiler")})) :ok (cond-> status-map true (update-in [:internal/cm-running-order] registry/remove-instruction-commands-from-command-vector) - (false? (:debug-result (utils/execute-config))) (select-keys [:status :registry :internal/cm-running-order :successes :warnings]))))) + (false? (:debug-result (utils/execute-config))) + (select-keys [:uuid + :status + :registry + :internal/cm-running-order + :successes + :warnings]))))) (defn ^:private compiler->status-map "Cause compiler contains only two :registry and :internal/cm-running-order keys @@ -171,24 +189,40 @@ (contains? compiler :status)) (case (:status compiler) :ok (if (true? (:debug-result (utils/execute-config))) - (-> (smap/status-map-pure compiler)) - (-> (smap/status-map-pure (select-keys compiler [:registry :internal/cm-running-order :successes :warnings])))) + (-> + (smap/status-map-pure compiler)) + (-> + (smap/status-map-pure (select-keys compiler + [:uuid + :registry + :internal/cm-running-order + :successes + :warnings])))) :failed compiler) - (-> (smap/status-map-pure compiler) - (smap/status-map-handle-error {:message "Corrupted compiler structure"})))) + (-> + (smap/status-map-pure compiler) + (smap/status-map-handle-error {:message "Corrupted compiler structure"})))) (defn execute [registry-or-compiler instruction] {:pre [(or (map? registry-or-compiler) (sequential? registry-or-compiler))]} - (let [;; Under (build-compiler) we ment the unfinished status map - status-map-with-compiler (-> (cond - (map? registry-or-compiler) (-> (compiler->status-map registry-or-compiler)) - (sequential? registry-or-compiler) - (compiler->status-map (build-compiler registry-or-compiler instruction))) - (assoc :instruction instruction))] - (cond-> (execute-commands! status-map-with-compiler) - (false? (:debug-result (utils/execute-config))) (dissoc :internal/cm-running-order) - (false? (:debug-result (utils/execute-config))) (dissoc :registry)))) + (binding [utils/*execute-internals* (utils/-execute-internals-push (str (random-uuid)))] + (let [ ;; Under (build-compiler) we ment the unfinished status map + start-time (utils/now) + status-map-with-compiler (-> (cond + (map? registry-or-compiler) + (-> + (compiler->status-map registry-or-compiler)) + (sequential? registry-or-compiler) + (-> + (build-compiler registry-or-compiler instruction) + (compiler->status-map))) + (assoc :instruction instruction))] + (cond-> (execute-commands! status-map-with-compiler) + (false? (:debug-result (utils/execute-config))) (dissoc :internal/cm-running-order) + (false? (:debug-result (utils/execute-config))) (dissoc :registry) + :always (smap/status-map-add-measurement "execute" start-time (utils/now)) + :always (utils/hook-process (:hook-execute-end (utils/execute-config))))))) (defn failed? [status-map] (smap/failed? status-map)) (defn ok? [status-map] (smap/ok? status-map)) diff --git a/src/commando/impl/dependency.cljc b/src/commando/impl/dependency.cljc index ef46787..50f753a 100644 --- a/src/commando/impl/dependency.cljc +++ b/src/commando/impl/dependency.cljc @@ -3,13 +3,29 @@ [commando.impl.command-map :as cm] [commando.impl.utils :as utils])) +(defn- build-path-trie + "Builds a trie from a list of CommandMapPath objects for efficient path-based lookups." + [cm-list] + (reduce + (fn [trie cmd] + (assoc-in trie (conj (cm/command-path cmd) ::command) cmd)) + {} + cm-list)) + +(defn- get-all-nested-commands + "Lazily traverses a trie. + Returns a lazy sequence of all command objects found." + [trie] + (->> (tree-seq map? (fn [node] (vals (dissoc node ::command))) trie) + (keep ::command))) + (defmulti find-command-dependencies "Finds command dependencies based on dependency-type. Returns a set of CommandMapPath objects that must execute before the given command. Modes: - - :all-inside - depends on all commands inside the Command: + - :all-inside - depends on all commands inside the Command: if it Map - the values, if it Vector - elements of vectors - :all-inside-recur - depends on all commands nested within this command's path - :point - depends on command(s) at a specific path, defined by :point-key @@ -20,28 +36,29 @@ walks up the path hierarchy to find parent commands that will create/modify the target path. - :none - no dependencies (not implemented, returns empty set by default)" - (fn [_command-path-obj _instruction _commands dependency-type] dependency-type)) + (fn [_command-path-obj _instruction _path-trie dependency-type] dependency-type)) (defmethod find-command-dependencies :default - [_command-path-obj _instruction _commands type] + [_command-path-obj _instruction _path-trie type] (throw (ex-info (str utils/exception-message-header "Undefined dependency mode: " type) {:message (str utils/exception-message-header "Undefined dependency mode: " type) :dependency-mode type}))) (defmethod find-command-dependencies :all-inside - [command-path-obj _instruction commands _type] - (let [command-path-obj-length (count (cm/command-path command-path-obj))] - (->> (disj commands command-path-obj) - (filter #(and - (= (dec (count (cm/command-path %))) command-path-obj-length) - (cm/start-with? % command-path-obj))) - (set)))) + [command-path-obj _instruction path-trie _type] + (let [command-path (cm/command-path command-path-obj) + sub-trie (get-in path-trie command-path)] + (->> (vals (dissoc sub-trie ::command)) + (keep ::command) + set))) (defmethod find-command-dependencies :all-inside-recur - [command-path-obj _instruction commands _type] - (->> (disj commands command-path-obj) - (filter #(cm/start-with? % command-path-obj)) - (set))) + [command-path-obj _instruction path-trie _type] + (let [command-path (cm/command-path command-path-obj) + sub-trie (get-in path-trie command-path)] + (->> (get-all-nested-commands sub-trie) + (remove #(= % command-path-obj)) + set))) (defn resolve-relative-path "Resolves path segments with relative navigation (../ and ./) against a base path." @@ -61,13 +78,6 @@ segments)] (if relative (concat relative path) path))) -(defn find-commands-at-target-path - "Finds all commands at or nested within the target path." - [commands target-path] - (->> commands - (filter #(cm/vector-starts-with? (cm/command-path %) target-path)) - set)) - (defn path-exists-in-instruction? "Checks if a path exists in the instruction map." [instruction path] @@ -97,46 +107,44 @@ vec))) (defmethod find-command-dependencies :point - [command-path-obj instruction commands _type] - (let [target-path (point-target-path instruction command-path-obj) - commands-at-target (cm/->CommandMapPath target-path {})] - (if-let [point-command (first (filter #(= % commands-at-target) commands))] + [command-path-obj instruction path-trie _type] + (let [target-path (point-target-path instruction command-path-obj)] + (if-let [point-command (get-in path-trie (conj target-path ::command))] #{point-command} (throw-point-error command-path-obj target-path instruction)))) -(defn point-find-parent-command - "Walks up the path hierarchy to find the first parent command that exists in commands." - [commands target-path] +(defn- point-find-parent-command + "Walks up the path hierarchy to find the first parent command that exists in the trie." + [path-trie target-path] (loop [current-path target-path] - (cond - (empty? current-path) nil - (contains? commands (cm/->CommandMapPath current-path {})) - (first (filter #(= (cm/command-path %) current-path) commands)) - :else (recur (butlast current-path))))) + (when (seq current-path) + (if-let [cmd (get-in path-trie (conj current-path ::command))] + cmd + (recur (butlast current-path)))))) (defmethod find-command-dependencies :point-and-all-inside-recur - [command-path-obj instruction commands _type] + [command-path-obj instruction path-trie _type] (let [target-path (point-target-path instruction command-path-obj) - commands-at-target (find-commands-at-target-path commands target-path) - parent-command (point-find-parent-command commands target-path)] + sub-trie (get-in path-trie target-path) + commands-at-target (set (get-all-nested-commands sub-trie)) + parent-command (point-find-parent-command path-trie target-path)] (cond (not-empty commands-at-target) commands-at-target parent-command #{parent-command} (path-exists-in-instruction? instruction target-path) #{} :else (throw-point-error command-path-obj target-path instruction)))) -(defmethod find-command-dependencies :none [_command-path-obj _instruction _commands _type] #{}) +(defmethod find-command-dependencies :none [_command-path-obj _instruction _path-trie _type] #{}) (defn build-dependency-graph "Builds the dependency map for all commands in `cm-list`. Returns a map from CommandMapPath objects to their dependency sets." [instruction cm-list] - (let [command-set (set cm-list)] + (let [path-trie (build-path-trie cm-list)] (reduce (fn [dependency-acc command-path-obj] (let [dependency-mode (get-in (cm/command-data command-path-obj) [:dependencies :mode])] - (assoc dependency-acc - command-path-obj - (find-command-dependencies command-path-obj instruction command-set dependency-mode) - ))) + (assoc dependency-acc + command-path-obj + (find-command-dependencies command-path-obj instruction path-trie dependency-mode)))) {} cm-list))) diff --git a/src/commando/impl/executing.cljc b/src/commando/impl/executing.cljc index 8f0a454..4823088 100644 --- a/src/commando/impl/executing.cljc +++ b/src/commando/impl/executing.cljc @@ -37,3 +37,4 @@ (if (:error execution-result) [current-instruction (:error execution-result)] (recur execution-result (rest remaining-commands))))))) + diff --git a/src/commando/impl/finding_commands.cljc b/src/commando/impl/finding_commands.cljc index fd15e67..a93557c 100644 --- a/src/commando/impl/finding_commands.cljc +++ b/src/commando/impl/finding_commands.cljc @@ -7,8 +7,8 @@ "Returns child paths for regular collections that should be traversed." [value current-path] (cond - (map? value) (for [[k _v] value] (conj current-path k)) - (coll? value) (for [[i _v] (map-indexed vector value)] (conj current-path i)) + (map? value) (doall (map (fn [[k _v]] (conj current-path k)) (seq value))) + (coll? value) (doall (map (fn [i] (conj current-path i)) (range (count value)))) :else [])) (defmulti ^:private command-child-paths @@ -74,13 +74,13 @@ (defn find-commands "Traverses the instruction tree (BFS algo) and collects all commands defined by the registry." [instruction command-registry] - (loop [queue [[]] + (loop [queue (vec [[]]) found-commands [] debug-stack-map {}] (if (empty? queue) found-commands (let [current-path (first queue) - remaining-paths (rest queue) + remaining-paths (subvec queue 1) current-value (get-in instruction current-path) debug-stack (if (:debug-result (utils/execute-config)) (get debug-stack-map current-path (list)) (list))] (if-let [command-spec (instruction-command-spec command-registry current-value current-path)] @@ -91,8 +91,9 @@ updated-debug-stack-map (if (:debug-result (utils/execute-config)) (reduce #(assoc %1 %2 (conj debug-stack command)) debug-stack-map child-paths) {})] - (recur (concat remaining-paths child-paths) (conj found-commands command) updated-debug-stack-map)) + (recur (into remaining-paths child-paths) (conj found-commands command) updated-debug-stack-map)) ;; No match - traverse children if coll, skip if leaf - (recur (concat remaining-paths (coll-child-paths current-value current-path)) + (recur (into remaining-paths (coll-child-paths current-value current-path)) found-commands debug-stack-map)))))) + diff --git a/src/commando/impl/graph.cljc b/src/commando/impl/graph.cljc index 68d1fb0..634fc6e 100644 --- a/src/commando/impl/graph.cljc +++ b/src/commando/impl/graph.cljc @@ -2,27 +2,51 @@ (:require [clojure.set :as set])) -(defn ^:private no-incoming - "Returns the set of nodes in graph g for which there are no incoming - edges, where g is a map of nodes to sets of nodes." +(defn topological-sort + "Efficiently sorts a directed acyclic graph using Kahn's algorithm with in-degree counting. + 'g' is a map of nodes to sequences of their dependencies. + Returns a map with :sorted containing the topologically sorted list of nodes, + and :cyclic containing the remaining nodes if a cycle is detected." [g] - (let [nodes (set (keys g)) have-incoming (apply set/union (vals g))] (set/difference nodes have-incoming))) + (let [;; Build the reverse graph to easily find dependents and collect all nodes. + rev-g (reduce-kv (fn [acc k vs] + (reduce (fn [a v] (update a v (fnil conj []) k)) acc vs)) + {} g) + all-nodes (set/union (set (keys g)) (set (keys rev-g))) + + ;; calculate in-degrees for all nodes. + in-degrees (reduce-kv (fn [acc node deps] + (assoc acc node (count deps))) + {} g) + + ;; Initialize the queue with nodes that have no incoming edges. + ;; Using a vector as a FIFO queue. + q (reduce (fn [queue node] + (if (zero? (get in-degrees node 0)) + (conj queue node) + queue)) + [] all-nodes)] + (loop [queue q + sorted-result [] + degrees in-degrees] + (if-let [node (first queue)] + (let [dependents (get rev-g node []) + ;; Reduce in-degree for all dependents + ;; and find new nodes with zero in-degree. + [next-degrees new-zero-nodes] + (reduce (fn [[degs zeros] dep] + (let [new-degree (dec (get degs dep))] + [(assoc degs dep new-degree) + (if (zero? new-degree) (conj zeros dep) zeros)])) + [degrees []] + dependents)] + (recur (into (subvec queue 1) new-zero-nodes) + (conj sorted-result node) + next-degrees)) + (if (= (count sorted-result) (count all-nodes)) + {:sorted sorted-result :cyclic {}} + (let [cyclic-nodes (->> degrees + (filter (fn [[_ v]] (pos? v))) + (into {}))] + {:sorted sorted-result :cyclic cyclic-nodes})))))) -(defn topological-sort - "Sort for acyclic directed graph g (using khan algo). - Where g is a map of nodes to sets of nodes. - If g contains cycles, it orders the acyclic parts and leaves cyclic parts as is. - Returns a map with :sorted containing result and :cyclic containing cycles if exists" - ([g] (topological-sort g [] (no-incoming g))) - ([g l s] - (if (empty? s) - (if (every? empty? (vals g)) - {:sorted l} - {:sorted l - :cyclic (filter (fn [[_ v]] (seq v)) g)}) - (let [[n s'] [(first s) (rest s)] - m (g n) - g' (assoc g n #{}) - new-nodes (set/intersection (no-incoming g') m) - s'' (set/union s' new-nodes)] - (recur g' (conj l n) s''))))) diff --git a/src/commando/impl/status_map.cljc b/src/commando/impl/status_map.cljc index 4c3c656..9270366 100644 --- a/src/commando/impl/status_map.cljc +++ b/src/commando/impl/status_map.cljc @@ -1,10 +1,25 @@ (ns commando.impl.status-map (:require + [commando.impl.utils :as utils] [malli.core :as malli])) (def ^:private status-map-message-schema [:map [:message [:string {:min 5}]]]) +;;;------ +;;; Stats +;;;------ + +(defn status-map-add-measurement + "Calculates the duration from `start-time-ns` and `end-time-ns` and appends it as a tuple + `[stat-key duration-ns formatted-duration]` to the `:stats` vector in the `status-map`." + [status-map stat-key start-time-ns end-time-ns] + (let [duration (- end-time-ns start-time-ns)] + (update status-map :stats (fnil conj []) + [stat-key + duration + (utils/format-time duration)]))) + (defn status-map-handle-warning [status-map m] (update status-map :warnings (fnil conj []) (malli/coerce status-map-message-schema m))) @@ -22,10 +37,12 @@ (defn status-map-pure ([] (status-map-pure nil)) ([m] - (merge {:status :ok + (merge {:uuid (:uuid utils/*execute-internals*) + :status :ok :errors [] :warnings [] - :successes []} + :successes [] + :stats []} m))) (defn status-map-undefined-status @@ -36,3 +53,5 @@ (defn failed? [status-map] (= (:status status-map) :failed)) (defn ok? [status-map] (= (:status status-map) :ok)) + + diff --git a/src/commando/impl/utils.cljc b/src/commando/impl/utils.cljc index d547c77..35e8aa2 100644 --- a/src/commando/impl/utils.cljc +++ b/src/commando/impl/utils.cljc @@ -1,5 +1,6 @@ (ns commando.impl.utils - (:require [malli.core :as malli])) + (:require [malli.core :as malli] + [clojure.string :as str])) (def exception-message-header "Commando. ") @@ -9,7 +10,9 @@ (def ^:private -execute-config-default {:debug-result false - :error-data-string true}) + :error-data-string true + :hook-execute-end nil + :hook-execute-start nil}) (def ^:dynamic *execute-config* @@ -19,25 +22,83 @@ - `:error-data-string` (boolean): When true, the `:data` key in serialized `ExceptionInfo` (via `commando.impl.utils/serialize-exception`) will be a string representation of the data. When false, it will return - the original map structure." + the original map structure. + - `:hook-execute-start` (fn [status-map]): if not nil, can run procedure + passed in value. + - `:hook-execute-end` (fn [status-map]): if not nil, can run procedure + passed in value. + + Example + (binding [commando.impl.utils/*execute-config* + {:debug-result true + :error-data-string false + :hook-execute-start (fn [e] (println (:uuid e))) + :hook-execute-end (fn [e] (println (:uuid e) (:stats e)))}] + (commando.core/execute + [commando.commands.builtin/command-from-spec] + {\"1\" 1 + \"2\" {:commando/from [\"1\"]} + \"3\" {:commando/from [\"2\"]}}))" -execute-config-default) +(def ^:dynamic + *execute-internals* + "Dynamic variable to keep context information about the execution + setup. + - `:uuid` the unique name of execution, generated everytime the user + invoke `commando.core/execute` + - `:stack` in case of user use `commando.commands.builtin/command-macro-spec`, + or `commando.commands.query-dsl/command-resolve-json-spec` or any sort of + commands what invoking `commando.core/execute` inside of parent instruction + by simulation recursive call, the :stack key will store the invocation stack + in vector of :uuids" + {:uuid nil + :stack []}) + +(defn -execute-internals-push + "Update *execute-internals* structure" + [uuid-execute-identifier] + (-> *execute-internals* + (assoc :uuid uuid-execute-identifier) + (update :stack conj uuid-execute-identifier))) + (defn execute-config "Returns the effective configuration for `commando/execute`, getting data from dynamic variable `commando.impl.utils/*execute-config*`" [] (merge -execute-config-default *execute-config*)) -(def ^{:dynamic true - :private true - :doc "For debugging purposes and some mysterious reason of setting it dynamically during execution"} +(defn hook-process + "Function will handle a hooks passed from users. + Available hooks: + - `:hook-execute-start`, + - `:hook-execute-end`. + + Read more: + `commando.impl.utils/*execute-config*`" + [status-map hook] + (when hook + (try + (hook status-map) + (catch #?(:clj Exception + :cljs :default) e + nil))) + status-map) + +(def ^:dynamic *command-map-spec-registry* + "Dynamic variable what keep the state of processed + `:registry` value from `status-map`" nil) (defn command-map-spec-registry - "For debugging purposes and some mysterious reason of setting it dynamically during execution" - [] - (or *command-map-spec-registry* [])) + "Return `:registry` value in dynamic scoupe. + Required to run `commando.core/execute` inside + of parent execute invocation. + See + `commando.core/execute` + `commando.core/execute-commands!`(binding)" + [] (or *command-map-spec-registry* [])) ;; ------------------ ;; Function Resolvers @@ -86,6 +147,25 @@ (fn [x] (some? (resolve-fn x)))])) +;; ----------------- +;; Performance Tools +;; ----------------- + +(defn now + "Returns a high-resolution timestamp in nanoseconds." + [] + #?(:clj (System/nanoTime) + :cljs (* (.now js/performance) 1000000))) + +(defn format-time + "Formats a time `t` in nanoseconds to a string with units (ns, µs, ms, or s)." + [t] + (cond + (< t 1000) (str t "ns") + (< t 1000000) (str (float (/ t 1000)) "µs") + (< t 1000000000) (str (float (/ t 1000000)) "ms") + :else (str (float (/ t 1000000000)) "s"))) + ;; ----------- ;; Error Tools ;; ----------- diff --git a/test/perf/commando/core_perf_test.clj b/test/perf/commando/core_perf_test.clj index 76146aa..13c0d01 100644 --- a/test/perf/commando/core_perf_test.clj +++ b/test/perf/commando/core_perf_test.clj @@ -1,398 +1,520 @@ (ns commando.core-perf-test - "Performance tests for Commando execute function. + (:require + [commando.impl.utils] + [commando.commands.builtin] + [commando.commands.query-dsl] + [commando.core] + [clojure.string :as str] + [cljfreechart.core :as cljfreechart])) - Run these tests individually in the REPL for detailed performance analysis. +;; ===================================== +;; PRINT UTILS +;; ===================================== - Performance tests were run with following setup: - Model Name: MacBook Pro - Model Identifier: Mac14,10 - Processor Name: Apple M2 Pro - Number of Processors: 1 - Total Number of Cores: 12 (8 performance and 4 efficiency) - L2 Cache (per Core): 4 MB - Memory: 16 GB +(defn print-stats + "Prints a formatted summary of the execution stats from a status-map." + ([status-map] + (print-stats status-map nil)) + ([status-map title] + (when-let [stats (:stats status-map)] + (let [max-key-len (apply max 0 (map (comp count name first) stats))] + (println (str "\nExecution Stats" (when title (str "(" title ")")) ":")) + (doseq [[index [stat-key _ formatted]] (map-indexed vector stats)] + (let [key-str (name stat-key) + padding (str/join "" (repeat (- max-key-len (count key-str)) " "))] + (println (str + " " (if (= "execute" key-str) "=" (str (inc index)) ) + " " key-str " " padding formatted)))))))) +(comment + (print-stats + (commando.core/execute + [commando.commands.builtin/command-fn-spec + commando.commands.builtin/command-from-spec + commando.commands.builtin/command-apply-spec] + {"1" 1 + "2" {:commando/from ["1"]} + "3" {:commando/from ["2"]}}))) - Basic performance tests: - (perf-simple-command) ; Single command execution - (perf-dependency-chain) ; 5-step dependency chain - (perf-medium-parallel) ; 50 independent commands - (perf-large-parallel) ; 200 independent commands +;; ======================================= +;; AVERAGE EXECUTION OF REAL WORLD EXAMPLE +;; ======================================= - Complex scenarios: - (perf-mixed-workflow) ; Multi-command workflow with dependencies - (perf-dependent-chain) ; 25-level dependency chain - (perf-wide-fanout) ; 1->100 fan-out pattern - (perf-deep-nested-instruction) ; 7 levels deep with 50+ cross-level references +(defn ^:private calculate-average-stats + "Takes a collection of status-maps and calculates the average duration for each stat-key." + [status-maps] + {:pre [(not-empty status-maps)]} + (let [keys-order (map first (:stats (first status-maps))) + all-stats (mapcat :stats status-maps) + grouped-stats (group-by first all-stats) + averages-grouped + (reduce (fn [acc [stat-key measurements]] + (let [total-duration (reduce + (map second measurements)) + count-measurements (count measurements) + average-duration (long (/ total-duration count-measurements))] + (assoc acc stat-key [stat-key average-duration (commando.impl.utils/format-time average-duration)]))) + {} grouped-stats)] + {:stats (mapv #(get averages-grouped %) keys-order)})) - Edge cases: - (perf-empty-instruction) ; Empty instruction overhead +(defmacro repeat-n-and-print-stats + "Repeats the execution of `body` `n` times, collects the status-maps," + [n & body] + `(let [results# (doall (for [_# (range ~n)] + ~@body)) + avg-stats# (calculate-average-stats results#)] + (print "Repeating instruction " ~n " times") + (print-stats avg-stats#))) - Comparative tests: - (perf-registry-vs-compiled) ; Direct vs pre-compiled performance +(defn real-word-calculation-average-of-50 [] + (println "\n=====================Benchmark=====================") + (println "Real Word calculation. Show average of 50 execution") + (println "===================================================") + (repeat-n-and-print-stats 50 + (commando.core/execute + [commando.commands.builtin/command-fn-spec + commando.commands.builtin/command-from-spec + commando.commands.builtin/command-apply-spec] + { ;; -------------------------------------------------------------------------------- + ;; RAW DATA & CONFIGURATION + ;; -------------------------------------------------------------------------------- + :config + {:commission-rates {:standard 0.07 :senior 0.12} + :bonus-threshold 50000 + :performance-bonus 2500 + :tax-rate 0.21 + :department-op-cost {:sales 15000 :marketing 10000 :engineering 25000}} - Run all benchmarks: - (run-all-benchmarks) ; Execute complete performance suite" - (:require - [commando.commands.builtin :as cmds-builtin] - [commando.core :as commando] - [criterium.core :as cc])) - -;;TODO include test with a really big registry, so like there are 50 command specs - -;; Test command definitions -(def test-add-id-command - {:type :test/add-id - :recognize-fn #(and (map? %) (contains? % :test/add-id)) - :apply (fn [_instruction _command-path-obj command-map] (assoc command-map :id :test-id)) - :dependencies {:mode :all-inside}}) - -;; Performance test functions -(defn perf-simple-command - [] - ;; === Simple single command execution === - ;; Evaluation count : 34368 in 6 samples of 5728 calls. - ;; Execution time mean : 18.710971 µs - ;; Execution time std-deviation : 317.972351 ns - ;; Execution time lower quantile : 18.283290 µs ( 2.5%) - ;; Execution time upper quantile : 19.018313 µs (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - instruction {"cmd" {:test/add-id "test"}}] - ;; Verify correctness first - (assert (= :ok (:status (commando/execute registry instruction)))) - (assert (contains? (get-in (commando/execute registry instruction) [:instruction "cmd"]) :id)) - (println "\n=== Simple single command execution ===") - (cc/quick-bench (commando/execute registry instruction)))) - -(defn perf-dependency-chain - "Benchmark 5-step dependency chain." - [] - ;; === 5-step dependency chain === - ;; Evaluation count : 10368 in 6 samples of 1728 calls. - ;; Execution time mean : 62.820517 µs - ;; Execution time std-deviation : 1.072545 µs - ;; Execution time lower quantile : 61.709531 µs ( 2.5%) - ;; Execution time upper quantile : 63.890134 µs (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - instruction {"step1" {:test/add-id "first"} - "step2" {:commando/from ["step1"]} - "step3" {:commando/from ["step2"]} - "step4" {:commando/from ["step3"]} - "step5" {:commando/from ["step4"]}}] - (let [result (commando/execute registry instruction)] - (assert (= :ok (:status result))) - (assert (every? #(contains? (get-in result [:instruction %]) :id) ["step1" "step2" "step3" "step4" "step5"]))) - (println "\n=== 5-step dependency chain ===") - (cc/quick-bench (commando/execute registry instruction)))) - -(defn perf-medium-parallel - "Benchmark 50 independent commands." - [] - ;; === 50 independent commands === - ;; Evaluation count : 132 in 6 samples of 22 calls. - ;; Execution time mean : 5.021323 ms - ;; Execution time std-deviation : 201.747532 µs - ;; Execution time lower quantile : 4.761436 ms ( 2.5%) - ;; Execution time upper quantile : 5.239934 ms (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - instruction (into {} - (for [i (range 50)] - [i {:test/add-id (str "value-" i)}]))] - (let [result (commando/execute registry instruction)] - (assert (= :ok (:status result))) - (assert (= 50 (count (filter #(contains? % :id) (vals (:instruction result))))))) - (println "\n=== 50 independent commands ===") - (cc/quick-bench (commando/execute registry instruction)))) - -(defn perf-large-parallel - "Benchmark 200 independent commands." - [] - ;; === 200 independent commands === - ;; Evaluation count : 12 in 6 samples of 2 calls. - ;; Execution time mean : 81.060446 ms - ;; Execution time std-deviation : 3.257158 ms - ;; Execution time lower quantile : 77.974832 ms ( 2.5%) - ;; Execution time upper quantile : 86.378269 ms (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - instruction (into {} - (for [i (range 200)] - [i {:test/add-id (str "value-" i)}]))] - (let [result (commando/execute registry instruction)] - (assert (= :ok (:status result))) - (assert (= 200 (count (filter #(contains? % :id) (vals (:instruction result))))))) - (println "\n=== 200 independent commands ===") - (cc/quick-bench (commando/execute registry instruction)))) - -(defn perf-mixed-workflow - "Benchmark complex mixed workflow with different command types" - [] - ;; === Complex mixed workflow === - ;; Evaluation count : 2472 in 6 samples of 412 calls. - ;; Execution time mean : 274.588157 µs - ;; Execution time std-deviation : 32.080465 µs - ;; Execution time lower quantile : 246.402883 µs ( 2.5%) - ;; Execution time upper quantile : 323.434512 µs (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command cmds-builtin/command-fn-spec] - instruction {"source" 100 - "doubled" {:commando/fn * - :args [{:commando/from ["source"]} 2]} - "tripled" {:commando/fn * - :args [{:commando/from ["source"]} 3]} - "sum" {:commando/fn + - :args [{:commando/from ["doubled"]} {:commando/from ["tripled"]}]} - "metadata" {:test/add-id "workflow"} - "result" {:commando/from ["sum"]}}] - (let [result (commando/execute registry instruction)] - (assert (= :ok (:status result))) - (assert (= 500 (get-in result [:instruction "sum"]))) - (assert (= 500 (get-in result [:instruction "result"]))) - (assert (contains? (get-in result [:instruction "metadata"]) :id))) - (println "\n=== Complex mixed workflow ===") - (cc/quick-bench (commando/execute registry instruction)))) - -(defn perf-dependent-chain - "Benchmark 25 dependent commands chain" - [] - ;; === 25 dependency chain === - ;; Evaluation count : 1278 in 6 samples of 213 calls. - ;; Execution time mean : 509.752558 µs - ;; Execution time std-deviation : 14.089436 µs - ;; Execution time lower quantile : 489.605357 µs ( 2.5%) - ;; Execution time upper quantile : 524.969061 µs (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - instruction - (reduce (fn [inst i] - (if (= i 0) (assoc inst i {:test/add-id (str "base-" i)}) (assoc inst i {:commando/from [(dec i)]}))) - {} - (range 25))] - (let [result (commando/execute registry instruction)] - (assert (= :ok (:status result))) - (assert (every? #(contains? (get-in result [:instruction %]) :id) (range 25)))) - (println "\n=== 25 dependency chain ===") - (cc/quick-bench (commando/execute registry instruction)))) - -(defn perf-wide-fanout - "Benchmark wide fan-out (1->100 dependencies)" - [] - ;; === Wide fan-out (1->100) === - ;; Evaluation count : 120 in 6 samples of 20 calls. - ;; Execution time mean : 5.575331 ms - ;; Execution time std-deviation : 307.267615 µs - ;; Execution time lower quantile : 5.198288 ms ( 2.5%) - ;; Execution time upper quantile : 6.052582 ms (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - base-instruction {"base" {:test/add-id "shared"}} - instruction (into base-instruction - (for [i (range 100)] - [i {:commando/from ["base"]}]))] - (let [result (commando/execute registry instruction)] - (assert (= :ok (:status result))) - (assert (contains? (get-in result [:instruction "base"]) :id)) - (assert (every? #(contains? (get-in result [:instruction %]) :id) (range 100)))) - (println "\n=== Wide fan-out (1->100) ===") - (cc/quick-bench (commando/execute registry instruction)))) - - -(defn perf-empty-instruction - "Benchmark empty instruction overhead" - [] - ;; === Empty instruction === - ;; Evaluation count : 59592 in 6 samples of 9932 calls. - ;; Execution time mean : 10.729503 µs - ;; Execution time std-deviation : 418.557691 ns - ;; Execution time lower quantile : 10.167363 µs ( 2.5%) - ;; Execution time upper quantile : 11.199093 µs (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command]] - (assert (= :ok (:status (commando/execute registry {})))) - (assert (= {} (:instruction (commando/execute registry {})))) - (println "\n=== Empty instruction ===") - (cc/quick-bench (commando/execute registry {})))) - -(defn perf-deep-nested-instruction - "Benchmark deeply nested instruction map with 50+ command references across multiple levels" - [] - ;; === Deep nested instruction (7 levels, 50+ commands) === - ;; Evaluation count : 234 in 6 samples of 39 calls. - ;; Execution time mean : 2.613528 ms - ;; Execution time std-deviation : 74.361807 µs - ;; Execution time lower quantile : 2.510160 ms ( 2.5%) - ;; Execution time upper quantile : 2.699545 ms (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - instruction {"root1" {:test/add-id "root-cmd-1"} - "root2" {:test/add-id "root-cmd-2"} - "root3" {:commando/from ["root1"]} - "level1" - {"l1-cmd1" {:test/add-id "level1-base"} - "l1-cmd2" {:commando/from ["root1"]} - "l1-cmd3" {:commando/from ["root2"]} - "l1-cmd4" {:commando/from ["level1" "l1-cmd1"]} - "level2" - {"l2-cmd1" {:test/add-id "level2-base"} - "l2-cmd2" {:commando/from ["level1" "l1-cmd1"]} - "l2-cmd3" {:commando/from ["root1"]} - "l2-cmd4" {:commando/from ["level1" "l1-cmd2"]} - "l2-cmd5" {:commando/from ["level1" "level2" "l2-cmd1"]} - "level3" - {"l3-cmd1" {:test/add-id "level3-base"} - "l3-cmd2" {:commando/from ["level1" "level2" "l2-cmd1"]} - "l3-cmd3" {:commando/from ["root2"]} - "l3-cmd4" {:commando/from ["level1" "l1-cmd1"]} - "l3-cmd5" {:commando/from ["level1" "level2" "l2-cmd2"]} - "l3-cmd6" {:commando/from ["level1" "level2" "level3" "l3-cmd1"]} - "level4" - {"l4-cmd1" {:test/add-id "level4-base"} - "l4-cmd2" {:commando/from ["level1" "level2" "level3" "l3-cmd1"]} - "l4-cmd3" {:commando/from ["level1" "l1-cmd1"]} - "l4-cmd4" {:commando/from ["root1"]} - "l4-cmd5" {:commando/from ["level1" "level2" "l2-cmd1"]} - "l4-cmd6" {:commando/from ["level1" "level2" "level3" "l3-cmd2"]} - "l4-cmd7" {:commando/from ["level1" "level2" "level3" "level4" "l4-cmd1"]} - "level5" - {"l5-cmd1" {:test/add-id "level5-base"} - "l5-cmd2" {:commando/from ["level1" "level2" "level3" "level4" "l4-cmd1"]} - "l5-cmd3" {:commando/from ["root2"]} - "l5-cmd4" {:commando/from ["level1" "level2" "l2-cmd1"]} - "l5-cmd5" {:commando/from ["level1" "level2" "level3" "l3-cmd1"]} - "l5-cmd6" {:commando/from ["level1" "level2" "level3" "level4" "l4-cmd2"]} - "l5-cmd7" {:commando/from ["level1" "level2" "level3" "level4" "level5" "l5-cmd1"]} - "l5-cmd8" {:commando/from ["level1" "l1-cmd3"]} - "level6" - {"l6-cmd1" {:test/add-id "level6-base"} - "l6-cmd2" {:commando/from ["level1" "level2" "level3" "level4" "level5" "l5-cmd1"]} - "l6-cmd3" {:commando/from ["root1"]} - "l6-cmd4" {:commando/from ["level1" "level2" "level3" "l3-cmd1"]} - "l6-cmd5" {:commando/from ["level1" "level2" "level3" "level4" "l4-cmd1"]} - "l6-cmd6" {:commando/from ["level1" "level2" "level3" "level4" "level5" "l5-cmd2"]} - "l6-cmd7" {:commando/from ["level1" "level2" "level3" "level4" "level5" "level6" "l6-cmd1"]} - "l6-cmd8" {:commando/from ["level1" "l1-cmd2"]} - "l6-cmd9" {:commando/from ["level1" "level2" "l2-cmd3"]} - "level7" - {"l7-cmd1" {:test/add-id "level7-base"} - "l7-cmd2" {:commando/from ["level1" "level2" "level3" "level4" "level5" "level6" "l6-cmd1"]} - "l7-cmd3" {:commando/from ["root2"]} - "l7-cmd4" {:commando/from ["level1" "level2" "level3" "level4" "l4-cmd1"]} - "l7-cmd5" {:commando/from ["level1" "level2" "level3" "level4" "level5" "l5-cmd1"]} - "l7-cmd6" {:commando/from ["level1" "level2" "level3" "level4" "level5" "level6" "l6-cmd2"]} - "l7-cmd7" {:commando/from - ["level1" "level2" "level3" "level4" "level5" "level6" "level7" "l7-cmd1"]} - "l7-cmd8" {:commando/from ["level1" "level2" "l2-cmd4"]} - "l7-cmd9" {:commando/from ["level1" "level2" "level3" "l3-cmd3"]} - "l7-cmd10" {:commando/from ["level1" "level2" "level3" "level4" "level5" "l5-cmd3"]}}}}}}}}} - result (commando/execute registry instruction)] - (assert (= :ok (:status result))) - ;; Verify commands at various levels have been processed - (assert (contains? (get-in result [:instruction "root1"]) :id)) - (assert (contains? (get-in result [:instruction "level1" "l1-cmd1"]) :id)) - (assert (contains? (get-in result [:instruction "level1" "level2" "l2-cmd1"]) :id)) - (assert (contains? (get-in result [:instruction "level1" "level2" "level3" "l3-cmd1"]) :id)) - (assert (contains? (get-in result [:instruction "level1" "level2" "level3" "level4" "l4-cmd1"]) :id)) - (assert (contains? (get-in result [:instruction "level1" "level2" "level3" "level4" "level5" "l5-cmd1"]) :id)) - (assert (contains? (get-in result [:instruction "level1" "level2" "level3" "level4" "level5" "level6" "l6-cmd1"]) - :id)) - (assert (contains? (get-in result - [:instruction "level1" "level2" "level3" "level4" "level5" "level6" "level7" "l7-cmd1"]) - :id)) - ;; Verify cross-level dependencies work - (assert (contains? (get-in result - [:instruction "level1" "level2" "level3" "level4" "level5" "level6" "level7" "l7-cmd2"]) - :id)) - (assert (contains? (get-in result - [:instruction "level1" "level2" "level3" "level4" "level5" "level6" "level7" "l7-cmd10"]) - :id)) - (println "\n=== Deep nested instruction (7 levels, 50+ commands) ===") - (cc/quick-bench (commando/execute registry instruction)))) - -(defn perf-registry-vs-compiled - "Compare direct registry vs pre-compiled performance" - [] - ;; === Direct registry execution === - ;; Baseline performance using registry directly - ;; Evaluation count : 16590 in 6 samples of 2765 calls. - ;; Execution time mean : 41.007615 µs - ;; Execution time std-deviation : 2.673693 µs - ;; Execution time lower quantile : 39.111703 µs ( 2.5%) - ;; Execution time upper quantile : 45.405028 µs (97.5%) - ;; Overhead used : 1.648411 ns - ;; - ;; === Pre-compiled execution === - ;; Performance using pre-compiled version - ;; Evaluation count : 173178 in 6 samples of 28863 calls. - ;; Execution time mean : 3.671263 µs - ;; Execution time std-deviation : 121.707808 ns - ;; Execution time lower quantile : 3.486865 µs ( 2.5%) - ;; Execution time upper quantile : 3.806872 µs (97.5%) - ;; Overhead used : 1.648411 ns - (let [registry [cmds-builtin/command-from-spec test-add-id-command] - instruction {"step1" {:test/add-id "first"} - "step2" {:commando/from ["step1"]} - "step3" {:commando/from ["step2"]}} - compiler (commando/build-compiler registry instruction)] - (assert (= (:instruction (commando/execute registry instruction)) - (:instruction (commando/execute compiler instruction)))) - (println "\n=== Direct registry execution ===") - (println "Baseline performance using registry directly") - (cc/quick-bench (commando/execute registry instruction)) - (println "\n=== Pre-compiled execution ===") - (println "Performance using pre-compiled version") - (cc/quick-bench (commando/execute compiler instruction)))) - -(defn run-all-benchmarks - "Run complete performance benchmark suite" - [] - (println "=================================================================") - (println "COMMANDO PERFORMANCE BENCHMARK SUITE") - (println "=================================================================") - (println "Hardware specs will affect absolute timings.") - (println "Focus on relative performance and regression detection.") - (println "=================================================================") - (perf-simple-command) - (perf-dependency-chain) - (perf-medium-parallel) - (perf-large-parallel) - (perf-mixed-workflow) - (perf-empty-instruction) - (perf-wide-fanout) - (perf-dependent-chain) - (perf-deep-nested-instruction) - (perf-registry-vs-compiled) - (println "\n=================================================================") - (println "PERFORMANCE BENCHMARK SUITE COMPLETE") - (println "=================================================================")) - -(defn run-quick-benchmarks - "Run essential benchmarks for quick performance check" - [] - (println "=== QUICK PERFORMANCE CHECK ===") - (perf-simple-command) - (perf-dependency-chain) - (perf-medium-parallel) - (perf-mixed-workflow) - (perf-deep-nested-instruction) - (println "=== QUICK CHECK COMPLETE ===")) + :products + {"prod-001" {:name "Alpha Widget" :price 250.0} + "prod-002" {:name "Beta Gadget" :price 475.0} + "prod-003" {:name "Gamma Gizmo" :price 1200.0}} + + :employees + {"emp-101" {:name "John Doe" :department :sales :level :senior} + "emp-102" {:name "Jane Smith" :department :sales :level :standard} + "emp-103" {:name "Peter Jones" :department :marketing :level :senior} + "emp-201" {:name "Mary Major" :department :engineering :level :standard}} + + :sales-records + [ ;; John's Sales + {:employee-id "emp-101" :product-id "prod-003" :units-sold 50} + {:employee-id "emp-101" :product-id "prod-001" :units-sold 120} + ;; Jane's Sales + {:employee-id "emp-102" :product-id "prod-001" :units-sold 80} + {:employee-id "emp-102" :product-id "prod-002" :units-sold 40} + ;; Peter's Sales (Marketing can also sell) + {:employee-id "emp-103" :product-id "prod-002" :units-sold 10}] + + :calculations + {:sales-revenues + {:commando/fn (fn [sales products] + (mapv (fn [sale] + (let [product (get products (:product-id sale))] + (assoc sale :total-revenue (* (:units-sold sale) (:price product))))) + sales)) + :args [{:commando/from [:sales-records]} + {:commando/from [:products]}]} + + :employee-sales-totals + {:commando/fn (fn [sales-revenues] + (reduce (fn [acc sale] + (update acc + (:employee-id sale) + (fnil + 0) + (:total-revenue sale))) + {} + sales-revenues)) + :args [{:commando/from [:calculations :sales-revenues]}]} + + :employee-commissions + {:commando/apply + {:sales-totals {:commando/from [:calculations :employee-sales-totals]} + :employees {:commando/from [:employees]} + :rates {:commando/from [:config :commission-rates]}} + := (fn [{:keys [sales-totals employees rates]}] + (into {} + (map (fn [[emp-id total-sales]] + (let [employee (get employees emp-id) + rate-key (:level employee) + commission-rate (get rates rate-key 0)] + [emp-id (* total-sales commission-rate)])) + sales-totals)))} + + :employee-bonuses + {:commando/apply + {:sales-totals {:commando/from [:calculations :employee-sales-totals]} + :threshold {:commando/from [:config :bonus-threshold]} + :bonus-amount {:commando/from [:config :performance-bonus]}} + := (fn [{:keys [sales-totals threshold bonus-amount]}] + (into {} + (map (fn [[emp-id total-sales]] + [emp-id (if (> total-sales threshold) bonus-amount 0)]) + sales-totals)))} + + :employee-total-compensation + {:commando/fn (fn [commissions bonuses] + (merge-with + commissions bonuses)) + :args [{:commando/from [:calculations :employee-commissions]} + {:commando/from [:calculations :employee-bonuses]}]} + + :department-financials + {:commando/apply + {:employees {:commando/from [:employees]} + :sales-totals {:commando/from [:calculations :employee-sales-totals]} + :compensations {:commando/from [:calculations :employee-total-compensation]} + :op-costs {:commando/from [:config :department-op-cost]}} + := (fn [{:keys [employees sales-totals compensations op-costs]}] + (let [initial-agg {:sales {:total-revenue 0 :total-compensation 0} + :marketing {:total-revenue 0 :total-compensation 0} + :engineering {:total-revenue 0 :total-compensation 0}}] + (as-> (reduce-kv (fn [agg emp-id emp-data] + (let [dept (:department emp-data) + revenue (get sales-totals emp-id 0) + compensation (get compensations emp-id 0)] + (-> agg + (update-in [dept :total-revenue] + revenue) + (update-in [dept :total-compensation] + compensation)))) + initial-agg + employees) data + (merge-with + (fn [dept-data op-cost] + (let [profit (- (:total-revenue dept-data) + (+ (:total-compensation dept-data) op-cost))] + (assoc dept-data + :operating-cost op-cost + :net-profit profit))) + data + op-costs))))}} + + :final-report + {:commando/apply + {:dept-financials {:commando/from [:calculations :department-financials]} + :total-sales-per-employee {:commando/from [:calculations :employee-sales-totals]} + :total-compensation-per-employee {:commando/from [:calculations :employee-total-compensation]} + :tax-rate {:commando/from [:config :tax-rate]}} + := (fn [{:keys [dept-financials total-sales-per-employee total-compensation-per-employee tax-rate]}] + (let [company-total-revenue (reduce + (map :total-revenue (vals dept-financials))) + company-total-compensation (reduce + (map :total-compensation (vals dept-financials))) + company-total-op-cost (reduce + (map :operating-cost (vals dept-financials))) + company-gross-profit (- company-total-revenue + (+ company-total-compensation company-total-op-cost)) + taxes-payable (* company-gross-profit tax-rate) + company-net-profit (- company-gross-profit taxes-payable)] + {:company-summary + {:total-revenue company-total-revenue + :total-compensation company-total-compensation + :total-operating-cost company-total-op-cost + :gross-profit company-gross-profit + :taxes-payable taxes-payable + :net-profit-after-tax company-net-profit} + :department-breakdown dept-financials + :employee-performance + {:top-earner (key (apply max-key val total-compensation-per-employee)) + :top-seller (key (apply max-key val total-sales-per-employee))}}))}}))) + + +;; ============================== +;; FLAME FOR RECURSIVE INVOCATION +;; ============================== + +(defn ^:private flame-print-stats [stats indent] + (let [max-key-len (apply max 0 (map (comp count name first) stats))] + (doseq [[stat-key _ formatted] stats] + (let [key-str (name stat-key) + padding (clojure.string/join "" (repeat (- max-key-len (count key-str)) " "))] + (println (str indent + "" key-str " " padding formatted)))))) + +(defn ^:private flame-print [data & [indent]] + (let [indent (or indent "")] + (doseq [[k v] data] + (println (str indent "———" k)) + (when (:stats v) + (flame-print-stats (:stats v) (str indent " |"))) + (doseq [[child-k child-v] v + :when (map? child-v)] + (when (not= child-k :stats) + (flame-print {child-k child-v} (str indent " :"))))))) + +(defn ^:private flamegraph [data] + (println "Printing Flamegraph for executes:") + (flame-print data)) + +(defn ^:private execute-with-flame [registry instruction] + (let [stats-state (atom {}) + result + (binding [commando.impl.utils/*execute-config* + {; :debug-result true + :hook-execute-end + (fn [e] + (swap! stats-state + (fn [s] + (update-in s (:stack commando.impl.utils/*execute-internals*) + #(merge % {:stats (:stats e)})))))}] + (commando.core/execute + registry instruction))] + (flamegraph @stats-state) + result)) + +(defmethod commando.commands.query-dsl/command-resolve :query-B [_ {:keys [x QueryExpression]}] + (let [x (or x 10)] + (-> {:map {:a + {:b {:c x} + :d {:c (inc x) + :f (inc (inc x))}}} + :query-A (commando.commands.query-dsl/resolve-instruction-qe + "error" + {:commando/resolve :query-A + :x 1})} + (commando.commands.query-dsl/->query-run QueryExpression)))) + +(defmethod commando.commands.query-dsl/command-resolve :query-A [_ {:keys [x QueryExpression]}] + (let [x (or x 10)] + (-> {:map {:a + {:b {:c x} + :d {:c (inc x) + :f (inc (inc x))}}} + + :resolve-fn (commando.commands.query-dsl/resolve-fn + "error" + (fn [{:keys [x]}] + (let [y (or x 1) + range-y (if (< 10 y) 10 y)] + (for [z (range 0 range-y)] + {:a + {:b {:c (+ y z)} + :d {:c (inc (+ y z)) + :f (inc (inc (+ y z)))}}})))) + + + + :instruction-A (commando.commands.query-dsl/resolve-instruction + "error" + {:commando/fn (fn [& [y]] + {:a + {:b {:c y} + :d {:c (inc y) + :f (inc (inc y))}}}) + :args [x]}) + + + :query-A (commando.commands.query-dsl/resolve-instruction-qe + "error" + {:commando/resolve :query-A + :x 1}) + :query-B (commando.commands.query-dsl/resolve-instruction-qe + "error" + {:commando/resolve :query-B + :x 1})} + (commando.commands.query-dsl/->query-run QueryExpression)))) + +(defn run-execute-in-depth-with-using-queryDSL [] + (println "\n===================Benchmark=====================") + (println "Run commando/execute in depth with using queryDSL") + (println "=================================================") + (execute-with-flame + [commando.commands.query-dsl/command-resolve-spec + commando.commands.builtin/command-from-spec + commando.commands.builtin/command-fn-spec] + {:commando/resolve :query-A + :x 1 + :QueryExpression + [{:map + [{:a + [:b]}]} + {:instruction-A [:a]} + {:query-A + [{:map + [{:a + [:b]}]} + {:query-A + [{:map + [{:a + [:b]}]} + {:query-A + [{:map + [{:a + [:b]}]}]}]}]} + {:query-B + [{:map + [{:a + [:b]}]} + {:query-A + [{:map + [{:a + [:b]}]} + {:query-A + [{:instruction-A [:a]}]}]}]}]}) +) + +;; ===================================== +;; BUILDING DEPENDECY COMPLEX TEST CASES +;; ===================================== + +(defn instruction-build-v+m [{:keys [wide-n long-n]}] + {:dependecy-token (* 2 wide-n long-n) + :source-maps + (mapv (fn [_n] + (into {} (mapv (fn [v] [(keyword (str "k" v)) v]) + (range 1 wide-n)))) + (range 1 long-n)) + :result-maps + (mapv (fn [n] + (into {} + (mapv + (fn [v] + (let [k (keyword (str "k" v))] + [k {:commando/from [:source-maps n k]}])) + (range 1 wide-n)))) + (range 1 long-n))}) + +(defn instruction-build-m [{:keys [wide-n long-n]}] + {:dependecy-token (* 2 wide-n long-n) + :source-maps + (reduce (fn [acc n] + (assoc acc (keyword (str "r" n)) + (into {} (mapv (fn [v] [(keyword (str "k" v)) v]) + (range 1 wide-n))))) + {} + (range 1 long-n)) + :result-maps + (reduce (fn [acc n] + (assoc acc (keyword (str "r" n)) + (into {} + (mapv + (fn [v] + (let [k (keyword (str "k" v))] + [k {:commando/from [:source-maps (keyword (str "r" n)) k]}])) + (range 1 wide-n))))) + {} + (range 1 long-n))}) + +(defn execute-complexity [{:keys [mode wide-n long-n]}] + (let [instruction-builder (case mode + :m (instruction-build-m {:wide-n wide-n :long-n long-n}) + :v+m (instruction-build-v+m {:wide-n wide-n :long-n long-n}))] + (binding [commando.impl.utils/*execute-config* + {:debug-result true}] + (let [result (commando.core/execute + [commando.commands.builtin/command-from-spec] + instruction-builder) + stats-grouped (reduce (fn [acc [k v label]] + (assoc acc k v)) + {} + (:stats result))] + {:dependecy-token (:dependecy-token instruction-builder) + :stats (:stats result) + :stats-grouped stats-grouped})))) + +;; ================================ +;; PLOT LOAD TEST CASES IN PNG FILE +;; WITH USING JFREECHART +;; ================================ + +(defn ^:private chat-custom-styles [chart] + (let [plotObject (.getPlot chart) + plotObjectRenderer (.getRenderer plotObject)] + (.setBackgroundPaint chart (java.awt.Color/new 255, 255, 255)) + (.setBackgroundPaint plotObject (java.awt.Color/new 255, 255, 255)) + (.setSeriesPaint plotObjectRenderer 0 (java.awt.Color/new 64, 115, 62)) + (.setSeriesPaint plotObjectRenderer 1 (java.awt.Color/new 62, 65, 115)) + (.setSeriesPaint plotObjectRenderer 2 (java.awt.Color/new 115, 94, 62)) + (.setSeriesPaint plotObjectRenderer 3 (java.awt.Color/new 115, 62, 62)) + (.setOutlineVisible plotObject false) + chart)) + +(defn execute-steps-grow_s_x_dep [] + (println "\n==================Benchmark====================") + (println "execute-steps(massive dep grow) secs_x_deps.png") + (println "===============================================") + (let [instruction-stats-result [(execute-complexity {:mode :v+m :wide-n 50 :long-n 50}) + (execute-complexity {:mode :v+m :wide-n 50 :long-n 500}) + (execute-complexity {:mode :v+m :wide-n 50 :long-n 5000}) + (execute-complexity {:mode :v+m :wide-n 50 :long-n 50000})] + chart-data (mapv (fn [e] (let [{:keys [dependecy-token stats-grouped]} e] + (-> stats-grouped + (dissoc "execute") + (update-vals (fn [nanosecs-t] + ;; (/ nanosecs-t 1000000) ;; miliseconds + (/ nanosecs-t 1000000000) ;; seconds + )) + (assoc "dependecy-token" dependecy-token)))) + instruction-stats-result)] + (doseq [{:keys [dependecy-token stats]} instruction-stats-result] + (print-stats {:stats stats} (str "Dependency Counts: " dependecy-token))) + (cljfreechart/save-chart-as-file + (-> chart-data + (cljfreechart/make-category-dataset {:group-key "dependecy-token"}) + (cljfreechart/make-bar-chart "commando.core/execute steps on massive count of dependencies" + {:category-title "Dependency Counts" + :value-title "Seconds"}) + (chat-custom-styles)) + "./test/perf/commando/execute-steps(massive dep grow) secs_x_deps.png" {:width 1200 :height 400}))) + +(defn execute-steps-normal_ms_x_dep [] + (println "\n================Benchmark================") + (println "execute-steps(normal) milisecs_x_deps.png") + (println "=========================================") + (let [instruction-stats-result + [(execute-complexity {:mode :m :wide-n 5 :long-n 10}) + (execute-complexity {:mode :m :wide-n 5 :long-n 14}) + (execute-complexity {:mode :m :wide-n 5 :long-n 15}) + (execute-complexity {:mode :m :wide-n 5 :long-n 20})] + chart-data (mapv (fn [e] (let [{:keys [dependecy-token stats-grouped]} e] + (-> stats-grouped + (dissoc "execute") + (update-vals (fn [nanosecs-t] + (/ nanosecs-t 1000000) ;; miliseconds + ;; (/ nanosecs-t 1000000000) ;; seconds + )) + (assoc "dependecy-token" dependecy-token)))) + instruction-stats-result)] + (doseq [{:keys [dependecy-token stats]} instruction-stats-result] + (print-stats {:stats stats} (str "Dependency Counts: " dependecy-token))) + (cljfreechart/save-chart-as-file + (-> chart-data + (cljfreechart/make-category-dataset {:group-key "dependecy-token"}) + (cljfreechart/make-bar-chart "commando.core/execute steps" + {:category-title "Dependency Counts" + :value-title "Miliseconds"}) + (chat-custom-styles)) + "./test/perf/commando/execute-steps(normal) milisecs_x_deps.png" {:width 1200 :height 400}))) + +(defn execute-normal_ms_x_dep [] + (println "\n=============Benchmark=============") + (println "execute(normal) milisecs_x_deps.png") + (println "===================================") + (let [instruction-stats-result + [(execute-complexity {:mode :v+m :wide-n 25 :long-n 25}) + (execute-complexity {:mode :v+m :wide-n 50 :long-n 50}) + (execute-complexity {:mode :v+m :wide-n 100 :long-n 100}) + (execute-complexity {:mode :v+m :wide-n 200 :long-n 200})] + chart-data (mapv (fn [e] (let [{:keys [dependecy-token stats-grouped]} e] + (-> stats-grouped + (select-keys ["execute"]) + (update-vals (fn [nanosecs-t] + (float (/ nanosecs-t 1000000)) ;; miliseconds + ;; (float (/ nanosecs-t 1000000000)) ;; seconds + )) + (assoc "dependecy-token" dependecy-token)))) + instruction-stats-result)] + (doseq [{:keys [dependecy-token stats]} instruction-stats-result] + (print-stats {:stats stats} (str "Dependency Counts: " dependecy-token))) + (cljfreechart/save-chart-as-file + (-> chart-data + (cljfreechart/make-category-dataset {:group-key "dependecy-token"}) + (cljfreechart/make-bar-chart "commando.core/execute times" + {:category-title "Dependency Counts" + :value-title "Miliseconds"}) + (chat-custom-styles)) + "./test/perf/commando/execute(normal) milisecs_x_deps.png" {:width 1200 :height 400}))) + +(defn -main [] + ;; Execution stats. + (real-word-calculation-average-of-50) + (run-execute-in-depth-with-using-queryDSL) + ;; Drow plot for special cases. + (execute-steps-normal_ms_x_dep) + (execute-normal_ms_x_dep) + (execute-steps-grow_s_x_dep)) -(comment - ;; REPL usage examples: - ;; Run individual benchmarks - (perf-simple-command) - (perf-dependency-chain) - (perf-medium-parallel) - ;; Run quick performance check - (run-quick-benchmarks) - ;; Run complete benchmark suite (takes longer) - (run-all-benchmarks) - ;; Compare specific scenarios - (perf-registry-vs-compiled) - ;; Test edge cases - (perf-empty-instruction) - (perf-dependent-chain) - (perf-wide-fanout) - (perf-deep-nested-instruction)) diff --git a/test/perf/commando/execute(normal) milisecs_x_deps.png b/test/perf/commando/execute(normal) milisecs_x_deps.png new file mode 100644 index 0000000..4f28981 --- /dev/null +++ b/test/perf/commando/execute(normal) milisecs_x_deps.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c95740b0ee83130fb35d576357fcc8912fc9f4223e284f2faa87d8f5a47e3dbc +size 20366 diff --git a/test/perf/commando/execute-steps(massive dep grow) secs_x_deps.png b/test/perf/commando/execute-steps(massive dep grow) secs_x_deps.png new file mode 100644 index 0000000..df05d85 --- /dev/null +++ b/test/perf/commando/execute-steps(massive dep grow) secs_x_deps.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6114d208235d618bb388771c635eae98ffed6b1234887366b5463382bca8f1bd +size 18310 diff --git a/test/perf/commando/execute-steps(normal) milisecs_x_deps.png b/test/perf/commando/execute-steps(normal) milisecs_x_deps.png new file mode 100644 index 0000000..0979830 --- /dev/null +++ b/test/perf/commando/execute-steps(normal) milisecs_x_deps.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a12d26057edc1fb0406047f40774167eea092625014ea74f13c92127ab170f7 +size 21889 diff --git a/test/unit/commando/impl/utils_test.cljc b/test/unit/commando/impl/utils_test.cljc index 11c7f45..6b159ec 100644 --- a/test/unit/commando/impl/utils_test.cljc +++ b/test/unit/commando/impl/utils_test.cljc @@ -202,3 +202,4 @@ (is (= false (malli/validate sut/ResolvableFn 'UNKOWN/UNKOWN))) ))) +