diff --git a/di/grafana/LIVE_TESTING.md b/di/grafana/LIVE_TESTING.md new file mode 100644 index 00000000..027da2ba --- /dev/null +++ b/di/grafana/LIVE_TESTING.md @@ -0,0 +1,196 @@ +# di.grafana - Live verification (manual) + +An end-to-end check of `di.grafana` against a **real** Grafana instance: browser → +Grafana → HTTP (with the `X-Grafana-Org-Id` header) → the module's `.z.pp`/`.z.ph` +handlers → `search`/`query` → JSON → a rendered panel. + +This is a **manual** procedure for confidence/demos - it is **not** part of the +automated test suite (`k4unit.moduletest\`di.grafana`), which already covers the +module's logic in-process. Use this to see it work in the real UI. + +> **Verified working:** 2026-07-01 - Grafana **10.4.14** + `grafana-simple-json-datasource` +> v1.4.2, KDB-X 5.0, module on `di.grafana-pr`. Live panel rendered a moving +> per-sym price series driven by a kdb feed. + +--- + +## ⚠️ Grafana version compatibility (read this first) + +The module implements the **classic Grafana SimpleJSON protocol** - `/search`, +`/query`, `/annotations`, and a `GET /` health check. The only Grafana datasource +plugin that speaks this protocol natively is **`grafana-simple-json-datasource`**, +which is **Angular-based and deprecated**. Grafana has been phasing Angular out: + +| Grafana version | Status of the plugin | +|---|---| +| **≤ 10.x** | Angular enabled by default - plugin works out of the box ✅ | +| **11.x** | Angular disabled by default - plugin loads **only** if Angular is re-enabled (`[plugins] angular_support_enabled = true`, or env `GF_PLUGINS_ANGULAR_SUPPORT_ENABLED=true`) ⚠️ | +| **≥ 12.x** | Angular support **removed entirely** - the plugin will not load at all ❌ | + +**This guide pins Grafana 10.4.14** because Angular is on by default there - the +least friction. On Grafana 11 you must re-enable Angular (see Troubleshooting). + +### Future note - when you must move to Grafana ≥ 12 + +`grafana-simple-json-datasource` is a dead end from Grafana 12 onward. Options then: + +- **Infinity** (`yesoreyeram-infinity-datasource`, React, actively maintained): + point it at `POST http://:/query` with a JSON body and a custom + `X-Grafana-Org-Id: 1` header, and configure its parser to read the `datapoints`. + Works, but it's manual per-panel config - no metric dropdown from `/search`. +- **SimPod** (`simpod-json-datasource`, React): non-Angular, but newer versions + call `/metrics` instead of `/search`, so the metric dropdown won't populate + (the module serves `/search`); a manually-typed target may still hit `/query`. +- **Extend the module** to also serve the endpoints a modern React plugin expects + (e.g. `/metrics`) - a code change; weigh against keeping the module lean. + +The module itself is protocol-correct; this is purely about which Grafana plugin +can consume it. If this doc stops working after a Grafana upgrade, the cause is +almost certainly the Angular removal above, not the module. + +--- + +## Overview + +Two terminals plus a browser. Ports used below: **kdb HTTP = `6702`**, **Grafana +UI = `3000`** (change either if busy). + +--- + +## 1. kdb process (Terminal 1) + +Start kdb-x from the module repo so `use\`di.grafana` resolves, then wire it up as +a datasource with a small live feed: + +```bash +cd ~/kdb_x/kdbx-modules-memstats-rename +export QPATH=$(pwd):$QPATH +q # your KDB-X launcher +``` + +```q +\p 6702 / listen for HTTP + +/ a logger that prints, so incoming requests are visible +mylog:`info`warn`error!( + {[c;m] -1 "INFO [",string[c],"] ",m;}; + {[c;m] -1 "WARN [",string[c],"] ",m;}; + {[c;m] -2 "ERROR [",string[c],"] ",m;}); + +grafana:use`di.grafana +grafana.init[enlist[`log]!enlist mylog] + +/ ~16 min of recent data + a 1/sec live feed so the graph moves +trade:([]time:.z.p-0D00:00:01*til 1000;sym:1000?`AAPL`MSFT`GOOG;price:1000?100f;size:1000?500) +.z.ts:{`trade insert (.z.p;rand`AAPL`MSFT`GOOG;rand 100f;rand 500)}; system"t 1000" +``` + +Leave this session running. (`.z.ts` is yours; the module only owns `.z.pp`/`.z.ph`.) + +--- + +## 2. Grafana 10.4 (Terminal 2, no root required) + +Runs entirely from your home directory - no Docker, no `sudo`: + +```bash +cd ~ +wget https://dl.grafana.com/oss/release/grafana-10.4.14.linux-amd64.tar.gz +tar -zxf grafana-10.4.14.linux-amd64.tar.gz +cd grafana-v10.4.14 +mkdir -p data/plugins + +# install the SimpleJson plugin into a local (writable) dir +./bin/grafana-cli --pluginsDir "$PWD/data/plugins" plugins install grafana-simple-json-datasource + +# run it, all paths under $HOME +GF_SERVER_HTTP_PORT=3000 \ +GF_PATHS_DATA="$PWD/data" \ +GF_PATHS_LOGS="$PWD/data/log" \ +GF_PATHS_PLUGINS="$PWD/data/plugins" \ +GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=grafana-simple-json-datasource \ +./bin/grafana-server +``` + +Wait for both of these in the log (and **no** "Refusing to initialize … Angular"): + +``` +Plugin registered … pluginId=grafana-simple-json-datasource +HTTP Server Listen … :3000 +``` + +--- + +## 3. If Grafana is on a remote server (SSH / VS Code Remote) + +Grafana listens on the **server's** `localhost:3000`, which your local browser +can't reach directly. Forward the UI port: + +- **VS Code Remote:** open the **PORTS** panel → **Forward a Port** → `3000` → + open the resulting local address. +- **Plain SSH:** `ssh -L 3000:localhost:3000 @` then browse `http://localhost:3000`. + +Only the **Grafana UI port** needs forwarding. The datasource URL +`http://localhost:6702` is resolved by Grafana's **backend on the server** (that's +what "Access: Server" means), so kdb's port never leaves the server - do **not** +forward 6702. + +--- + +## 4. Add the datasource + +1. `http://localhost:3000` → login **admin / admin** (skip the password prompt). +2. **Connections → Data sources → Add data source →** search **"SimpleJson"** → select it. +3. **URL:** `http://localhost:6702` · **Access: Server (default)**. + - **Access must be Server** - that is what makes Grafana's backend attach the + `X-Grafana-Org-Id` header the module keys on. +4. **Save & Test** → expect green **"Data source is working"** (a `GET /` → `.z.ph` + → `200 OK`; visible in the q console). + +--- + +## 5. Build a panel + +1. **Dashboards → New → New dashboard → + Add visualization →** pick the datasource. +2. Open the **Metric** dropdown (populated by `/search`): `trade`, `t.trade`, + `g.trade`, `g.trade.price`, `g.trade.size`, `t.trade.AAPL`, … +3. Select **`g.trade.price`**; set the time range to **Last 15 minutes** + (optionally auto-refresh 5s to watch the live feed). + +Exercise the other paths too: +- **`g.trade`** - one line per numeric column +- **`t.trade`** - switch the panel type to **Table** for the whole table +- **`t.trade.AAPL`** - table filtered to one sym + +--- + +## What success looks like + +| Signal | Confirms | +|---|---| +| **Save & Test** green | GET handler + connectivity | +| Metric **dropdown populates** | `/search` | +| Panel shows **live lines** (AAPL/MSFT/GOOG) | `/query` → `tsfunc` → per-sym builder, end-to-end | +| `INFO [grafana] received … request` in Terminal 1 | requests flowing through `.z.pp` | + +--- + +## Teardown + +- Grafana: **Ctrl-C** in Terminal 2. +- kdb: `system"t 0"` to stop the feed, then exit q. + +--- + +## Troubleshooting + +| Symptom | Cause / fix | +|---|---| +| `Refusing to initialize plugin … Angular` in the Grafana log | You're on Grafana 11 - use the 10.4 tarball here, or set `GF_PLUGINS_ANGULAR_SUPPORT_ENABLED=true` (config `[plugins] angular_support_enabled = true`). On Grafana ≥ 12 it can't be enabled - see "Future note" above. | +| `docker: permission denied … docker.sock` | Not in the `docker` group. Use the **binary** method here (no Docker), or `sudo docker …`. | +| `bind: address already in use` on start | UI port taken - set `GF_SERVER_HTTP_PORT` to a free port (e.g. `6905`). | +| Plugin install: `permission denied … /var/lib/grafana/plugins` | The CLI defaulted to the system dir - pass `--pluginsDir "$PWD/data/plugins"`. | +| Browser: page won't load (remote server) | Forward the UI port over SSH - see section 3. | +| **Save & Test: "Bad Gateway"** | Grafana's backend couldn't reach/parse kdb. From the server: `curl -i -H "X-Grafana-Org-Id: 1" http://localhost:6702/` - `Connection refused` means the kdb session isn't listening on 6702 (restart it); anything non-`200` means the wrong handler is wired. | +| Panel empty / "No data" | Time range doesn't overlap the data - use **Last 15 minutes** (data is `.z.p`-recent). | +| Metric dropdown empty | Wrong plugin - must be **SimpleJson** (`/search`+`/query`), not the newer SimPod (`/metrics`). | diff --git a/di/grafana/grafana.md b/di/grafana/grafana.md new file mode 100644 index 00000000..2ec9e366 --- /dev/null +++ b/di/grafana/grafana.md @@ -0,0 +1,189 @@ +# di.grafana + +Grafana JSON datasource adaptor for kdb+. Implements the +[SimPod JSON datasource](https://github.com/simPod/GrafanaJsonDatasource) API so +a Grafana server can query a kdb+ process directly: the module installs HTTP +handlers that answer Grafana's `/search` and `/query` requests with the JSON it +expects, serving the process's tables as timeseries and table panels. + +--- + +## Features + +- Serves the `/search` endpoint - populates Grafana's metric dropdowns from the tables in the process +- Serves the `/query` endpoint - returns either timeseries or table panel data +- Answers the Grafana test-connection `GET` with a `200 OK` +- Wraps any pre-existing `.z.pp`/`.z.ph` handlers rather than overwriting them, so other HTTP endpoints keep working +- Accepts a `kx.log` instance directly, or a plain `info`warn`error` dict +- Loads standalone - no hard dependencies on other modules + +--- + +## Dependencies + +| Dependency | Key | Required | Description | +|---|---|---|---| +| logger | `` `log `` | yes | `info`, `warn`, and `error` - each a binary `{[c;m]}` where `c` is a symbol context and `m` is a string. All three are required because this module calls all three. | + +**Hard dependency:** none - the module loads standalone. + +The `log` dependency must be passed to `init` inside the `deps` dict keyed on +`` `log ``. The module throws immediately if `log` is absent, is not a +dictionary, or is missing any of `info`/`warn`/`error`. + +A `kx.log` instance can be passed directly - the module normalises its monadic +functions to the binary `{[c;m]}` contract automatically (detected by the +`getlvl`/`sinks`/`fmts` keys). Plain `info`warn`error` dicts pass through +unchanged. + +```q +/ plain custom logger +mylog:`info`warn`error!( + {[c;m] -1 "INFO [",string[c],"] ",m;}; + {[c;m] -1 "WARN [",string[c],"] ",m;}; + {[c;m] -2 "ERROR [",string[c],"] ",m;}); +grafana:use`di.grafana +grafana.init[enlist[`log]!enlist mylog] + +/ or a kx.log instance directly +kxlog:use`kx.log +grafana.init[enlist[`log]!enlist kxlog.createLog[]] +``` + +Configuration keys `timecol`, `sym`, `timebackdate`, `ticks`, `del`, and +`allowfunctions` are all optional - omit any or all of them and the module falls +back to sensible defaults. See Initialisation for full details. + +--- + +## Initialisation + +`init[deps]` takes a single dictionary combining the `log` dependency with any +configuration overrides. + +| Key | Required | Description | +|---|---|---| +| `` `log `` | yes | Log dep - `info`/`warn`/`error` `{[c;m]}` functions, or a `kx.log` instance | +| `` `timecol `` | no | Name of the time column used for timeseries queries. Default: `` `time `` | +| `` `sym `` | no | Name of the column used to split data by instrument. Default: `` `sym `` | +| `` `timebackdate `` | no | How far back to look when finding distinct syms for the dropdowns. Default: `2D` | +| `` `ticks `` | no | Number of rows returned for a table request. Default: `1000` | +| `` `del `` | no | Delimiter separating the arguments within a query target. Default: `"."` | +| `` `allowfunctions `` | no | Enable `f.` function targets, which evaluate arbitrary q (see Security note). Default: `0b` (disabled) | + +`init` must be called before the module will serve any requests. It validates +the logger, applies config, and installs the `.z.pp`/`.z.ph` handlers - once; +re-calling `init` updates the logger/config without re-wrapping the handlers. + +```q +/ defaults only +grafana.init[enlist[`log]!enlist logdep] + +/ with config overrides (config keys sit alongside `log) +grafana.init[`log`timecol`ticks!(logdep;`ts;500)] +``` + +--- + +## Exported Functions + +### `init[deps]` +Initialise the module: validate the log dependency, apply config, and install the HTTP handlers. Errors immediately if `log` is missing, is not a dict, or lacks `info`/`warn`/`error`. +```q +grafana.init[enlist[`log]!enlist logdep] +/ or with config: +grafana.init[`log`ticks!(logdep;500)] +``` + +### `getconfig[]` +Return the currently active configuration as a dictionary - useful for confirming how the running module is tuned. +```q +grafana.getconfig[] +/ `timecol`sym`timebackdate`ticks`del!(`time;`sym;2D;1000;".") +``` + +--- + +## Query target syntax + +The Grafana metric strings produced by `/search` encode the table, panel type, +and arguments, separated by `del` (default `"."`): + +| Target | Meaning | +|---|---| +| `t.` | table panel for the whole table | +| `t.
.` | table panel filtered to one sym | +| `g.
` | graph panel, one series per numeric column | +| `g.
.` | graph panel, one series per sym for a column | +| `o.
...` | "other" panels (single-stat, gauge, etc.) | +| `f.<...>` | the target is a q expression to evaluate rather than a table - **disabled unless `allowfunctions` is set** (see Security) | + +--- + +## Usage Example + +```q +/ load and initialise with a printing logger +mylog:`info`warn`error!( + {[c;m] -1 "INFO [",string[c],"] ",m;}; + {[c;m] -1 "WARN [",string[c],"] ",m;}; + {[c;m] -2 "ERROR [",string[c],"] ",m;}); + +grafana:use`di.grafana +grafana.init[enlist[`log]!enlist mylog] + +/ open an HTTP port and create some data with time + sym columns +\p 5000 +trade:([]time:.z.p-0D00:00:01*til 100;sym:100?`AAPL`MSFT`GOOG;price:100?100f) +``` + +Add a SimPod JSON datasource in Grafana pointing at the process's HTTP port, +then build panels using the metrics offered in the dropdowns (`t.trade`, +`g.trade.price`, ...). + +--- + +## Running Tests + +```q +k4unit:use`di.k4unit +k4unit.moduletest`di.grafana +``` + +The suite injects a no-op mock logger and covers: the export surface; dependency +validation (non-dict deps, missing `log` key, null log value, missing log keys); +handler wiring and the install-once `wired` guard; config defaults and overrides +via `getconfig`; the target-parsing helpers (`isfunc`/`istab`/`istype`/`prefix`); +`finddistinctsyms`/`memvals`/`catchvals`; the `/query` table and timeseries +builders (`tbfunc`, and `tsfunc` including the per-column and per-sym graph +paths); the live `.z.pp`/`.z.ph` routes including non-Grafana fall-through; and a +`kx.log` integration test confirming the `normlog` wrapping works end-to-end. + +--- + +## Security + +The module answers any HTTP request carrying the `X-Grafana-Org-Id` header, so +treat its port as a trust boundary - put it behind Grafana's authentication and +firewall it from untrusted networks. + +- **Table targets are resolved by name, never evaluated.** `t.`/`g.`/`o.` and + bare targets are looked up as symbols against `tables[]`; an unknown name is + rejected. A target string can therefore never be executed as q code through + these paths. +- **`f.` function targets evaluate arbitrary q and are disabled by default.** + They run only when `allowfunctions` is set to `1b` in `init`. Enable it solely + on processes where every client able to reach the port is trusted, since an + `f.` target is, by design, remote code execution. + +--- + +## Notes + +- The module installs `.z.pp`/`.z.ph` by closure-wrapping any pre-existing handler: Grafana requests (those carrying the `X-Grafana-Org-Id` header) are routed to the adaptor, and every other request falls through to the captured handler - so adding the module to a process that already serves HTTP does not break existing endpoints. This follows the `di.timer` precedent of a module owning a `.z.*` callback. +- The intended end-state is to register the handlers through an injected `di.handlers` dependency rather than assigning `.z.*` directly. That is deferred until `di.handlers` exists and supports return-yielding `.z.pp`/`.z.ph` chains - a side-effect-only chain cannot return the HTTP response. `di.handlers` is not implemented yet. +- A panel with multiple queries works: each target in the request is dispatched by its own type and the per-target results are merged into one JSON array +- Table panels return the last `ticks` rows (default 1000); raise `ticks` via config if a panel needs more +- Sym dropdowns list only syms seen within `timebackdate` (default 2 days) +- Data tables must expose the configured `timecol` and `sym` columns (defaults `` `time ``/`` `sym ``) +- The `/annotations` endpoint returns a "not yet implemented" marker, matching the TorQ original diff --git a/di/grafana/grafana.q b/di/grafana/grafana.q new file mode 100644 index 00000000..883bf4c3 --- /dev/null +++ b/di/grafana/grafana.q @@ -0,0 +1,317 @@ +/ grafana json datasource adaptor for kdb-x +/ implements the simpod json datasource api so a grafana server can query +/ kdb+ tables and timeseries data over http - the /search endpoint populates +/ the grafana dropdowns and the /query endpoint returns timeseries or table data + +/ ----------------------------------------------------------------------------- +/ configuration - load-time defaults; overridden via the deps dict in init and +/ read back through .z.m at every call site +/ ----------------------------------------------------------------------------- + +/ name of the time column used for timeseries queries +timecol:`time; +/ name of the sym column used to split data by instrument +sym:`sym; +/ how far back to look when finding distinct syms +timebackdate:2D; +/ number of ticks to return for table requests +ticks:1000; +/ delimiter separating the arguments within a query target +del:"."; +/ allow f. function targets to evaluate arbitrary q (off by default - see grafana.md) +allowfunctions:0b; + +/ json type for each kdb datatype, keyed by .Q.t character +types:.Q.t!`array`boolean,(3#`null),(5#`number),11#`string; +/ milliseconds between 1970.01.01 and 2000.01.01 +epoch:946684800000; + +/ internal flag - set true once the http handlers have been installed +wired:0b; + +/ ----------------------------------------------------------------------------- +/ request handling +/ ----------------------------------------------------------------------------- + +zpp:{[x] + / parse a grafana http post request and dispatch to the matching handler + / kdb passes .z.pp (requeststring;headers); requeststring is " " + / (method and leading / already stripped), so cut at the first space gives + / (endpoint;body) - e.g. "query {...}" -> ("query";"{...}") + r:(0;n?" ")cut n:first x; + rqt:.j.k r 1; + .z.m.log[`info][`grafana;"received ",(r 0)," request"]; + handler:$["query"~r 0;query;"search"~r 0;search;annotation]; + :.[handler;enlist rqt;{[e].z.m.log[`error][`grafana;"failed to process request: ",e];'e}]; + }; + +annotation:{[rqt] + / annotation endpoint is not yet implemented + .z.m.log[`warn][`grafana;"annotation url not yet implemented"]; + :`$"Annotation url nyi"; + }; + +query:{[rqt] + / dispatch each target to the timeseries or table builder by its own type and + / merge the per-target json arrays - grafana can send several targets at once + tgts:rqt`targets; + / a single target may arrive as a bare dict; normalise to a list of targets + tgts:$[99h=type tgts;enlist tgts;tgts]; + / iterate by index so a list or a table of targets is handled the same way + build:{[rqt;tgts;i]t:tgts i;r:@[rqt;`targets;:;t];$[t[`type]~"timeserie";tsfunc r;tbfunc r]}; + / strip the [ ] from each per-target array, drop empties, re-wrap as one array + inners:{1_-1_x}each build[rqt;tgts]each til count tgts; + inners:inners where 0;.z.m.timecol;(-;.z.p;.z.m.timebackdate));1b;{x!x}enlist .z.m.sym] .z.m.sym; + }; + +prefix:{[c;s] + / prefix string c and the delimiter to each string in s + :(c,.z.m.del),/:s; + }; + +/ ----------------------------------------------------------------------------- +/ fetching the last n ticks +/ ----------------------------------------------------------------------------- + +diskvals:{[x] + / last `ticks` rows of an on-disk partitioned table + c:(count[x]-.z.m.ticks)+til .z.m.ticks; + :get'[.Q.ind[x;c]]; + }; + +memvals:{[x] + / last `ticks` rows of an in-memory table + :get'[?[x;enlist(within;`i;count[x]-.z.m.ticks,0);0b;()]]; + }; + +catchvals:{[x] + / try the on-disk path first, fall back to the in-memory path on error + :@[diskvals;x;{[x;y]memvals x}[x]]; + }; + +/ ----------------------------------------------------------------------------- +/ target parsing helpers +/ ----------------------------------------------------------------------------- + +istype:{[targ;char] + / test whether the target is prefixed with char followed by the delimiter + :(char,.z.m.del)~2#targ; + }; +isfunc:istype[;"f"]; +istab:istype[;"t"]; + +resolvetab:{[t] + / resolve a table-name symbol to its unkeyed table, rejecting anything that is + / not a known table - the name is looked up, never evaluated as code + if[not t in tables[]; + .z.m.log[`error][`grafana;"unknown table: ",string t]; + '"di.grafana: unknown table ",string t]; + :0!value t; + }; + +evalfunc:{[s] + / evaluate an f. function-target expression - gated behind allowfunctions + / because it executes arbitrary q (disabled by default) + if[not .z.m.allowfunctions; + .z.m.log[`error][`grafana;"function target rejected; allowfunctions is disabled"]; + '"di.grafana: function targets are disabled (set allowfunctions to enable)"]; + :value s; + }; + +/ ----------------------------------------------------------------------------- +/ building json responses +/ ----------------------------------------------------------------------------- + +tabresponse:{[colname;coltype;rqt] + / build a table response in the json datasource schema + :.j.j enlist`columns`rows`type!(flip`text`type!(colname;coltype);catchvals rqt;`table); + }; + +tbfunc:{[rqt] + / process a table request and return the json datasource table response + rqt:raze rqt[`targets]`target; + symname:0b; + / f. targets evaluate a q expression (opt-in); t./bare targets resolve a known + / table by name and are never evaluated as code + rqt:$[isfunc rqt;0!evalfunc $[istab 2_rqt;4_rqt;2_rqt]; + istab rqt;[parts:`$.z.m.del vs rqt;if[2meta[rqt][.z.m.timecol;`t];rqt:@[rqt;.z.m.timecol;+;.z.D]]; + / restrict to the time range requested by grafana - grafana sends iso-8601 utc; + / strip a trailing Z only if present rather than blindly dropping the last char + range:"P"${$["Z"=last x;-1_x;x]}each x[`range]`from`to; + rqt:?[rqt;enlist(within;.z.m.timecol;range);0b;()]; + / add the milliseconds-since-epoch column grafana expects + rqt:@[rqt;`msec;:;mil rqt .z.m.timecol]; + / dispatch on the number of arguments and the panel type + $[(2type deps;'"di.grafana: deps must be a dict with `log key"]; + if[not`log in key deps;'"di.grafana: log dependency is required; pass `info`warn`error functions keyed on `log"]; + if[99h<>type deps`log;'"di.grafana: log value must be a dict; pass `info`warn`error functions"]; + / normalise a kx.log instance into the binary {[c;m]} contract; plain dicts pass through + lg:normlog deps`log; + if[not all`info`warn`error in key lg;'"di.grafana: log dict must have `info`warn`error keys; got: ",", " sv string key lg]; + .z.m.log:lg; + / overlay any supplied config - config values sit alongside `log in deps + if[`timecol in key deps;.z.m.timecol:deps`timecol]; + if[`sym in key deps;.z.m.sym:deps`sym]; + if[`timebackdate in key deps;.z.m.timebackdate:deps`timebackdate]; + if[`ticks in key deps;.z.m.ticks:deps`ticks]; + if[`del in key deps;.z.m.del:deps`del]; + if[`allowfunctions in key deps;.z.m.allowfunctions:deps`allowfunctions]; + / install the http handlers once, preserving any existing definitions + if[not .z.m.wired;sethandlers[]]; + .z.m.log[`info][`grafana;"initialised grafana json datasource adaptor"]; + }; + +getconfig:{ + / return the currently active configuration + :`timecol`sym`timebackdate`ticks`del`allowfunctions!(.z.m.timecol;.z.m.sym;.z.m.timebackdate;.z.m.ticks;.z.m.del;.z.m.allowfunctions); + }; diff --git a/di/grafana/init.q b/di/grafana/init.q new file mode 100644 index 00000000..daaae423 --- /dev/null +++ b/di/grafana/init.q @@ -0,0 +1,3 @@ +\l ::grafana.q + +export:([init;getconfig]) diff --git a/di/grafana/test.csv b/di/grafana/test.csv new file mode 100644 index 00000000..0c13aabf --- /dev/null +++ b/di/grafana/test.csv @@ -0,0 +1,92 @@ +action,ms,bytes,lang,code,repeat,minver,comment +comment,,,,,,,Setup +before,0,0,q,grafana:use`di.grafana,1,,load module into session +before,0,0,q,mocklog:`info`warn`error!({[c;m]};{[c;m]};{[c;m]}),1,,mock logger for injection + +comment,,,,,,,Export surface and required dependency +true,0,0,q,(asc key grafana)~`getconfig`init,1,,only init and getconfig are exported +fail,0,0,q,grafana.init[(::)],1,,init errors when no log dependency is supplied +fail,0,0,q,grafana.init[enlist[`log]!enlist(::)],1,,init errors when log dependency is null +fail,0,0,q,grafana.init[enlist[`log]!enlist `info`warn!({[c;m]};{[c;m]})],1,,init errors when log dict is missing keys + +comment,,,,,,,Initialisation wires handlers and applies defaults +run,0,0,q,grafana.init[enlist[`log]!enlist mocklog],1,,initialise with mock logger +true,0,0,q,.m.di.0grafana.wired,1,,handlers wired after init +true,0,0,q,1000=.m.di.0grafana.ticks,1,,ticks default applied +true,0,0,q,(`timecol`sym`timebackdate`ticks`del`allowfunctions)~key grafana.getconfig[],1,,getconfig exposes the config keys + +comment,,,,,,,Target parsing helpers +true,0,0,q,.m.di.0grafana.isfunc "f.x",1,,isfunc detects function prefix +true,0,0,q,.m.di.0grafana.istab "t.x",1,,istab detects table prefix +true,0,0,q,not .m.di.0grafana.isfunc "t.x",1,,isfunc rejects non-function prefix +true,0,0,q,.m.di.0grafana.istype["o.x";"o"],1,,istype matches arbitrary prefix +true,0,0,q,(("t.trade";"t.quote"))~.m.di.0grafana.prefix["t";("trade";"quote")],1,,prefix prepends char and delimiter + +comment,,,,,,,Build a recent test table in root +run,0,0,q,trade:([]time:.z.p-0D00:00:01*til 10;sym:10#`a`b;price:10?100f),1,,create test trade table with recent timestamps +true,0,0,q,(`a`b)~asc .m.di.0grafana.finddistinctsyms trade,1,,distinct syms found within lookback +true,0,0,q,10=count .m.di.0grafana.memvals trade,1,,memvals returns last ticks rows from memory +true,0,0,q,0