Skip to content

New basic logging module using kx.log as reference#90

Open
DI-Dexter wants to merge 3 commits into
DataIntellectTech:mainfrom
DI-Dexter:loggingModule
Open

New basic logging module using kx.log as reference#90
DI-Dexter wants to merge 3 commits into
DataIntellectTech:mainfrom
DI-Dexter:loggingModule

Conversation

@DI-Dexter

Copy link
Copy Markdown
Contributor

Log

log.q is the default logging implementation for di.* modules. It writes formatted lines to stdout and satisfies the log dependency contract expected by modules such as di.email.

Usage

log:use`di.log
log.trace[`mymodule;"entering function"]
log.debug[`mymodule;"value is 42"]
log.info[`mymodule;"starting up"]
log.warn[`mymodule;"disk usage above 80%"]
log.error[`mymodule;"connection failed"]
log.fatal[`mymodule;"unrecoverable error, shutting down"]

Output format:

2026-04-09T12:00:00.000000000 [TRACE] [mymodule] entering function
2026-04-09T12:00:00.001000000 [DEBUG] [mymodule] value is 42
2026-04-09T12:00:00.002000000 [INFO] [mymodule] starting up
2026-04-09T12:00:00.003000000 [WARN] [mymodule] disk usage above 80%
2026-04-09T12:00:00.004000000 [ERROR] [mymodule] connection failed
2026-04-09T12:00:00.005000000 [FATAL] [mymodule] unrecoverable error, shutting down

Injecting into other modules

All di.* modules that accept a log dependency expect a dictionary with keys `info`warn`error, each a function with signature {[ctx;msg]}.

log:use`di.log
logdep:`info`warn`error!(log.info;log.warn;log.error)

email:use`di.email
email.init[emailconfig;`log`send!(logdep;::)]

You can extend the injected dictionary with trace, debug, and fatal for modules that support them:

logdep:`trace`debug`info`warn`error`fatal!(log.trace;log.debug;log.info;log.warn;log.error;log.fatal)

createLog

createLog is a factory that returns an independent logger instance with level filtering, multiple output sinks, and configurable format templates. Each call to createLog produces a separate instance with its own state.

log:use`di.log
mylog:log.createLog[]

mylog.setlvl `warn                        / suppress trace, debug, info
mylog.info "this is suppressed"           / returns () silently
mylog.warn "this appears"

mylog.setfmt `syslog                      / switch to syslog format
mylog.addfmt[`compact;"$l $m"]           / add a custom format
mylog.setfmt `compact

mylog.add[2i;`error`fatal]               / add stderr sink for error and fatal
mylog.remove[1i;`trace]                  / remove stdout from trace level

Sinks

A sink is a handle (integer file descriptor or function) passed to add. If a function is provided it is called with the formatted line string. Built-in handles follow standard q conventions: 1i is stdout, 2i is stderr.

buf:();
capture:{[msg] buf,:enlist msg};         / function sink - captures output
mylog.add[capture;`info`warn`error]
mylog.info "captured"
buf                                      / ("2026-...captured\n")
mylog.remove[capture;`info]

Format templates

Three built-in formats are available:

Name Template Example output
basic (default) $p $l PID[$i] HOST[$h] $m 2026-04-09T12:00:00.000000000 INFO PID[1234] HOST[myhost] message
syslog <$s> $m <6> message
raw $m message

Template variables: $p timestamp, $l level, $i PID, $h hostname, $m message, $s syslog severity number.

API

trace

Parameters: [ctx; msg]

Write a trace-level message to stdout.

  • ctx — symbol context tag (e.g. `mymodule)
  • msg — string message

debug

Parameters: [ctx; msg]

Write a debug-level message to stdout.

info

Parameters: [ctx; msg]

Write an info-level message to stdout.

warn

Parameters: [ctx; msg]

Write a warning-level message to stdout.

error

Parameters: [ctx; msg]

Write an error-level message to stdout.

fatal

Parameters: [ctx; msg]

Write a fatal-level message to stdout.

createLog

Parameters: none

Returns an independent logger instance as a dictionary of functions. Each call returns a new instance with isolated state.

Returned keys: `trace`debug`info`warn`error`fatal`add`remove`setfmt`getfmt`addfmt`setlvl`getlvl

Function Parameters Description
trace..fatal [msg] Write a message at the given level (filtered by active level)
setlvl [lvl] Set minimum level; one of `trace`debug`info`warn`error`fatal
getlvl [_] Return current minimum level
setfmt [name] Switch to a named format template
getfmt [_] Return current format name
addfmt [name;template] Register a new named format template
add [handle;lvls] Add a sink for one or more levels; returns the handle
remove [handle;lvl] Remove a sink from a level

Log dependency contract

The log dependency contract used across di.* modules requires a dictionary:

`info`warn`error!({[ctx;msg] ...};{[ctx;msg] ...};{[ctx;msg] ...})

di.log satisfies this contract. You can also supply any custom implementation with the same signatures.

Comment thread di/log/log.q
{[x;y] string .z.h};
{[x;y] y};
{[x;y] string .z.M.syslogLvl`$lower x};
{[x;y] "$"});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.z.M.syslogLvl references the module namespace dictionary's syslogLvl key, but syslogLvl is a plain name defined at the top level of the file (line 12). At the time the pattern dict is evaluated, .z.M may not yet be set (it is typically assigned by the module loader after the file is loaded), so this lookup will fail with a rank/type error or silently return null. Access the file-level name directly as syslogLvl instead of .z.M.syslogLvl.

Comment thread di/log/log.q
{[id;gv;lvl;msg]
if[(lvls?lvl)<lvls?gv`lvl;:()];
txt:fmtapply[gv`prep;upper string lvl;fmtmsg msg],nl;
{[txt;pair]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error handler in makelevel swallows all write errors silently: @[last pair;txt;{x}] catches any signal and returns the error string unnoticed. A failed write to a closed handle or a function sink that signals will be silently discarded with no indication of failure. Use a handler that at minimum writes to stderr, e.g. {-2 "log sink error: ",x}.

Comment thread di/log/log.q
part:parts[1+i];
code:first part;
rest:1_part;
if[not code in key subs;'`$"unsupported format char: ",enlist code];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fmtmsg signals `type for a list message whose first element is not a 10h/string argument type (e.g. a symbol format string like (`fmt;arg) where fmt is a symbol), but only after partially processing. More critically, the args indexing args[i] will silently return null when i is out of range (more %s/%r specifiers in the format string than supplied arguments), producing malformed log lines with no error. There is no guard that count args matches the number of format specifiers.

Comment thread di/log/log.q
fatal:{[ctx;msg] logline["FATAL";ctx;msg];};

/ pre-wrapped log dependency dict ready to pass straight into any di.* module's init
/ e.g. email.init[mylog.defaultdict]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add accepts either a bare handle or a 2-element (handle;fn) pair, but the remove function (line 151) only matches on first p (the handle component). If the user registers the same handle twice with different sender functions, remove will remove both entries for that handle rather than just the one for the specified sender. The intent of storing (handle;sender) pairs is to allow the same handle with different senders, but remove conflates them. The fix is to match on the full pair: not h~p when h is a pair, or keep the existing handle-only match only when h is a scalar.

Comment thread di/log/log.q
}[id;gv;lvl;]
};

/ update field k with value v in instance id's state

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updInst writes back via .z.m.inst:state, but in add (line 112) the same assignment is done inline as .z.m.inst:state — bypassing updInst. This is inconsistent but not itself a bug. The real issue is that neither path is protected against concurrent modification: if two log instances call add/remove or setlvl concurrently (e.g. from a timer and a message handler in the same process), the read-modify-write on .z.m.inst is a race condition. This is a structural issue with the shared mutable dictionary design.

@DI-Software-Engineering

Copy link
Copy Markdown

DIReview Summary

1 critical | 4 warning(s) | 0 suggestion(s)

⚠️ Spec check skipped — tracker lookup failed (NO_REF_FOUND). Standards axis only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants