diff --git a/docs/concepts/llm.md b/docs/concepts/llm.md new file mode 100644 index 0000000..28caad5 --- /dev/null +++ b/docs/concepts/llm.md @@ -0,0 +1,151 @@ +# Large Language Model (LLM) Configuration + +OpenAF provides first-class support for Local Language Model (LLM) prompts via the `$llm` function in JavaScript jobs and the `oafp` CLI. You can configure which model to use by setting environment variables at runtime. + +## Environment Variables + +### OAF_MODEL + +Used by the `$llm` function and OpenAF JavaScript APIs to select and configure a GPT model. If no explicit model is passed to `$llm()`, OpenAF will look at `OAF_MODEL` as a JSON or SLON string. + +- Type: string (JSON/SLON) or simple `` name +- Default key: `OAF_MODEL` + +Examples: + +```bash +# Full JSON configuration (SLON is also supported) +export OAF_MODEL='{ \ + type: "openai", \ + key: "YOUR_OPENAI_API_KEY", + model: "gpt-4", + temperature: 0.7, + timeout: 900000 +}' + +# Inline shorthand (must parse as a map) +export OAF_MODEL='(type: openai, key: "$OPENAI_KEY", model: gpt-4, temperature: 0.7, timeout: 900000)' +``` + +With `OAF_MODEL` set, you can use `$llm()` without arguments: + +```javascript +// Inside an oJob or script +var response = $llm().prompt("Summarize the following text...") +cprint(response) +``` + +You can still override at call time: + +```javascript +var response = $llm({"type": "ollama", "model": "mistral", "url": "https://models.local", "timeout": 900000}) + .prompt("Analyze this data...") +``` + +### OAFP_MODEL + +Used by the `oafp` command-line tool to drive prompts from your shell or pipelines. Set `OAFP_MODEL` in the same format as `OAF_MODEL`. + +Examples: + +```bash +# Use OpenAI via CLI +export OAFP_MODEL='(type: openai, key: "$OPENAI_KEY", model: gpt-4, temperature: 0.7, timeout: 900000)' + +# Use Gemini via CLI +export OAFP_MODEL='(type: gemini, model: gemini-2.5-flash-preview-05-20, key: YOUR_GEMINI_KEY, timeout: 900000, temperature: 0, params: (tools: [(googleSearch: ())]))' +``` + +Then run: + +```bash +# Single prompt +oafp in=llm data="Translate this to French: Hello, world" + +# Prompt from stdin +TOPIC=love && printf "Write a poem about ${TOPIC}" | oafp in=llm +``` + +## Tips + +- **Environment isolation**: Set these variables in your CI/CD or container environment to avoid hard-coding secrets. +- **SLON support**: OpenAF will parse SLON (Single Line Object Notation) if you prefer a more compact syntax. +- **Multiple providers**: You can switch providers by changing `type` (e.g. `openai`, `gemini`, `ollama`, `anthropic`). + +## Reference Table: OAF_MODEL / OAFP_MODEL Fields + +| Field | Type | Required | Description | Provider Notes | +|-------------- |----------|----------|-----------------------------------------------------------------------------|-------------------------------------------------| +| type | string | Yes | Provider type (e.g. `openai`, `gemini`, `ollama`, `anthropic`, etc.) | Must match supported provider | +| options | map | Yes | Provider-specific options (see below) | Structure varies by provider | +| tools | map | No | Tool definitions for function calling | OpenAI, Gemini, Anthropic | +| timeout | integer | No | Request timeout in milliseconds | All | +| noSystem | boolean | No | If true, suppress system messages in output | All | +| headers | map | No | Custom HTTP headers | All | +| params | map | No | Additional provider-specific parameters (e.g. `max_tokens`, `top_p`) | OpenAI, Gemini, Anthropic, Ollama | + +**Provider-specific `options` fields:** + +| Provider | Option | Type | Required | Description | +|------------|-------------|----------|----------|----------------------------------------------| +| openai | key | string | Yes | API key | +| openai | model | string | Yes | Model name (e.g. `gpt-3.5-turbo`) | +| openai | temperature | float | No | Sampling temperature | +| openai | url | string | No | API endpoint (default: OpenAI) | +| gemini | key | string | No | API key (if required) | +| gemini | model | string | Yes | Model name (e.g. `gemini-pro`) | +| ollama | model | string | Yes | Model name (e.g. `llama2`) | +| ollama | url | string | No | Ollama server URL | +| anthropic | key | string | Yes | API key | +| anthropic | model | string | Yes | Model name (e.g. `claude-3-opus-20240229`) | + +> to be completed + +## Advanced Usage Examples + +### Using $llm with JSON Context + +```javascript +// Pass a JSON context to the prompt +var context = { + user: "Alice", + data: [1, 2, 3, 4], + task: "Summarize the numbers above." +}; +var response = $llm().withContext(context, "context data").prompt( + `Given the following context, provide a summary.` +) +print(response) +``` + +### Using $llm with Image Input + +```javascript +// Prompt with an image (file path or base64) +var response = $llm().promptImage( + "Describe the contents of this image.", + "/path/to/image.png" +); +print(response); +``` + +### Using oafp CLI with JSON Context + +```bash +# Pass JSON context as input +oafp data='{ user: "Bob", numbers: [5,6,7] }' llmcontext="numbers used by user" llmprompt: "Summarize the numbers for the user."}' +``` + +### Using oafp CLI with Image Input + +```bash +# Prompt with an image (path or base64) +oafp in=llm data='What is in this image?' llmimage="/path/to/image.jpg" +``` + +--- + +## Further Reading + +- [oJob LLM Integration](../ojob.md#job-llm) +- [OpenWrap AI API Reference](../api/openaf.md#owai-gpt) diff --git a/docs/guides/ojob/build-mcp-server.md b/docs/guides/ojob/build-mcp-server.md new file mode 100644 index 0000000..a9b62e0 --- /dev/null +++ b/docs/guides/ojob/build-mcp-server.md @@ -0,0 +1,198 @@ +--- +layout: default +title: Build MCP servers with oJob +parent: oJob +grand_parent: Guides +--- + +# Build Model Context Protocol servers + +OpenAF can expose [Model Context Protocol](https://modelcontextprotocol.io/) tools through regular oJobs. The `https://github.com/openaf/mini-a/mcps` directory in the distribution contains working examples (`mcp-db.yaml`, `mcp-ch.yaml`, …) and the shared runtime lives in `https://github.com/openaf/oJob-common/oJobMCP.yaml`. This guide distils the key pieces so you can author your own MCP server and test it locally or over HTTP. + +## Understand the shared MCP runtime + +`oJob-common/oJobMCP.yaml` publishes two shortcuts that do most of the heavy lifting: + +- `httpdMCP` starts an HTTP JSON-RPC endpoint (defaults to `/mcp`) and wires requests to jobs you expose. +- `stdioMCP` speaks MCP over STDIO, allowing you to run the server as a child process. + +Both variants expect: + +- `description.serverInfo` with `name`, `title` and `version`. +- `fnsMeta`: MCP tool metadata (JSON Schema, annotations, etc.). +- `fns`: the mapping between MCP tool names and oJob job names. + +Your YAML file only needs to prepare these maps and include the shortcut. + +## Start from the MCP skeleton + +Create a new file under `mini-a/mcps/`, e.g. `mcp-myservice.yaml`, following the structure below (adapted from `mini-a/mcps/CREATING.md`): + +```yaml +# Author: You +help: + text : A STDIO/HTTP MCP server for MyService + expects: + - name : onport + desc : If defined starts an HTTP MCP server on the provided port + example : "8888" + mandatory: false + - name : myConfig + desc : Example extra parameter + example : "value" + mandatory: true + +todo: +- Init MyService +- (if ): "isDef(args.onport)" + ((then)): + - (httpdMCP): &MCPSERVER + description: + serverInfo: + name : mini-a-myservice + title : OpenAF MCP MyService + version: 1.0.0 + ((fnsMeta)): &MCPFNSMETA + myservice-tool: + name : myservice-tool + description: Performs the main operation + inputSchema: + type : object + properties: + param1: + type : string + description: Example parameter + required: [ param1 ] + annotations: + title : MyService Tool + readOnlyHint : true + idempotentHint: true + ((fns )): &MCPFNS + myservice-tool: MyService Tool + ((else)): + - (stdioMCP): *MCPSERVER + ((fnsMeta)): *MCPFNSMETA + ((fns )): *MCPFNS + +ojob: + opacks: + - openaf : 20250915 + - oJob-common: 20250914 + daemon : true + argsFromEnvs: true + logToConsole: false + +include: +- oJobMCP.yaml + +jobs: +- name : Init MyService + check: + in: + myConfig: isString + exec : | #js + // Initialise SDK clients, cache credentials, etc. + global.myServiceConfig = args + +- name : MyService Tool + check: + in: + param1: isString + exec : | #js + if (!isDef(global.myServiceConfig)) return "[ERROR] Service not initialised" + return { + content: [{ + type: "text", + text: `Processed ${args.param1}` + }] + } + +- name : Cleanup MyService + type : shutdown + exec : | #js + delete global.myServiceConfig +``` + +Key points: + +- Keep the `help` section up to date so users know which arguments are required. +- The `todo` block first runs any initialisation job(s) and then chooses between HTTP or STDIO mode using the `onport` argument. +- `fnsMeta` entries must describe your tools with valid JSON Schema; `fns` maps the tool names to the job that should run. +- Always add a shutdown job for cleanup — it runs when the MCP exits. + +## Tips for implementing tools + +- Return errors as strings that start with `[ERROR] …`. The existing MCP tooling looks for that pattern. +- Use `check.in` validations so bad inputs fail fast. +- For read/write operations, consider a flag (e.g. `allowWrite`) to prevent accidental changes, similar to `mcp-ssh.yaml`. +- Inspect other MCPs (database, time, SSH, etc.) in `mini-a/mcps/` for real-world patterns. + +## Smoke-test the server with oJob + +Run in STDIO mode during development: + +```bash +ojob mcps/mcp-myservice.yaml myConfig=value +``` + +Run as an HTTP server: + +```bash +ojob mcps/mcp-myservice.yaml onport=12345 myConfig=value +``` + +Both rely on the shortcuts from `oJob-common/oJobMCP.yaml`. If you need extra HTTP middleware, add it before the `httpdMCP` call. + +## Test with `$mcp` inside OpenAF + +`openaf/js/openaf.js` ships the `$mcp` client helper. You can use it from the REPL or inside your scripts: + +```javascript +var client = $mcp({ + cmd : "ojob mcps/mcp-myservice.yaml myConfig=value", + debug : false, + strict: true +}) + +client.initialize() + +log(client.listTools()) // -> { tools: [...] } + +var result = client.callTool("myservice-tool", { param1: "demo" }) +log(result) + +client.destroy() +``` + +For HTTP mode, switch to `type: "remote"` and provide `url: "http://localhost:12345/mcp"`. + +## Test with `oafp in=mcp` + +`oafp` can act as a CLI MCP client (see `oafp/src/docs/USAGE.md`): + +- List tools exposed by your STDIO server: + + ```bash + oafp in=mcp inmcptoolslist=true data="(cmd: 'ojob mcps/mcp-myservice.yaml myConfig=value')" + ``` + +- Invoke a tool via STDIO: + + ```bash + oafp in=mcp data="(cmd: 'ojob mcps/mcp-myservice.yaml myConfig=value', tool: 'myservice-tool', params: (param1: 'demo'))" + ``` + +- Invoke the same tool over HTTP: + + ```bash + oafp in=mcp data="(type: remote, url: 'http://localhost:12345/mcp', tool: 'myservice-tool', params: (param1: 'demo'))" + ``` + +Set `inmcplistprompts=true` to inspect available prompts (if you expose any) or use the other connection fields (`__timeout__`, `__clientInfo__`, …) listed in the `oafp` usage document. + +## Where to go next + +- Browse the catalog in `https://github.com/openaf/mini-a/mcps/README.md` for inspiration. +- Revisit `https://github.com/openaf/mini-a/mcps/CREATING.md` whenever you need the full checklist. +- Extend `https://github.com/openaf/oJob-common/oJobMCP.yaml` if you need new shared behaviours (additional logging, authentication, etc.). + diff --git a/docs/reference/db.md b/docs/reference/db.md index d4439bd..6c49f53 100644 --- a/docs/reference/db.md +++ b/docs/reference/db.md @@ -34,7 +34,13 @@ Closes the corresponding prepared statement. If an error occurs during this proc __DB.commit()__ ```` -Commits to the database the current database session on the current DB object instance. In case of error an exception will be thrown. +Commits the current transaction associated with this DB object. An exception will be thrown if any database error occurs. + +Example: + +db.u("INSERT INTO A VALUES (1)"); +db.commit(); + ```` ### DB.convertDates @@ -96,14 +102,14 @@ Stops a H2 server started for this DB instance. __DB.q(aQuery) : Map__ ```` -Performs aQuery (SQL) on the current DB object instance. It returns a Map with a results array that will have an element per result set line. In case of error an exception will be thrown. +Executes the SQL query provided in aQuery on the current DB connection. It returns a Map with a results array where each element corresponds to one row. In case of error an exception will be thrown. ```` ### DB.qLob __DB.qLob(aSQL) : Object__ ```` -Performs aSQL query on the current DB object instance. It tries to return only the first result set row and the first object that can be either a CLOB (returns a string) or a BLOB (byte array). In case of error an exception will be thrown. +Executes aSQL on the current DB object returning only the first row and column. If that column is a CLOB a string is returned; if it is a BLOB a byte array is provided. Errors will raise an exception. ```` ### DB.qs diff --git a/docs/reference/httpd.md b/docs/reference/httpd.md index 57670a2..3e070e5 100644 --- a/docs/reference/httpd.md +++ b/docs/reference/httpd.md @@ -84,6 +84,13 @@ __HTTPd.delSession(aSessionID)__ ```` Removes the provided session ID data from memory. ```` +### HTTPd.getImpl + +__HTTPd.getImpl() : string__ + +```` +Returns the implementation used by the HTTP server instance. Possible values are "java" for Java built-in HttpServer, "nwu2" for NWU2 HTTP server, and "nwu" for legacy NWU HTTP server. +```` ### HTTPd.getPort __HTTPd.getPort() : number__ @@ -107,10 +114,10 @@ Returns true if the provided session ID was add to memory. False otherwise. ```` ### HTTPd.HTTPd -__HTTPd.HTTPd(aPort, aLocalInteface, keyStorePath, keyStorePassword, logFunction, webSockets, aTimeout)__ +__HTTPd.HTTPd(aPort, aLocalInteface, keyStorePath, keyStorePassword, logFunction, webSockets, aTimeout, aImpl)__ ```` -Creates a HTTP server instance on the provided port and optionally on the identified local interface. If the port provided is 0 or negative a random port will be assigned. To determine what this port is you can use the function HTTPServer.getPort(). If keyStorePath is defined, the corresponding SSL Key Store will be used (connections will be https) with the provided keyStorePassword. Do note that the keyStorePath should be included in the OpenAF classpath. The logFunction, if defined, will be called by the server whenever there is any logging to be performed by the HTTPServer. This function will receive 3 arguments. Example: +Creates a HTTP server instance on the provided port and optionally on the identified local interface. If the port provided is 0 or negative a random port will be assigned. To determine what this port is you can use the function HTTPServer.getPort(). If aImpl = "java" is provided the Java built-in HttpServer implementation will be used. If keyStorePath is defined, the corresponding SSL Key Store will be used (connections will be https) with the provided keyStorePassword. Do note that the keyStorePath should be included in the OpenAF classpath. The logFunction, if defined, will be called by the server whenever there is any logging to be performed by the HTTPServer. This function will receive 3 arguments. Example: plugin("HTTPServer"); var s = new HTTPd(8091, void 0, void 0, void 0, function(aType, aMsg, anException) { @@ -131,7 +138,7 @@ And then add keystore.jks to the openaf.jar and have keyStorePath = "/keystore.j To support websockets you need to build IWebSock object and provide a timeout. For example: plugin("HTTPServer"); -var webSock = new Packages.com.nwu.httpd.IWebSock({ +var webSock = new Packages.com.nwu2.httpd.IWebSock({ // onOpen callback oOpen: _ws => { log("Connection open") }, // onClose callback diff --git a/docs/reference/index.md b/docs/reference/index.md index 56052b3..2c8c888 100644 --- a/docs/reference/index.md +++ b/docs/reference/index.md @@ -6,5 +6,4 @@ has_children: true nav_order: 4 --- # Reference -_version: 20250315_ - +_version: 20250725_ \ No newline at end of file diff --git a/docs/reference/ow_ai.md b/docs/reference/ow_ai.md index 53aa895..d798c05 100644 --- a/docs/reference/ow_ai.md +++ b/docs/reference/ow_ai.md @@ -14,6 +14,19 @@ __$gpt(aModel) : $gpt__ ```` Creates a GPT AI model of aType (e.g. "openai" or "ollama") with aOptions. + +aModel can be a map with the following properties: +- type: the type of the model (e.g. "openai", "ollama", "anthropic") +- options: a map with the options for the model (e.g. { key: "your-api-key", model: "gpt-3.5-turbo", temperature: 0.7, url: "https://api.openai.com/" }) +- conversation: an array of messages to start the conversation (e.g. [{ role: "system", content: "You are a helpful assistant." }, { role: "user", content: "Hello!" }]) +- tools: a map with the tools to use in the conversation (e.g. { "tool1": { type: "function", function: { name: "tool1", description: "A tool that does something", parameters: { type: "object", properties: { param1: { type: "string" } } } }, fn: function(args) { return args.param1; } } }) +- timeout: the timeout for the requests in milliseconds (defaults to 15 minutes) +- noSystem: if true, it will not output the system messages (defaults to true) +- instructions: a string or an array of strings with the instructions for the model (e.g. "json", "boolean", "sql", "js", "path") +- headers: a map with the headers to use in the requests (e.g. { "Content-Type": "application/json" }) +- params: a map with the parameters to use in the requests (e.g. { "max_tokens": 1000, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }) + +If aModel is not provided, it will try to get the model from the environment variable "OAF_MODEL" with the map in JSON or SLON format. ```` ### $gpt.close diff --git a/docs/reference/ow_ch.md b/docs/reference/ow_ch.md index 73f40c9..1c42571 100644 --- a/docs/reference/ow_ch.md +++ b/docs/reference/ow_ch.md @@ -20,7 +20,12 @@ Returns a function to be used with ow.ch.subscribe for REST communication to the __ow.ch.create(aName, shouldCompress, type, options) : ow.ch__ ```` -Creates a channel of key/values with aName. Optionally you can specify if keys should also be compressed in memory (shouldCompress = true), a channels implementation type and corresponding options in a map. +Creates a channel of key/values with aName. Optionally you can specify if keys should also be compressed in memory (shouldCompress = true), a channels implementation type and corresponding options in a map. + +Example: + +ow.ch.create("myChan", false, "db", { db: myDB, from: "TABLE" }); + ```` ### ow.ch.destroy @@ -282,7 +287,7 @@ __ow.ch.types.db__ ```` This OpenAF channel implementation wraps access to a db table. The creation options are: - - db (Database) The database object to access the database table. + - db (Database) The database object to access the database table (or a JSON/SLON string with 'url', 'user', 'pass', 'timeout', 'driver'). - from (String) The name of the database table or object (don't use double quotes). - keys (Array) An array of fields keys to use (don't use double quotes). - cs (Boolean) Determines if the database is case sensitive for table and field names (defaults to false). @@ -327,6 +332,7 @@ This OpenAF implementation implements a simple channel on a single JSON or YAML - multipath (Boolean) Supports string keys with paths (e.g. ow.obj.setPath) (defaults to false) - lock (String) If defined the filepath to a dummy file for filesystem lock while accessing the file - gzip (Boolean) If true the output file will be gzip (defaults to false) + - lz4 (Boolean) If true the output file will be lz4 (defaults to false) - tmp (Boolean) If true "file" will be temporary and destroyed upon execution/process end (*) - Be aware that althought there is a very small probability of collision between the unique id (sha-512) for filenames it still exists @@ -365,7 +371,12 @@ The map will be created if it doesn't exist. Operations getKeys/getAll can be pa __ow.ch.types.ops__ ```` -This OpenAF channel implementation encapsulates access based on functions. The creation options a map of keys where each value is a function. +This OpenAF channel implementation encapsulates access based on functions. The creation options are a map of keys where each value is a function. + +Example: + +ow.ch.create("logic", false, "ops", { hello: name => "hi " + name }); + ```` ### ow.ch.types.prometheus diff --git a/docs/reference/ow_format.md b/docs/reference/ow_format.md index 34932d1..c07f96a 100644 --- a/docs/reference/ow_format.md +++ b/docs/reference/ow_format.md @@ -493,9 +493,9 @@ __ow.format.int2IP(aIPInt) : String__ ```` Converts the decimal IP address representation aIPInt into an IP address. ```` -### ow.format.IP2Int +### ow.format.IP2int -__ow.format.IP2Int(aIP) : String__ +__ow.format.IP2int(aIP) : String__ ```` Converts an IP address into it's decimal IP address representation. @@ -1362,15 +1362,15 @@ Creates a cell styler object, for the aXLS object (XLS plugin object instance), - align ("center", "centerSelection", "fill", "general", "justify", "left", "right") Color names:\ -aqua,auto,black,blue,blue_grey,bright_green,brown,coral,cornflower_blue,dark_blue,dark_green,dark_red,dark_teal, dark_yellow,gold,green,grey25,grey40,grey50,grey80,indigo,lavender,lemon_chiffon,light_blue,light_cornflower_blue, light_green,light_orange,light_turquoise,light_yellow,lime,maroon,olive_green,orange,orchid,pale_blue,pink,plum, red,rose,royal_blue,sea_green,sky_blue,tan,teal,turquoise,violet,white,yellow + aqua,auto,black,blue,blue_grey,bright_green,brown,coral,cornflower_blue,dark_blue,dark_green,dark_red,dark_teal, dark_yellow,gold,green,grey25,grey40,grey50,grey80,indigo,lavender,lemon_chiffon,light_blue,light_cornflower_blue, light_green,light_orange,light_turquoise,light_yellow,lime,maroon,olive_green,orange,orchid,pale_blue,pink,plum, red,rose,royal_blue,sea_green,sky_blue,tan,teal,turquoise,violet,white,yellow Border names: -dash_dot,dash_dot_dot,dashed,dotted,double,hair,medium,medium_dash_dot,medium_dash_dot_dot,medium_dashed,none, slanted_dash_dot,thick,thin + dash_dot,dash_dot_dot,dashed,dotted,double,hair,medium,medium_dash_dot,medium_dash_dot_dot,medium_dashed,none, slanted_dash_dot,thick,thin Fill patterns: -solid_foreground + solid_foreground ```` diff --git a/docs/reference/ow_java.md b/docs/reference/ow_java.md index 404412a..7c7dfd4 100644 --- a/docs/reference/ow_java.md +++ b/docs/reference/ow_java.md @@ -588,6 +588,48 @@ __ow.java.parseHSPerf(aByteArrayOrFile, retFlat) : Map__ ```` Given aByteArray or a file path for a java (hotspot jvm) hsperf file (using ow.java.getLocalJavaPIDs or similar) will return the java performance information parsed into a map. If retFlat = true the returned map will be a flat map with each java performance metric and correspondent value plus additional calculations with the prefix "__" ```` +### ow.java.parseJFR + +__ow.java.parseJFR(aFile, aFnRec, includeDesc) : Array__ + +```` +Given aFile (a Java Flight Recorder file) will parse it and call aFnRec for each event found, if defined. Optionally you can provide aFnRec to receive each event as a map with the following fields: startTime, endTime, duration, name, thread and fields. If includeDesc is true it will include a field (__desc_) for each field added with the description of the field. +```` +### ow.java.pidCheckJFR + +__ow.java.pidCheckJFR(aPid) : String__ + +```` +Tries to attach to local JVM with aPid and check the current Java Flight Recorder recording status. +```` +### ow.java.pidDumpJFR + +__ow.java.pidDumpJFR(aPid, aName, aFile) : String__ + +```` +Tries to attach to local JVM with aPid and dump a Java Flight Recorder returning the output. +```` +### ow.java.pidStartJFR + +__ow.java.pidStartJFR(aPid, aDuration, aName, aMaxSize) : String__ + +```` +Tries to attach to local JVM with aPid and start a Java Flight Recorder returning the output. +```` +### ow.java.pidStopJFR + +__ow.java.pidStopJFR(aPid, aName) : String__ + +```` +Tries to attach to local JVM with aPid and stop a Java Flight Recorder returning the output. +```` +### ow.java.pidSystemProperties + +__ow.java.pidSystemProperties(aPid) : Map__ + +```` +Tries to attach to local JVM with aPid and retrieve the system properties. +```` ### ow.java.pidThreadDump __ow.java.pidThreadDump(aPid) : String__ diff --git a/docs/reference/ow_obj.md b/docs/reference/ow_obj.md index 58e81b7..25223b9 100644 --- a/docs/reference/ow_obj.md +++ b/docs/reference/ow_obj.md @@ -971,17 +971,17 @@ Creates an instance of a thread-safe array/list. Optionally it can be initialize ```` ### ow.obj.syncArray.add -__ow.obj.syncArray.add(aObject) : boolean__ +__ow.obj.syncArray.add(aObject, aPosition) : boolean__ ```` -Adds aObject to the internal array/list. +Adds aObject to the internal array/list. If aPosition is provided it will be added at that position, otherwise it will be added at the end. ```` ### ow.obj.syncArray.addAll -__ow.obj.syncArray.addAll(anArray)__ +__ow.obj.syncArray.addAll(anArray, aPosition)__ ```` -Concatenates anArray with the internal array/list. +Concatenates anArray with the internal array/list. If aPosition is provided it will be added at that position, otherwise it will be added at the end. ```` ### ow.obj.syncArray.clear @@ -1011,6 +1011,13 @@ __ow.obj.syncArray.getJavaObject() : Object__ ```` Returns the internal java object. ```` +### ow.obj.syncArray.getUnsafeJavaObject + +__ow.obj.syncArray.getUnsafeJavaObject() : Object__ + +```` +Returns the internal java object without synchronization. Use with care! This is useful when you want to use the internal java object directly without the overhead of synchronization. +```` ### ow.obj.syncArray.indexOf __ow.obj.syncArray.indexOf(aObject) : Number__ @@ -1018,6 +1025,13 @@ __ow.obj.syncArray.indexOf(aObject) : Number__ ```` Returns the position of aObject in the internal array/list. ```` +### ow.obj.syncArray.isEmpty + +__ow.obj.syncArray.isEmpty() : boolean__ + +```` +Returns true if the internal array/list is empty. +```` ### ow.obj.syncArray.length __ow.obj.syncArray.length() : Number__ @@ -1039,6 +1053,13 @@ __ow.obj.syncArray.set(aIndex, aObject) : Object__ ```` Sets aObject overwriting the previous value at aIndex on the internal array/list. ```` +### ow.obj.syncArray.subList + +__ow.obj.syncArray.subList(aStart, aEnd) : Array__ + +```` +Returns a sublist of the internal array/list from aStart to aEnd (exclusive). If aStart or aEnd are out of bounds it will throw an exception. The returned array will be a new array with the elements from aStart to aEnd. +```` ### ow.obj.syncArray.toArray __ow.obj.syncArray.toArray() : Array__ @@ -1046,6 +1067,13 @@ __ow.obj.syncArray.toArray() : Array__ ```` Returns the internal array/list as a javascript array. ```` +### ow.obj.syncArray.trimToSize + +__ow.obj.syncArray.trimToSize()__ + +```` +Trims the internal array/list to its current size. +```` ### ow.obj.syncMap __ow.obj.syncMap(aMap) : ow.obj.syncMap__ diff --git a/docs/reference/ow_server.md b/docs/reference/ow_server.md index a37526b..ccac1b8 100644 --- a/docs/reference/ow_server.md +++ b/docs/reference/ow_server.md @@ -383,6 +383,52 @@ aMapOptions: oauthCliendId (mandatory) The OAuth configured client id oauthClientSecret (mandatory) The OAuth configured client secret\ +```` +### ow.server.httpd.browse.files + +__ow.server.httpd.browse.files(aURI, aOptions) : Map__ + +```` +Provides a simple file browser for the provided aURI. The aOptions map can contain the following keys: + + path (string) The path to use as root (defaults to ".") + browse (boolean) If true the file browser will be shown (defaults to true) + showURI (boolean) If true the URI will be shown in the file browser (defaults to false) + sortTab (boolean) If true the table will be sorted (defaults to false) + default (string) The default file to show (defaults to undefined) + logo (string) The logo to show in the file browser (defaults to undefined) + footer (string) The footer to show in the file browser (defaults to undefined) + +```` +### ow.server.httpd.browse.odoc + +__ow.server.httpd.browse.odoc(aURI, aOptions) : Map__ + +```` +Provides a simple oDoc browser. The aOptions map can contain the following keys: + + browse (boolean) If true the file browser will be shown (defaults to true) + showURI (boolean) If true the URI will be shown in the file browser (defaults to false) + sortTab (boolean) If true the table will be sorted (defaults to false) + logo (string) The logo to show in the file browser (defaults to undefined) + footer (string) The footer to show in the file browser (defaults to undefined) + ttl (number) The time to live for the cache (defaults to 5 minutes) + +```` +### ow.server.httpd.browse.opacks + +__ow.server.httpd.browse.opacks(aURI, aOptions) : Map__ + +```` +Provides a simple oPack browser. The aOptions map can contain the following keys: + + browse (boolean) If true the file browser will be shown (defaults to true) + showURI (boolean) If true the URI will be shown in the file browser (defaults to false) + sortTab (boolean) If true the table will be sorted (defaults to false) + logo (string) The logo to show in the file browser (defaults to undefined) + footer (string) The footer to show in the file browser (defaults to undefined) + ttl (number) The time to live for the cache (defaults to 5 minutes) + ```` ### ow.server.httpd.getFromOpenAF @@ -432,6 +478,20 @@ __ow.server.httpd.reply(aObject, aStatus, aMimeType, aHeadersMap) : Map__ ```` Builds the map response for a HTTP server request using aObject (map, array, string or bytearray), aStatus (default to 200), aMimeType (defaults to application/octet-stream if not recognized) and aHeadersMap. +```` +### ow.server.httpd.replyBrowse + +__ow.server.httpd.replyBrowse(server, request, _options) : Map__ + +```` +Provides a helper function to reply with a list of files or a file content. The _options map is used to provide the following functions: + + getList(request, _options) : Function to get the list of files or directories + getObj(request, _options) : Function to get the object requested + renderList(list, server, request, _options) : Function to render the list of files + renderObj(obj, server, request, _options) : Function to render the object requested + renderEmpty(request, _options) : Function to render an empty list + ```` ### ow.server.httpd.replyFile @@ -485,6 +545,39 @@ r => hs.replyOKText("nothing here...") ); +```` +### ow.server.httpd.replyJSONRPC + +__ow.server.httpd.replyJSONRPC(server, request, mapOfFunctions, logFn, debugFn) : Map__ + +```` +Implements a JSON-RPC 2.0 endpoint using the provided mapOfFunctions. The request must be a POST with a JSON-RPC body. +Optionally you can provide logFn and debugFn functions to log errors and debug information respectively. + +Example usage: + ow.server.httpd.route(hs, { + "/rpc": (req) => ow.server.httpd.replyJSONRPC(hs, req, { sum: (a, b) => a + b }, logErr, log) + }) + +```` +### ow.server.httpd.replyMCP + +__ow.server.httpd.replyMCP(server, request, mapOfFunctions, logFn, debugFn) : Map__ + +```` +Implements a Model Context Protocol (MCP) endpoint using the provided mapOfFunctions. The request must be a POST with a MCP body. +Optionally you can provide logFn and debugFn functions to log errors and debug information respectively. +\ Example usage: +ow.server.httpd.route(hs, { + "/mcp": req => ow.server.httpd.replyMCP(hs, req, { + initialize : params => ({ serverInfo: { name: "OpenAF", title: "OpenAF test ", version: "1.0.0" }, capabilities: { prompts: { listChanged: true }, tools: { listChanged: true } } }), + "notifications/initialized": params => ({}), + "tools/call" : () => ({content:[{type:"text",text:"PONG!"}],isError: false}), + "tools/list" : params => { cprint(params); return { tools: [{name:"ping",description:"pings",title:"ping"}] } }, + "prompts/list" : params => ({}) + }, logErr, log), + "/echo": req => ow.server.httpd.reply(stringify(req)) + }) * ```` ### ow.server.httpd.replyRedirect @@ -532,10 +625,12 @@ Using aHTTPd sets aURI to act as a websocket server. ```` ### ow.server.httpd.start -__ow.server.httpd.start(aPort, aHost, keyStorePath, password, errorFunction, aWebSockets, aTimeout) : Object__ +__ow.server.httpd.start(aPort, aHost, keyStorePath, password, errorFunction, aWebSockets, aTimeout, aImpl) : Object__ ```` -Will prepare a HTTP server to be used returning a HTTPServer object. Optionally you can provide aPort where you want the HTTP server to run. Otherwise a free port will be assigned. (available after ow.loadServer()). +Will prepare a HTTP server to be used returning a HTTPServer object. Optionally you can provide aPort where you want the HTTP server to run. Otherwise a free port will be assigned. Optionally you can provide a different aImpl (implementation) for the HTTP server. If aHost is provided it will be used as the host to bind the server to. If keyStorePath and password are provided the server will be started as a secure HTTPS server. If errorFunction is provided it will be called whenever an error occurs in the server. This function will receive the error as a parameter. If aWebSockets is provided it will be used to handle WebSockets connections. If aTimeout is provided it will be used as the timeout for the server to wait for a request before closing the connection. + +(available after ow.loadServer()). aWebSockets, if used, should be a map with the following functions: @@ -608,6 +703,13 @@ __ow.server.jmx.set(aJMXObject, objName, aProperty, aNewValue)__ ```` Sets aProperty with the value aNewValue on the object objName using aJMXObject connected to a JMX server. ```` +### ow.server.jsonRPC + +__ow.server.jsonRPC(data, mapOfFns) : Map__ + +```` +Processes a JSON-RPC request (data) using the provided mapOfFns where each key is the method name and the value is the function to be executed. The data should be a map with the following entries: jsonrpc (should be "2.0"), method (the method name to be executed), params (optional, the parameters to be passed to the method) and id (optional, the request id). Returns a map with the following entries: jsonrpc (should be "2.0"), id (the request id), result (the result of the method execution) or error (if the method is unknown or if the request is invalid). +```` ### ow.server.jwt.decode __ow.server.jwt.decode(aToken) : Map__ @@ -755,6 +857,17 @@ __ow.server.locks.whenUnLocked(aLockName, aFunction, aTryTimeout, aNumRetries) : ```` A wrapper for ow.server.locks.lock that will try to lock aLockName, execute the provide function and then unlock it even in case an exception is raised. Returns if the lock was successfull (true) or not (false). ```` +### ow.server.mcpStdio + +__ow.server.mcpStdio(initData, fnsMeta, fns, lgF)__ + +```` +Processes a MCP (Model Context Protocol) request using standard input/output. The initData is a map with initial data to be sent to the client, fnsMeta is an array of function metadata and fns is a map of functions to be executed. The lgF is a function that will be used to log messages. If not provided, it will default to a function that writes logs to a file named "log.ndjson". The initData should contain the server information and capabilities. The fnsMeta should contain metadata about the functions available, such as their names and descriptions. The fns should contain the actual functions that can be called by the client. The function will listen for incoming MCP requests on standard input and respond accordingly. + +Example usage: + + ow.server.mcpStdio({ serverInfo: { name: "MyServer", title: "My Server", version: "1.0.0" }, capabilities: { prompts: { listChanged: true }, tools: { listChanged: true } } }, [{ name: "ping", description: "Ping the server" }, { name: "get_user", description: "Get user information", input_schema: { type: "object", properties: { userId: { type: "string", description: "The ID of the user to retrieve" } }, required: ["userId"] } }], { ping: params => { return "Pong! Server is running." }, get_user: params => { return { name: "Alice", userId: params.userId } } }) +```` ### ow.server.openafServer.exec __ow.server.openafServer.exec(aId, aScript, aServerPid)__ diff --git a/docs/reference/ow_template.md b/docs/reference/ow_template.md index 537a4c2..94b54fe 100644 --- a/docs/reference/ow_template.md +++ b/docs/reference/ow_template.md @@ -98,11 +98,13 @@ Adds custom helpers: - $a4m -- shortcut to the OpenAF's $a4m function - $m2a -- shortcut to the OpenAF's $m2a function - $m4a -- shortcut to the OpenAF's $m4a function + - $nvl -- shortcut to the OpenAF's nvl function - $pass -- returns an empty string - $p -- returns the provided literal - $sline -- shortcut to the OpenAF's format withSideLine - $set -- block set of a provided key - $concat -- concatenates all arguments as a single value + - $nl -- returns a new line character ```` ### ow.template.addPartial diff --git a/docs/reference/scope.md b/docs/reference/scope.md index 23ed847..f6bf11c 100644 --- a/docs/reference/scope.md +++ b/docs/reference/scope.md @@ -121,6 +121,13 @@ __$bottleneck.maxWait(aMs) : Object__ ```` Creates a bottleneck holding the function execution for a max period of aMs. ```` +### $cache.byDefault + +__$cache.byDefault(useDefault, aDefault) : Object__ + +```` +Changes the behaviour of the cache to either use the default value (aDefault) if useDefault is true (launching the cache function in background) or try to use the previous value in the cache if useDefault is false (if a previous value is not available the cache function will be called and the .get will wait for it). +```` ### $cache.byPopularity __$cache.byPopularity() : Object__ @@ -290,10 +297,10 @@ Instantiates and returns a oPromise. If you provide aFunction, this aFunction wi ```` ### $doA2B -__$doA2B(aAFunction, aBFunction, numberOfDoPromises, defaultTimeout, aErrorFunction)__ +__$doA2B(aAFunction, aBFunction, numberOfDoPromises, defaultTimeout, aErrorFunction, useVirtualThreads)__ ```` -Will call aAFunction with a function as argument that should be used to "send" values to aBFunction. aBFunction will be call asynchronously in individual $do up to the numberOfDoPromises limit. The defaultTimeout it 2500ms. If aErrorFunction is defined it will received any exceptions thrown from aBFunction with the corresponding arguments array. +Will call aAFunction with a function as argument that should be used to "send" values to aBFunction. aBFunction will be call asynchronously in individual $do up to the numberOfDoPromises limit. The defaultTimeout it 2500ms. If aErrorFunction is defined it will received any exceptions thrown from aBFunction with the corresponding arguments array. If useVirtualThreads is true it will use virtual threads to execute aBFunction. If useVirtualThreads is false (default) it will use the thread pool. Use virtual threads if you want to execute aBFunction without blocking the current thread (IO bound operations / high concurrency). ```` ### $doAll @@ -309,6 +316,13 @@ __$doFirst(anArray) : oPromise__ ```` Returns an oPromise that will be resolved when any oPromise part of the anArray is fullfiled. If any of the oPromises fails/rejects the returned oPromise will also be rejected/fail. ```` +### $doV + +__$doV(aFunction, aRejFunction) : oPromise__ + +```` +Equivalent to $do but using virtual threads (if available) to execute the aFunction. This will allow the aFunction to be executed without blocking the current thread. If aFunction is not provided it will return a new oPromise that will be resolved when the first executor is added to it. If aRejFunction is provided it will be used as a "catch" method for the oPromise. +```` ### $doWait __$doWait(aPromise, aWaitTimeout) : oPromise__ @@ -316,6 +330,13 @@ __$doWait(aPromise, aWaitTimeout) : oPromise__ ```` Blocks until aPromise is fullfilled or rejected. Optionally you can specify aWaitTimeout between checks. Returns aPromise. ```` +### $err + +__$err(exception, rethrow, returnStr, code)__ + +```` +Prints the exception stack trace and the code where it was thrown. If rethrow is true it will throw the exception again. If returnStr is true it will return the string instead of printing it. +```` ### $f __$f(aString, arg1, arg2, ...) : String__ @@ -486,6 +507,28 @@ __$job(aJob, args, aId, isolate) : Object__ ```` Shortcut to oJobRunJob and ow.oJob.runJob to execute aJob with args and returned the changed arguments. Optionally aId can be also provided. If isolate=true it will also clean the key 'res' and try to return the result of ow.oJob.output. +```` +### $jsonrpc + +__$jsonrpc(aOptions) : Map__ + +```` +Creates a JSON-RPC client that can be used to communicate with a JSON-RPC server or a local process using stdio. The aOptions parameter is a map with the following possible keys: type (string, default "stdio" for local process or "remote" for remote server), url (string, required for remote server), timeout (number, default 60000 ms for remote server), cmd (string, required for local process), and options (map, optional additional options for remote server). The returned map has the following methods: type (to set the type), url (to set the URL for remote server), sh (to set the command for local process), exec (to execute a method with parameters), and destroy (to stop the client). The exec method returns a promise that resolves to the result of the method call or an error if the call fails. Example usage: + +var client = $jsonrpc({type: "remote", url: "http://example.com/api", timeout: 5000}); +client.exec("methodName", {param1: "value1", param2: "value2"}).then(result => { + log("Result:", result); +}).catch(error => { + logErr("Error:", error); +}); + +var localClient = $jsonrpc({type: "stdio", cmd: "myLocalProcess"}); +localClient.exec("localMethod", {param1: "value1"}).then(result => { + log("Local Result:", result); +}).catch(error => { + logErr("Local Error:", error); +}); + ```` ### $llm @@ -674,6 +717,24 @@ __$pyStop()__ ```` Stops the background python process started by $pyStart. ```` +### $queue + +__$queue(anArray) : Object__ + +```` +Returns an object with the following methods: +- add(aItem) : adds aItem to the queue and returns true if successful +- remove(aItem) : removes aItem from the queue and returns true if successful +- addAll(aItems) : adds all aItems to the queue and returns true if successful +- has(aItem) : returns true if aItem is in the queue +- isEmpty() : returns true if the queue is empty +- size() : returns the size of the queue +- toArray() : returns the queue as an array +- peek() : returns the first item in the queue without removing it +- poll() : returns the first item in the queue and removes it + The queue is implemented using a java.util.concurrent.ConcurrentLinkedQueue, which is thread-safe and allows concurrent access. +If anArray is provided, it will be added to the queue using addAll. +```` ### $rest.delete __$rest.delete(aBaseURI, aIdxMap) : Map__ @@ -1166,6 +1227,20 @@ __$stream__ ```` Shortcut for the streamjs library for easy query and access to streams of data. To see all the available options please refer to https://github.com/winterbe/streamjs/blob/master/APIDOC.md. ```` +### $sync + +__$sync() : Object__ + +```` +Returns an object with a 'run(aFn)' function that will execute the provided aFn in a synchronized way. The run function will lock the execution until the aFn is finished. This is useful to ensure that only one thread is executing the aFn at a time. + +Example: + +var s = $sync() +s.run(() => { + // Your code here, only one thread will execute this at a time +}) +```` ### $tb __$tb(aFunction) : Result__ @@ -1297,6 +1372,13 @@ __AF.getEncoding(anArrayOfBytesORString) : String__ ```` Given anArrayOfBytesORString will try to detect which encode is used and returns a string with the identified charset encoding. ```` +### af.nvl + +__af.nvl(aValue, aDefault) : Object__ + +```` +Returns aValue if it is defined or aDefault otherwise. +```` ### AF.printStackTrace __AF.printStackTrace(aFunction) : Object__ @@ -1332,6 +1414,13 @@ __AF.setInteractiveTerminal()__ ```` Sets the current terminal to be interactive (no echo, no buffering). ```` +### AF.swap + +__AF.swap(anArray, anIndex1, anIndex2) : Array__ + +```` +Swaps the elements at anIndex1 and anIndex2 in anArray. Returns the new array with the elements swapped. +```` ### AF.toCSLON __AF.toCSLON(aObject, aTheme) : String__ @@ -1889,6 +1978,13 @@ __io.listFilesTAR(aTARfile, isGzip) : Array__ ```` Given aTARfile (or output stream (with isGzip = true/false)) will return an array with the TAR file entries. Each entry will have: isDirectory (boolean), isFile (boolean), canonicalPath (string), filepath (string), filename (string), size (number), lastModified (date), groupId (string), group (string), userId (string) and user (string). ```` +### io.lz4 + +__io.lz4(anInput) : bytes__ + +```` +Compresses anInput using LZ4 compression and returns the compressed bytes. +```` ### io.onDirEvent __io.onDirEvent(aPath, aFn, aFnErr) : Promise__ @@ -1934,6 +2030,13 @@ __io.readFileJSON(aJSONFile) : Object__ ```` Tries to read aJSONFile into a javascript object. ```` +### io.readFileLZ4Stream + +__io.readFileLZ4Stream(aFile) : InputStream__ + +```` +Reads aFile as a LZ4 compressed stream and returns the InputStream to read from. This is useful to read large files that are compressed with LZ4 without decompressing them into memory. +```` ### io.readFileTAR2Stream __io.readFileTAR2Stream(aTARfile, isGzip, aFunc)__ @@ -1989,6 +2092,13 @@ Example: path => (/^\$\.log\.entries\[\d+\]\.request/).test(path)) +```` +### io.unlz4 + +__io.unlz4(anInput) : Map/String__ + +```` +Decompresses anInput using LZ4 compression and returns the decompressed map or string. ```` ### io.unzip @@ -2653,10 +2763,10 @@ Tries to open anURL on the current OS desktop browser. Returns false if it's una ```` ### oPromise -__oPromise(aFunction, aRejFunction) : oPromise__ +__oPromise(aFunction, aRejFunction, useVirtualThreads) : oPromise__ ```` -Custom Promise-like implementation. If you provide aFunction, this aFunction will be executed async in a thread and oPromise object will be immediatelly returned. Optionally this aFunction can receive a resolve and reject functions for to you use inside aFunction to provide a result with resolve(aResult) or an exception with reject(aReason). If you don't call theses functions the returned value will be used for resolve or any exception thrown will be use for reject. You can use the "then" method to add more aFunction that will execute once the previous as executed successfully (in a stack fashion). The return/resolve value from the previous function will be passed as the value for the second. You can use the "catch" method to add aFunction that will receive a string or exception for any exception thrown with the reject functions. You can also provide a aRejFunction that works like a "catch" method as previously described. +Custom Promise-like implementation. If you provide aFunction, this aFunction will be executed async in a thread and oPromise object will be immediatelly returned. Optionally this aFunction can receive a resolve and reject functions for to you use inside aFunction to provide a result with resolve(aResult) or an exception with reject(aReason). If you don't call theses functions the returned value will be used for resolve or any exception thrown will be use for reject. You can use the "then" method to add more aFunction that will execute once the previous as executed successfully (in a stack fashion). The return/resolve value from the previous function will be passed as the value for the second. You can use the "catch" method to add aFunction that will receive a string or exception for any exception thrown with the reject functions. You can also provide a aRejFunction that works like a "catch" method as previously described. Optionally if useVirtualThreads is true, the aFunction will be executed in a virtual thread, otherwise it will be executed in a normal thread. ```` ### oPromise.all diff --git a/docs/reference/threads.md b/docs/reference/threads.md index 8155cf5..38cfdf0 100644 --- a/docs/reference/threads.md +++ b/docs/reference/threads.md @@ -64,6 +64,13 @@ __Threads.addThread(aFunction) : String__ ```` Add a thread to call aFunction as callback. Returns an UUID associated with the thread. The aFunction will receive the corresponding UUID as the first parameter. ```` +### Threads.addVirtualThread + +__Threads.addVirtualThread(aFunction) : String__ + +```` +Adds to the virtual thread executor aFunction to be executed. Returns an UUID associated with the thread. +```` ### Threads.getNumberOfCores __Threads.getNumberOfCores() : number__ @@ -99,6 +106,13 @@ __Threads.initSingleThreadPool(numberOfThreads)__ ```` Uses a thread pool situable for single threads where you can specify the numberOfThreads to use (by default the number of cores). Note: it ignores any previous thread added using addThread; It won't work if any of the other start* or init* methods has been used. ```` +### Threads.initVirtualThreadPerTaskExecutor + +__Threads.initVirtualThreadPerTaskExecutor()__ + +```` +Uses a virtual thread per task executor (Java 21). +```` ### Threads.start __Threads.start()__