Skip to content

Commit 9df9109

Browse files
committed
Add control over including schemas for structured output
And note that by default, the implementation may do so. Closes #125.
1 parent 66ab78e commit 9df9109

File tree

2 files changed

+13
-0
lines changed

2 files changed

+13
-0
lines changed

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -314,6 +314,18 @@ The returned value will be a string that matches the input `RegExp`. If the user
314314

315315
If a value that is neither a `RegExp` object or a valid JSON schema object is given, the method will error with a `TypeError`.
316316

317+
By default, the implementation may include the schema or regular expression as part of the message sent to the underlying language model, which will use up some of the [input quota](#tokenization-context-window-length-limits-and-overflow). You can measure how much it will use up by passing the `responseConstraint` option to `session.measureInputUsage()`. If you want to avoid this behavior, you can use the `omitResponseConstraintInput` option. In such cases, it's strongly recommended to include some guidance in the prompt string itself:
318+
319+
```js
320+
const result = await session.prompt(`
321+
Summarize this feedback into a rating between 0-5, only outputting a JSON
322+
object { rating }, with a single property whose value is a number:
323+
The food was delicious, service was excellent, will recommend.
324+
`, { responseConstraint: schema, omitResponseConstraintInput: true });
325+
```
326+
327+
If `omitResponseConstraintInput` is set to `true` without `responseConstraint` set, then the method will error with a `TypeError`.
328+
317329
### Constraining responses by providing a prefix
318330

319331
As discussed in [Customizing the role per prompt](#customizing-the-role-per-prompt), it is possible to prompt the language model to add a new `"assistant"`-role response in addition to a previous one. Usually it will elaborate on its previous messages. For example:

index.bs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ dictionary LanguageModelCreateOptions : LanguageModelCreateCoreOptions {
9494

9595
dictionary LanguageModelPromptOptions {
9696
object responseConstraint;
97+
boolean omitResponseConstraintInput = false;
9798
AbortSignal signal;
9899
};
99100

0 commit comments

Comments
 (0)