[Bug]: O1 models not supported

[yueqianh]

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
• I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

O1 models return the following error:
"Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead."

Steps to reproduce

Use an Azure OpenAI hosted O1 model.

Expected Behavior

No response

GraphRAG Config Used

# Paste your config here

Logs and screenshots

No response

Additional Information

• GraphRAG Version: 1.2.0
• Operating System:
• Python Version:
• Related Issues:

[rjurney]

Me too!

settings.yaml:

### This config file contains required core defaults that must be set, along with a handful of common optional settings.
### For a full list of available settings, see https://microsoft.github.io/graphrag/config/yaml/

### LLM settings ###
## There are a number of settings to tune the threading and token limits for LLM calls - check the docs.

encoding_model: cl100k_base # this needs to be matched to your model!

llm:
  api_key: ${GRAPHRAG_API_KEY} # set this in the generated .env file
  type: openai_chat # or azure_openai_chat
  model: o1
  model_supports_json: true # recommended if this is available for your model.
  # audience: "https://cognitiveservices.azure.com/.default"
  # api_base: https://<instance>.openai.azure.com
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>

parallelization:
  stagger: 0.3
  # num_threads: 50

async_mode: threaded # or asyncio

embeddings:
  async_mode: threaded # or asyncio
  vector_store:
    type: lancedb # one of [lancedb, azure_ai_search, cosmosdb]
    db_uri: "output/lancedb"
    collection_name: default
    overwrite: true
  llm:
    api_key: ${GRAPHRAG_API_KEY}
    type: openai_embedding # or azure_openai_embedding
    model: text-embedding-3-large
    # api_base: https://<instance>.openai.azure.com
    # api_version: 2024-02-15-preview
    # audience: "https://cognitiveservices.azure.com/.default"
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>

### Input settings ###

input:
  type: file # or blob
  file_type: text # or csv
  base_dir: "data/txt"
  file_encoding: utf-8
  file_pattern: ".*\\.txt$"

chunks:
  size: 1200
  overlap: 100
  group_by_columns: [id]

### Storage settings ###
## If blob storage is specified in the following four sections,
## connection_string and container_name must be provided

cache:
  type: file # one of [blob, cosmosdb, file]
  base_dir: "cache"

reporting:
  type: file # or console, blob
  base_dir: "logs"

storage:
  type: file # one of [blob, cosmosdb, file]
  base_dir: "output"

## only turn this on if running `graphrag index` with custom settings
## we normally use `graphrag update` with the defaults
update_index_storage:
  # type: file # or blob
  # base_dir: "update_output"

### Workflow settings ###

skip_workflows: []

entity_extraction:
  prompt: "prompts/entity_extraction.txt"
  entity_types: [organization, person, geo, event]
  max_gleanings: 1

summarize_descriptions:
  prompt: "prompts/summarize_descriptions.txt"
  max_length: 500

claim_extraction:
  enabled: false
  prompt: "prompts/claim_extraction.txt"
  description: "Any claims or facts that could be relevant to information discovery."
  max_gleanings: 1

community_reports:
  prompt: "prompts/community_report.txt"
  max_length: 2000
  max_input_length: 8000

cluster_graph:
  max_cluster_size: 10

embed_graph:
  enabled: false # if true, will generate node2vec embeddings for nodes

umap:
  enabled: false # if true, will generate UMAP embeddings for nodes (embed_graph must also be enabled)

snapshots:
  graphml: false
  embeddings: false
  transient: false

### Query settings ###
## The prompt locations are required here, but each search method has a number of optional knobs that can be tuned.
## See the config docs: https://microsoft.github.io/graphrag/config/yaml/#query

local_search:
  prompt: "prompts/local_search_system_prompt.txt"

global_search:
  map_prompt: "prompts/global_search_map_system_prompt.txt"
  reduce_prompt: "prompts/global_search_reduce_system_prompt.txt"
  knowledge_prompt: "prompts/global_search_knowledge_system_prompt.txt"

drift_search:
  prompt: "prompts/drift_search_system_prompt.txt"
  reduce_prompt: "prompts/drift_search_reduce_prompt.txt"

basic_search:
  prompt: "prompts/basic_search_system_prompt.txt"

[spikezz]

Probably not openai_chat type anymore, but more likely the o1/o3 reasoner interface are not implemented yet,

[komal-blkmmb]

I got the same error trying to use using o3-mini model.

[tr5tn]

This change should be straight-forward. The API implements a different parameter for max tokens for the o-series models. Instead of max_tokens, it's called max_completion_tokens. You see the same error when interacting directly with the API. No idea why they made the breaking change, but it's here to stay it would seem, and it would be great to be able to use the reasoning models.

[ricky-bug]

You can check the openai official website, they will tell you whether the model support json. If not, change the setting:
model_supports_json: fasle

[natoverse]

This is on our radar and being investigated, but it's not quite as simple as just updating the parameter. There are areas within GraphRAG where we request (and rely on) a specific response length using max_tokens (for example, using logit bias for gleanings). The new reasoning parameter doesn't provide a way to control the balance between actual generated response tokens and reasoning tokens, so we have to work through different approaches to achieve the functionality.

[majdalsado]

Bumping this issue. Seems all new OpenAI models are not being supported (tested gpt-4.1, gpt-4.1-mini, o3-mini, o4-mini). These models are getting much better and it'd be a shame if we don't get support for these new models.

[natoverse]

PR #1841 implements this, we've just been running through a bunch of review/eval to ensure quality since we made changes to the prompts. Should be merged/released imminently.

[natoverse]

This is deployed in 2.2.0. Note that there can still be issues with 4.1 because tiktoken has not added it to the encoding model lookup yet. You can work around this by specifying the encoding model directly in the model config so we don't try to look it up from tiktoken.

[natoverse]

We also now have a docs page dedicated to the nuances of configuring models: https://microsoft.github.io/graphrag/config/models/