CCM-33842: Capture LiteLLM token usage in spans

[shreyas70]

Summary

• Capture LiteLLM response usage on the SDK-owned span before it ends so UDP ingest receives gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and gen_ai.usage.total_tokens.
• Normalize LiteLLM GenAI semantic convention fields by emitting gen_ai.provider.name instead of gen_ai.system.
• Add LiteLLM response metadata before span end: gen_ai.response.model, gen_ai.response.id, and gen_ai.response.finish_reasons.
• Emit dotted cache/reasoning token attributes when available: gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_creation.input_tokens, and gen_ai.usage.reasoning.output_tokens.
• Do not register LiteLLM's own OTEL callback from the SDK wrapper, since it adds legacy gen_ai.system and duplicates response attributes after the wrapper now enriches the span directly.

Why

llm-model-service Bedrock/LiteLLM responses include token usage, but exported litellm_request spans showed input_tokens=0 and output_tokens=0 because LiteLLM response enrichment happened after the wrapper-owned span had already ended. Copying response metadata before span.end() makes non-streaming LiteLLM spans usable for cost attribution.

Out of scope

• Streaming usage accumulation.
• Tenant/user allocation attributes.
• Resource-level deployment.environment.name changes.

Test plan

• .venv/bin/python -m pytest test/instrumentation/litellm/litellm_instrumentation_test.py

Result: 7 passed.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[t-santoshsahu]

Can you test this, I guess this is required for the instrumentation.