image inference guide

[philschmid]

No description provided.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[jeffboudier]

is there a recommended range of image dimensions that makes sense for most multimodal models?

[alvarobartt]

LGTM, thanks!

[alvarobartt]

Suggested change

	* Using a URL
	```python
	from huggingface_hub import InferenceClient
	import base64
	client = InferenceClient(base_url="http://localhost:8080", api_key="-")
	# Using a URL
	image_url = "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
	chat_completion = client.chat.completions.create(
	messages=[
	{
	"role": "user",
	"content": [
	{
	"type": "text",
	"text": "Describe this image in detail.",
	},
	{
	"type": "image_url",
	"image_url": {"url": image_url},
	},
	],
	},
	],
	temperature=0.7,
	max_tokens=128,
	)
	print(chat_completion.choices[0].message.content)
	```
	##### Using a URL
	```python
	from huggingface_hub import InferenceClient
	import base64
	client = InferenceClient(base_url="http://localhost:8080", api_key="-")
	image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png"
	chat_completion = client.chat.completions.create(
	messages=[
	{
	"role": "user",
	"content": [
	{
	"type": "text",
	"text": "Describe this image in detail.",
	},
	{
	"type": "image_url",
	"image_url": {"url": image_url},
	},
	],
	},
	],
	temperature=0.7,
	max_tokens=128,
	)
	print(chat_completion.choices[0].message.content)

[alvarobartt]

Same here, to use a Hugging Face image instead (in case the other link is eventually broken / not working):

Suggested change

	image_url = "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" # or your base64 data URL
	image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png" # or your base64 data URL

[alvarobartt]

Same here:

Suggested change

	"image_url":{"url":"https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"}
	"image_url":{"url":"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png"}

[alvarobartt]

Suggested change

	* Using a local image (base64 encoded)
	##### Using a local image (base64 encoded)