Try HuggingFace instead of OpenAI

[wrongu]

Many of the features that make guidance cool are not supported by the OpenAI API.

Guidance supports using any of the HuggingFace open-source instruct-tuned models.

To-do:

• add transformers dependency
• read up on pretrained models and pick a few to try out
• try em out and see if they're unusably slow or memory hungry or cpu hungry or just bad at doing tasks
• add choice of llm to the config
• go write a bunch of guidance programs that make use of the actual guidance features (Build a better prompt for general scientific writing #4)

[j6k4m8]

SUPER +1 — maybe make this an optional dependency, so you can run this with llm4papers[openai] or llm4papers[hf], or llm4papers[all]..? that way users can opt into the backends they want

[wrongu]

Does "add choice of llm to the config" meet that criterion for you?

[j6k4m8]

yes besides installing subsets of dependencies (extras)