Multi LLM Support

[harryfrzz]

Add support for using multiple LLMs and switching between them at runtime without restarting sassyshell. It should automatically pull the LLM from huggingface/ollama and load it up without exiting sassyshell.
If you assign this issue to me, I can take it on.
@ParthJain18

[ParthJain18]

Hey @harryfrzz, thanks for opening the first issue! This is a great idea, we can have a sub-command to switch between LLM providers, see the current status, etc.
I'm assigning this to you.

[harryfrzz]

Hey @harryfrzz, thanks for opening the first issue! This is a great idea, we can have a sub-command to switch between LLM providers, see the current status, etc. I'm assigning this to you.

Thanks!

[harryfrzz]

Hey,
I noticed that your project currently uses the Gemini API for reasoning. I suggest adding a model-switching mechanism that allows switching between local LLMs (via LangChain & Ollama) and online inference providers like Gemini, OpenAI, and Anthropic.
I'll go ahead with implementing this feature if you give me a heads up! @ParthJain18

[ParthJain18]

Currently, I have only tested this on Gemini but since we are using Langchain, using other providers shouldn't require many changes. Afaik, we only need to set the model name, provider and the correct API key in config.py file.

But to switch between them, and to effectively manage the correct API key, we might need to create a dedicated sub-command. If you face any trouble with managing API keys, we can remove the current way of passing the API key directly during llm initialization and rely on environment variables instead.