A temporary workaround to make Anthropic FIM working with Continue

We need to run the normalizer and denormalizer even if there is nothing
in the output pipeline for Anthropic because it uses a special format
for messages. However, we can't run the pipeline for all providers
because we are missing a normalizer/denormalizer for llama.cpp. We need
to add that, but let's add a temporary hack to get FIM with Anthropic
working.

Co-authored-by: Pankaj Telang <pankaj@stacklok.com>
Co-authored-by: Alejandro Ponce <aponcedeleonch@stacklok.com>

Author: 3 people

src/codegate/providers/base.py

@@ -81,7 +81,9 @@ async def _run_output_stream_pipeline(
         if out_pipeline_processor is None:
             logger.info("No output pipeline processor found, passing through")
             return model_stream
-        if len(out_pipeline_processor.pipeline_steps) == 0:
+
+        # HACK! for anthropic we always need to run the output FIM pipeline even if empty to run the normalizers
+        if len(out_pipeline_processor.pipeline_steps) == 0 and self.provider_route_name != "anthropic":
             logger.info("No output pipeline steps configured, passing through")
             return model_stream