[Feature request] MLC tool calling & Android equivalent of Apple Intelligence

[rogueturnip]

Just wanted to ask if there were any estimates on the android support? The tooling is amazing for apple and would really like to be able to do the same for android with the fall back to external LLMs.

[grabbou]

Unfortunately we are limited due to incomplete/missing presence of on-device Android APIs. Is there any particular device/hardware that you would like us to experiment with?

[rogueturnip]

I actually think if you were able to get the MLC working fully with tools and such, the react-native-speech-recognition library is really good for iOS and android so taht could be used.

For speech, ultimately using something like Kokoro would be really useful. I did get this working with onnx on react native but the phonemics isn't optimal.

You might also want to look at the AI sdk providers structure for things so you follow the other provider standards
https://ai-sdk.dev/providers/community-providers/built-in-ai

[weemundo]

Hi @grabbou - stumbled upon https://developers.google.com/ml-kit/genai APIs on android (ref: https://github.com/googlesamples/mlkit/tree/master/android/genai).

Would this be the Android equivalent of the on-device AI?

[artus9033]

Hi @rogueturnip ! Unfortunately, MLC does not currently support tool calling, but they have it on their roadmap. Therefore, this issue will be blocked by mlc-ai/mlc-llm#2267

[artus9033]

Hi @weemundo the ML Kit you linked seems to be a more basic variant of what Apple Intelligence offers. We could take a look into that in the future.