Add support for Mamba-MoE#129

lchu6 wants to merge 10 commits into

mainfoundation-model-stack/fms-fsdp:mainfrom

moefoundation-model-stack/fms-fsdp:moeCopy head branch name to clipboard