Skip to content

Commit 71ebc50

Browse files
authored
Expose static llama in OSS
Differential Revision: D88875745 Pull Request resolved: #16184
1 parent ee236cb commit 71ebc50

File tree

4 files changed

+870
-0
lines changed

4 files changed

+870
-0
lines changed

.ci/scripts/test_ane_static_llama.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,13 @@ pushd $EXECUTORCH_ROOT/examples/apple/coreml/llama
2828
# Download stories llama110m artifacts
2929
download_stories_model_artifacts
3030

31+
# Test static ANE llama model
32+
python export_static_llm_coreml.py --checkpoint stories110M.pt --params params.json --output model.pte
33+
34+
# The ANE cannot run in github CI
35+
# python run_static_llm.py --model model.pte --params params.json --tokenizer tokenizer.model --prompt "Once upon a time," --lookahead
36+
37+
# Test export of deprecated model
3138
python export.py -n model.pte -p params.json -c stories110M.pt --seq_length 32 --max_seq_length 64 --dtype fp16 --coreml-quantize c4w --embedding-quantize 4,32
3239

3340
popd

0 commit comments

Comments
 (0)