Optimize implementation, add multi stream support - #6

Optimize implementation, add multi stream support#6

timkhronos merged 5 commits into

MSAtimkhronos/llama.cpp:MSAfrom

10%Decode-speed,-10%-PP-speed,-20%-smalle-compute-buffer,-F16-indexer-cache-and-Multi-stream-supporttimkhronos/llama.cpp:10%Decode-speed,-10%-PP-speed,-20%-smalle-compute-buffer,-F16-indexer-cache-and-Multi-stream-supportCopy head branch name to clipboard