Search results

About total token

Status: Open.

#8 In ASC-Competition/ASC24-LLM-inference-optimization;

· Thewillman opened on Dec 25, 2023

Want a discussion area here in this repository for we competitors to discuss and learn from each other.

Status: Open.

#6 In ASC-Competition/ASC24-LLM-inference-optimization;

· Kevin-shihello-world opened on Dec 21, 2023

How about some ml-based method?

Status: Open.

#5 In ASC-Competition/ASC24-LLM-inference-optimization;

· Thewillman opened on Dec 19, 2023

Is that we should fully use all 70b parameter? I mean can we use some skill like discrimination to make it smaller Or use another some small model to help its inference Or separate the whole big model into some small part and use Moe to help it move faster?

Status: Open.

#1 In ASC-Competition/ASC24-LLM-inference-optimization;

· Kevin-shihello-world opened on Dec 16, 2023