Cryptographic inference for large language model via secret sharing - CHENG Ke; XIA Yuheng; DAI Chuanyun; FU Jiaxuan; ZHU Xinghui; SHEN Yulong (Tongxin xuebao)
Inference services based on large language models may lead to the leakage of user input hints to the server or proprietary model weights to the user. Cryptographic techniques such as secure multi-party computation and homomorphic encryption provide feasible solutions to the above problems, but they are still difficult to apply practically to the task of inference over large language models due to the excessive computational and communication overhead. Based on this, a lightweight secret-sharing-based cryptographic inference scheme for large language models was proposed, by which inference could be performed efficiently and accurately while ensuring that neither user inputs nor model parameters were revealed. The experimental results show that the proposed scheme improves the efficiency by 1.2~10 times and reduces the communication cost by 20%~90% compared with the existing state-of-the-art works.