ilotoki_thu/alpaca_lora_4bit
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

add test result

Browse Source
This commit is contained in:
John Smith
2023-04-26 18:02:43 +08:00
parent 73f51188bf
commit 2f704b93c9
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -9,6 +9,10 @@ pip install git+https://github.com/johnsmith0031/alpaca_lora_4bit@winglian-setup
Better inference performance with text_generation_webui, about <b>40% faster</b> Better inference performance with text_generation_webui, about <b>40% faster</b>
Simple expriment results:<br>
7b model with groupsize=128 no act-order<br>
improved from 13 tokens/sec to 20 tokens/sec
<b>Step:</b> <b>Step:</b>
1. run model server process 1. run model server process
2. run webui process with monkey patch 2. run webui process with monkey patch