ilotoki_thu
/
alpaca_lora_4bit
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Update README.md

This commit is contained in:
John Smith 2023-03-23 16:31:49 +08:00 committed by GitHub
parent 44978669cf
commit 619a177fbb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -6,12 +6,16 @@ Made some adjust for the code in peft and gptq for llama, and make it possible f
# Update Logs # Update Logs
* Resolved numerically unstable issue * Resolved numerically unstable issue
<br> <br>
* Reconstruct fp16 matrix from 4bit data and call torch.matmul largely increased the inference speed. * Reconstruct fp16 matrix from 4bit data and call torch.matmul largely increased the inference speed.
<br> <br>
* Added install script for windows and linux. * Added install script for windows and linux.
<br> <br>
* Added Gradient Checkpointing. Now It can finetune 30b model 4bit on a single GPU with 24G VRAM. (finetune.py updated)
* Added Gradient Checkpointing. Now It can finetune 30b model 4bit on a single GPU with 24G VRAM with Gradient Checkpointing enabled. (finetune.py updated) (but would reduce training speed, so if having enough VRAM this option is not needed)
<br> <br>
* Added install manual by s4rduk4r * Added install manual by s4rduk4r
<br> <br>