ilotoki_thu
/
alpaca_lora_4bit
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
168 Commits 1 Branch 0 Tags 599 KiB
Commit Graph

14 Commits

Author SHA1 Message Date
John Smith 4261bd8070 add xformers support 2023-04-12 12:59:44 +08:00
John Smith 8cf3bd4086 add g_idx support on cuda backend 2023-04-09 12:26:22 +08:00
John Smith dba3773b30 add triton backend support for v2 model 2023-04-07 15:34:06 +08:00
John Smith 9351f49542 merge pull request in new branch 2023-04-07 10:40:24 +08:00
yamashi 95cd390d25
Update Finetune4bConfig.py 2023-04-07 00:43:15 +02:00
Andrey Glushenkov f20570343f
GPTQv2 support 
GPTQv2 support.
1. Adds dependency on `triton`
2. Refactors autograd_4bit to include both GPTQv1 and GPTQv2
3. Introduces new environment variable GPTQ_VERSION to select autograd_4bit version
4. Fixes triton kernels
5. Matrix multiplications are in fp16
 2023-04-06 02:29:36 +03:00
Wing Lian b7361da58a better multi-gpu support, support gpt4all training data 2023-03-29 11:21:47 -04:00
John Smith 1c02d4262d add resume checkpoint to continue a training 2023-03-29 14:35:39 +08:00
John Smith 2a1cb42966 add padding support as an option 2023-03-29 11:20:16 +08:00
John Smith 0768d0fdff update finetune data format 2023-03-28 21:45:33 +08:00
John Smith bff039de95 add v2 model support 2023-03-28 20:33:55 +08:00
kooshi 8e471516b8
distributed data parallelism with torchrun 2023-03-24 23:56:06 -05:00
Andrey Glushenkov 397f5041c3
Reflect last changes in main 
Reflect commits:
4906961bf1
60b227d0ba
 2023-03-24 15:46:03 +03:00
Andrey Glushenkov 50dbb101e9
Refactor finetune.py 
1. Add command line arguments support
2. Add Stanford Alpaca-like dataset support. Used code from - https://github.com/tloen/alpaca-lora
3. Fix LoRA pre-train application
 2023-03-24 14:15:07 +03:00