John Smith
|
f3a25342e1
|
fix device_map bug when using lora_apply_dir
|
2023-03-31 19:44:36 +08:00 |
Wing Lian
|
8791eaee9a
|
fix gpt4all training to more closely match the released logic, other small fixes and optimizations
|
2023-03-30 22:40:40 -04:00 |
|
Wing Lian
|
b7361da58a
|
better multi-gpu support, support gpt4all training data
|
2023-03-29 11:21:47 -04:00 |
|
John Smith
|
1c02d4262d
|
add resume checkpoint to continue a training
|
2023-03-29 14:35:39 +08:00 |
|
John Smith
|
2a1cb42966
|
add padding support as an option
|
2023-03-29 11:20:16 +08:00 |
|
John Smith
|
0768d0fdff
|
update finetune data format
|
2023-03-28 21:45:33 +08:00 |
|
John Smith
|
211af574b6
|
fix bug
|
2023-03-28 21:12:51 +08:00 |
|
John Smith
|
bff039de95
|
add v2 model support
|
2023-03-28 20:33:55 +08:00 |
|
Wing Lian
|
62e54ac1c7
|
backwards support for pre-py3.10, add datasets requirement used in train
|
2023-03-27 16:08:20 -04:00 |
Star Dorminey
|
399c3d124e
|
Tested and should be ready!
|
2023-03-25 20:52:38 -07:00 |
kooshi
|
8e471516b8
|
distributed data parallelism with torchrun
|
2023-03-24 23:56:06 -05:00 |
|
kooshi
|
2bc64597aa
|
model parallelism
|
2023-03-24 23:03:43 -05:00 |
|
John Smith
|
0879580006
|
Merge branch 'main' into finetune-refactor
|
2023-03-25 10:29:02 +08:00 |
Andrey Glushenkov
|
397f5041c3
|
Reflect last changes in main
Reflect commits:
4906961bf1
60b227d0ba
|
2023-03-24 15:46:03 +03:00 |
|
Andrey Glushenkov
|
50dbb101e9
|
Refactor finetune.py
1. Add command line arguments support
2. Add Stanford Alpaca-like dataset support. Used code from - https://github.com/tloen/alpaca-lora
3. Fix LoRA pre-train application
|
2023-03-24 14:15:07 +03:00 |
|
John Smith
|
60b227d0ba
|
fix minor bug
|
2023-03-23 08:43:18 +00:00 |
|
John Smith
|
44978669cf
|
Add gradient checkpointing
|
2023-03-23 08:25:29 +00:00 |
|
John Smith
|
dc036373b2
|
add more scripts and adjust code for transformer branch
|
2023-03-22 04:09:04 +00:00 |