All issues

Multimodal MoE

#61

· xrsrke opened

on Dec 11, 2023

Distributed CLIP

#60

· xrsrke opened

on Dec 10, 2023

DiLoCo replication (DiLoCo: Distributed Low-Communication Training of Language Models)

#59

· xrsrke opened

on Dec 10, 2023

Making pipeline parallelism compatible with `transformers`

#52

· xrsrke opened

on Dec 1, 2023

End-to-end FP8 training

#45

· xrsrke opened

on Nov 26, 2023

Automatic module mapping using torch.fx

good first issue

#40

· xrsrke opened

on Nov 22, 2023

·

Tensor Parallelism

#37

· 3outeille opened

on Nov 20, 2023

Distributed Logger

good first issue

#33

· xrsrke opened

on Nov 14, 2023

Save and load checkpoints

good first issue

#29

· xrsrke opened

on Nov 8, 2023

Lazy initialization of massive models

#25

· xrsrke opened

on Oct 27, 2023

Sequence Parallelism

#22

· xrsrke opened

on Oct 25, 2023

Mixture of Experts

#19

· xrsrke opened

on Oct 25, 2023