This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Integrate distributed inference into torchchat cli#1327

Merged

lessw2020 merged 25 commits intopytorch:mainfrom

mreso:refactor/dist_run

Oct 25, 2024

Commits on Oct 15, 2024

Commits on Oct 17, 2024

add elastic_launch
lessw2020
committed
working, can now launch from cli
lessw2020
committed
Remove numpy < 2.0 pin to align with pytorch (#1301)

larryliu0820
authored and
mreso
committed
Update torchtune pin to 0.4.0-dev20241010 (#1300)

authored and
mreso
committed
Unbreak gguf util CI job by fixing numpy version (#1307)

larryliu0820
authored and
mreso
committed
Remove apparently-unused import torchvision in model.py (#1305)

authored and
mreso
committed
remove global var for tokenizer type + patch tokenizer to allow list of sequences
mreso
committed
make pp tp visible in interface
mreso
committed
Add llama 3.1 to dist_run.py
mreso
committed

Commits on Oct 18, 2024

[WIP] Move dist inf into its own generator
mreso
committed

Commits on Oct 21, 2024

Add initial generator interface to dist inference
mreso
committed

Commits on Oct 23, 2024

Commits on Oct 24, 2024