Replace WeightOnlyInt8Linear with TorchAO int8_weight_only quantization #3211

Job	Run time
gather-models-cpu	13s
test-tinystories-executorch (16-core-ubuntu)	16m 42s
torchchat-command-load-test (macos-14)	1m 43s
test-tinystories-executorch (macos-14-xlarge)	16m 51s
gather-models-gpu	12s
runner-aoti (16-core-ubuntu)	4m 49s
runner-et (16-core-ubuntu)	6m 44s
runner-aoti (macos-14-xlarge)	4m 1s
compile-gguf (macos-14)	4m 53s
runner-et (macos-14-xlarge)	8m 39s
test-gguf-util (macos-14)	4m 59s
test-mps (macos-m1-stable)	1m 33s
test-mps-dtype (macos-m1-stable)	2m 13s
test-torchao-experimental (macos-14-xlarge)	13m 49s
test-build-runner-et-android / linux-job	10m 40s
test-gpu-aoti-float16 (cuda, stories15M) / linux-job	12m 6s
test-gpu-aoti-float32 (cuda, stories15M) / linux-job	12m 14s
test-gpu-eval-sanity-check (cuda, stories15M) / linux-job	11m 26s
test-gpu-aoti-bfloat16 (cuda, stories15M) / linux-job	12m 56s
test-gpu-compile (cuda, stories15M) / linux-job	22m 52s
test-cpu-compile (x86_64, stories15M)	15m 0s
test-cpu-aoti (x86_64, stories15M)	16m 39s
test-cpu-aoti (aarch64, stories15M)	9m 41s
test-cpu-compile (aarch64, stories15M)	9m 17s
test-cpu-eval-sanity-check (x86_64, stories15M)	9m 16s
test-cpu-eval-sanity-check (aarch64, stories15M)	9m 25s
test-cpu-eval-sanity-check-float32 (x86_64, stories15M)	37m 10s
test-cpu-eval-sanity-check-float32 (aarch64, stories15M)	8m 42s
test-cpu-eval-sanity-check-float16 (x86_64, stories15M)	29m 9s
test-cpu-eval-sanity-check-float16 (aarch64, stories15M)	7m 17s
	5h 21m 11s

Provide feedback