@@ -75,18 +75,22 @@ pip install dlinfer-ascend
7575
7676## LMDeploy
7777
78- | | 华为Atlas 800T A2(bf16, w4a16) | 沐曦C500 | 寒武纪云端智能加速卡(开发中) |
79- | --- | --- | --- | --- |
80- | InternLM2.5-7B/20B | √,√ | √ | |
81- | InternLM2-7B/20B | √,√ | √ | |
82- | InternVL2-2B | √,√ | √ | |
83- | InternVL1-5 | √,√ | √ | |
84- | Llama3-8B | √,√ | √ | |
85- | Mixtral8x7B | √,X | √ | |
86- | Qwen2-7B | √,X | √ | |
87- | Qwen2-57B-A14B | √,X | √ | |
88- | CogVLM | √,X | √ | |
89- | CogVLM2 | √,X | √ | |
78+ | | | 华为Atlas 800T A2 | | 沐曦C500 | 寒武纪云端智能加速卡(开发中) |
79+ | --- | --- | --- | --- | --- | --- |
80+ | | bf16(eager) | w4a16(eager) | bf16(graph) | | |
81+ | InternLM2.5-7B/20B | √ | √ | √ | √ | |
82+ | InternLM2-7B/20B | √ | √ | √ | √ | |
83+ | InternVL2-2B | √ | √ | √ | √ | |
84+ | InternVL1-5 | √ | √ | - | √ | |
85+ | Llama3(.1)-8B | √ | √ | √ | √ | |
86+ | Mixtral8x7B | √ | X | √ | √ | |
87+ | Qwen2(.5)-7B | √ | X | √ | √ | |
88+ | Qwen2-57B-A14B | √ | X | - | √ | |
89+ | CogVLM | √ | X | - | √ | |
90+ | CogVLM2 | √ | X | - | √ | |
91+ | glm-4v-9b | √ | - | - | - | |
92+
93+ ‘√’代表测试通过,‘X’代表不支持,‘-’代表未测试
9094
9195### 使用LMDeploy
9296
@@ -113,7 +117,7 @@ if __name__ == "__main__":
113117```
114118
115119> [ !TIP]
116- > 图模式已经支持了Atlas 800T A2。目前,单卡下的LLaMa3-8B/LLaMa2-7B/Qwen2-7B已经通过测试。
120+ > 图模式已经支持了Atlas 800T A2。
117121> 用户可以在离线模式下设定` PytorchEngineConfig ` 中的` eager_mode=False ` 来开启图模式,或者设定` eager_mode=True ` 来关闭图模式。
118122> 在线模式下默认开启图模式,请添加` --eager-mode ` 来关闭图模式。
119123> (启动图模式需要事先` source /usr/local/Ascend/nnal/atb/set_env.sh ` )
0 commit comments