Update readme.

Author: quic-zhanweiw

README.md

@@ -4,7 +4,7 @@
 
 #### QAI AppBuilder
 Quick AI Application Builder(this repository) is also referred to as *QAI AppBuilder* in the source and documentation. QAI AppBuilder is extension for Qualcomm® AI Runtime SDK. We need some libraries in Qualcomm® AI Runtime SDK for using QAI AppBuilder. <br>
-QAI AppBuilder is designed for developer to using Qualcomm® AI Runtime SDK to execute model on Windows on Snapdragon(WoS) and Linux platforms easily. We encapsulated Qualcomm® AI Runtime SDK APIs to several simple APIs for loading the models to CPU or HTP and executing inference.
+QAI AppBuilder is designed for developer to using Qualcomm® AI Runtime SDK to execute model on Windows on Snapdragon(WoS) and Linux platforms easily. We encapsulated Qualcomm® AI Runtime SDK APIs to several simple APIs for loading the models to CPU and HTP and executing inference.
 
 #### Qualcomm® AI Runtime SDK
 
@@ -27,37 +27,41 @@ Developers can use QAI AppBuilder in both C++ and Python projects <br>
 • Faster for testing models. <br>
 • Plenty of sample code. <br>
 
-Using the Python extensions with ARM64 Python will make it easier for developers to build GUI app for Windows on Snapdragon(WoS) platforms. Python 3.12.6 ARM64 version has support for following modules: PyQt6, OpenCV, Numpy, PyTorch*, Torchvision*, ONNX*, ONNX Runtime*. Developers can design apps that benefit from rich Python ecosystem. <br>
+** Support ARM64 Windows, Linux and Ubuntu (e.g.: X Elite Windows, QCS8550 Linux and QCM6490 Ubuntu)*
 
-**PyTorch, Torchvision, ONNX, ONNX Runtime: need to compile from source code.* <br>
-**Also support using x64 Python to run QNN mode on WoS HTP, with this, we can install all the Python extension directly (Refer to the samples code here for detail: https://github.com/quic/ai-engine-direct-helper/tree/main/samples/python))* <br>
-**Support ARM64 Windows, Linux and Ubuntu (e.g.: X Elite Windows, QCS8550 Linux and QCM6490 Ubuntu)*
+## Environment Setup
+Refere to [python.md](docs/python.md) on how to setup Python environment for using QAI AppBuilder on Windows on Snapdragon (WoS) platforms.
+
+## Samples
+We have several [samples](samples/) which can be run directly:<br>
+1. [Sample code](samples/python/README.md): Guide to run several [AI-Hub](https://aihub.qualcomm.com/compute/models) models throug sample code.
+2. OpenAI Compatibility API Service(LLM Service):<br>
+2.1 [Python based service](samples/genie/python/README.md): Guide to run OpenAI compatibility API services developed with python.<br>
+2.2 [C++ based service](samples/genie/c++/README.md): Guide to run OpenAI compatibility API services developed with C++.<br>
+3. [WebUI samples](samples/webui/README.md): Guide to run several WebUI based AI applications.
 
 ## Components
 There're two ways to use QAI AppBuilder:
 ### 1. Using the QAI AppBuilder C++ libraries to develop C++ based AI application.
 Download prebuild binary package *QAI_AppBuilder-win_arm64-{Qualcomm® AI Runtime SDK version}-Release.zip* to get these files: https://github.com/quic/ai-engine-direct-helper/releases
 
-**libappbuilder.dll {libappbuilder.lib, LibAppBuilder.hpp}** –– C++ projects can use this lib to run models in HTP.
-**QAIAppSvc.exe** –– Due to HTP limitations, we can only load models smaller than 4GB in one process. This app is used to help us load the models in new processes(Multiple processes can be created) and inference to avoid HTP restrictions. [*Depress: the above limitation has been fixed.*]
-
 ### 2. Using the QAI AppBuilder Python binding extension to develop Python based AI application.
-Download Python extension *qai_appbuilder-{version}-cp312-cp312-win_arm64.whl* and install it with the command below:
+Download Python extension *qai_appbuilder-{version}-cp312-cp312-win_amd64.whl* and install it with the command below:
 https://github.com/quic/ai-engine-direct-helper/releases
 
 ```
-pip install qai_appbuilder-{version}-cp312-cp312-win_arm64.whl
+pip install qai_appbuilder-{version}-cp312-cp312-win_amd64.whl
 ```
 
 ## User Guide
-Please refere to [User Guide](docs/user_guide.md) on how to use QAI AppBuilder in your project.
+Refere to [User Guide](docs/user_guide.md) on how to use QAI AppBuilder to program AI application.
 
 ## Build
-Build project with Visual Studio 2022 on WoS device:<br>
+Build QAI AppBuilder from source with Visual Studio 2022 on WoS device:<br>
 - Install Visual Studio 2022: 
   - https://docs.qualcomm.com/bundle/publicresource/topics/80-62010-1/Install-Visual-Studio-2022.html?product=Windows%20on%20Snapdragon
-- Install Python-3.12.6 ARM64: 
-  - https://www.python.org/ftp/python/3.12.6/python-3.12.6-arm64.exe
+- Install x64 version Python-3.12.6: 
+  - https://www.python.org/ftp/python/3.12.8/python-3.12.8-amd64.exe
 - Use the commands below to install Python dependency: 
 ```
 pip install wheel setuptools pybind11
@@ -76,7 +80,7 @@ cd C:\Source\ai-engine-direct-helper
 python setup.py bdist_wheel
 
 # Install the extension:
-pip install dist\qai_appbuilder-2.34.0-cp312-cp312-win_arm64.whl
+pip install dist\qai_appbuilder-2.34.0-cp312-cp312-win_amd64.whl
 ```
 
 ## License

docs/python.md

@@ -0,0 +1,37 @@
+# python(x64)
+
+## Introduction 
+This guide helps developers setup Python environment for using QAI AppBuilder on Windows on Snapdragon (WoS) platforms.
+
+## Setting Up QAI AppBuilder Python Environment:
+
+### Step 1: Install Dependencies
+Download and install [git](https://github.com/dennisameling/git/releases/download/v2.47.0.windows.2/Git-2.47.0.2-arm64.exe) and [x64 Python 3.12.8](https://www.python.org/ftp/python/3.12.8/python-3.12.8-amd64.exe)
+
+*Make sure to check 'Add python.exe to PATH' while install Python*
+
+### Step 2: Install basic Python dependencies:
+Run below commands in Windows terminal:
+```
+pip install requests wget tqdm importlib-metadata
+```
+
+### Step 3: Download QAI AppBuilder repository:
+Run below commands in Windows terminal:
+```
+git clone https://github.com/quic/ai-engine-direct-helper.git
+```
+
+### Step 4: Setup QAI AppBuilder Python Environment:
+Run below commands in Windows terminal:
+```
+cd ai-engine-direct-helper\samples
+python python\setup.py
+```
+
+### Step 5: Now you can refer to the following contents to experience running the AI model on the WoS platform: <br>
+1. Run [sample code](../samples/python/README.md) for the models from Qualcomm [AI-Hub](https://aihub.qualcomm.com/compute/models).
+2. Run OpenAI Compatibility API Service(LLM Service):<br>
+2.1 [Python based service](../samples/genie/python/README.md)<br>
+2.2 [C++ based service](../samples/genie/c++/README.md)<br>
+3. Run [WebUI samples](../samples/webui/README.md).

docs/python_arm64.md

@@ -0,0 +1,4 @@
+Using the Python extensions with ARM64 Python will make it easier for developers to build GUI app for Windows on Snapdragon(WoS) platforms. Python 3.12.6 ARM64 version has support for following modules: PyQt6, OpenCV, Numpy, PyTorch*, Torchvision*, ONNX*, ONNX Runtime*. Developers can design apps that benefit from rich Python ecosystem. <br>
+
+**PyTorch, Torchvision, ONNX, ONNX Runtime: need to compile from source code.* <br>
+

docs/user_guide.md

@@ -6,17 +6,6 @@
 
 <b>We need below libraries from QNN SDK for using AppBuilder on Snapdragon X Elite(Windows on Snapdragon device):</b>
 
-If use ARM64 Python, use the libraries below from QNN SDK(ARM64 Python has better performance in Snapdragon X Elite platform):
-```
-C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnHtp.dll  (backend for running model on HTP)
-C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnCpu.dll  (backend for running model on CPU)
-C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnHtpPrepare.dll
-C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnSystem.dll
-C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnHtpV73Stub.dll
-C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\hexagon-v73\unsigned\libQnnHtpV73Skel.so
-C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\hexagon-v73\unsigned\libqnnhtpv73.cat
-```
-
 If use x64 Python, use the libraries below from QNN SDK:
 ```
 C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\arm64x-windows-msvc\QnnHtp.dll  (backend for running model on HTP)
@@ -28,6 +17,17 @@ C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\hexagon-v73\unsigned\libQnnHtpV73Ske
 C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\hexagon-v73\unsigned\libqnnhtpv73.cat
 ```
 
+If use ARM64 Python, use the libraries below from QNN SDK(ARM64 Python has better performance in Snapdragon X Elite platform):
+```
+C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnHtp.dll  (backend for running model on HTP)
+C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnCpu.dll  (backend for running model on CPU)
+C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnHtpPrepare.dll
+C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnSystem.dll
+C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\aarch64-windows-msvc\QnnHtpV73Stub.dll
+C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\hexagon-v73\unsigned\libQnnHtpV73Skel.so
+C:\Qualcomm\AIStack\QAIRT\{SDK Version}\lib\hexagon-v73\unsigned\libqnnhtpv73.cat
+```
+
 We can copy these libraries to one folder. E.g.: ```C:\<Project Name>\qnn\``` <br>
 
 ### 2. Python and common python extensions: 

samples/genie/python/README.md

@@ -4,9 +4,9 @@
 This sample helps developers use QAI AppBuilder + Python to build Genie based Open AI compatibility API service on Windows on Snapdragon (WoS) platform.
 
 ## Setting Up Environment For Service:
-### Step 1: Install basic dependencies
-Refer to following link to setup basic dependencies: <br>
-https://github.com/quic/ai-engine-direct-helper/blob/main/samples/python/README.md#setting-up-qai-appbuilder-python-environment <br>
+
+### Step 1: Install Dependencies
+Refer to [python.md](../../../docs/python.md) on how to setup x64 version Python environment.
 
 ### Step 2: Install basic Python dependencies for service
 Run following commands in Windows terminal:
@@ -48,4 +48,4 @@ python genie\python\GenieAPIClientImage.py --prompt "<Your prompt>"
 | Phi 3.5 mini * | [model files](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/phi_3_5_mini_instruct/v1/snapdragon_x_elite/models.zip)<br>[tokenizer.json](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/resolve/main/tokenizer.json?download=true) |
 
 *. For Phi-3.5-Mini-Instruct model, to see appropriate spaces in the output, remove lines 193-196 (Strip rule) in the tokenizer.json file.<br>
-**. Refer to here to [setup Stable Diffusion v2.1 models](../../python/README.md) before run 'GenieAPIService.py'.
+**. Refer to [setup Stable Diffusion v2.1 models](../../python/README.md) before run 'GenieAPIService.py' (Our Python version 'GenieAPIService.py' support generating image, it depends on Stable Diffusion v2.1 sample code.)

samples/python/README.md

@@ -1,50 +1,52 @@
 # README
 
 ## Introduction 
-This guide helps developers use QAI AppBuilder with the QNN SDK to execute models on Windows on Snapdragon (WoS) platforms.
+This guide helps developers setup Python environment for using QAI AppBuilder to run sample code on Windows on Snapdragon (WoS) platforms.
 
-## Setting Up QAI AppBuilder Python Environment:
+## Setting Up QAI AppBuilder Python Environment
 
 ### Step 1: Install Dependencies
-Download and install [git](https://github.com/dennisameling/git/releases/download/v2.47.0.windows.2/Git-2.47.0.2-arm64.exe) and [x64 Python 3.12.8](https://www.python.org/ftp/python/3.12.8/python-3.12.8-amd64.exe)
+Refer to [python.md](../../docs/python.md) on how to setup x64 version Python environment.
 
-*Make sure to check 'Add python.exe to PATH' while install Python*
-
-### Step 2: Install basic Python dependencies:
+### Step 2: Install basic Python dependencies
 Run below commands in Windows terminal:
 ```
-pip install requests wget tqdm importlib-metadata qai-hub qai_hub_models huggingface_hub Pillow numpy opencv-python torch torchvision torchaudio transformers diffusers ultralytics==8.0.193
+pip install qai-hub qai_hub_models huggingface_hub Pillow numpy opencv-python torch torchvision torchaudio transformers diffusers ultralytics==8.0.193
 ```
 
-### Step 3: Download QAI AppBuilder repository:
-Run below commands in Windows terminal:
-```
-git clone https://github.com/quic/ai-engine-direct-helper.git
-cd ai-engine-direct-helper\samples
-```
+### Step 3: Prepare Stable Diffusion models.
+Before running Stable Diffusion python script, please download Stable Diffusion models from following AI-Hub website and save them to path: 'samples\python\stable_diffusion_v1_5\models' & 'samples\python\stable_diffusion_v2_1\models' manually.<br>
+For other models, the sample python script will download them automatically.
 
-### Step 4: Setup QAI AppBuilder Python Environment:
-Run below commands in Windows terminal:
-```
-python python\setup.py
-```
+There're 3 models for each Stable Diffusion need to be downloaded: TextEncoderQuantizable, UnetQuantizable, VaeDecoderQuantizable. <br>
+Make sure to select the right model when download them:<br>
+1. Choose runtime: *Qualcomm® AI Engine Direct*<br>
+2. Choose device: *Snapdragon® X Elite*<br>
+
+Do *not* rename the model names, just download and copy them to the 'models' folder. <br>
+
+Models AI-Hub links:<br>
+[stable_diffusion_v1_5](https://aihub.qualcomm.com/compute/models/stable_diffusion_v1_5_w8a16_quantized)<br>
+[stable_diffusion_v2_1](https://aihub.qualcomm.com/compute/models/stable_diffusion_v2_1_quantized)<br>
 
-### Step 5: Run Model:
+### Step 4: Run Model
 Run below commands in Windows terminal:
 ```
+cd ai-engine-direct-helper\samples
 python <Python script for running model> <Parameter of Python script>
 ```
 Where `<Python script for running model>` is the Python script you want to run. For example, if you want to run `stable_diffusion_v2_1`, you can run below command:
 ```
+cd ai-engine-direct-helper\samples
 python python\stable_diffusion_v2_1\stable_diffusion_v2_1.py --prompt "spectacular view of northern lights from Alaska"
 ```
 
 ### Support Automatically Setting Up Model List:
 
 |  Model   | Command  |
 |  ----  | :---- |
-| stable_diffusion_v2_1 * | python python\stable_diffusion_v2_1\stable_diffusion_v2_1.py --prompt "the prompt string ..." |
-| stable_diffusion_v1_5 * | python python\stable_diffusion_v1_5\stable_diffusion_v1_5.py --prompt "the prompt string ..." |
+| stable_diffusion_v2_1 | python python\stable_diffusion_v2_1\stable_diffusion_v2_1.py --prompt "the prompt string ..." |
+| stable_diffusion_v1_5 | python python\stable_diffusion_v1_5\stable_diffusion_v1_5.py --prompt "the prompt string ..." |
 | real_esrgan_x4plus  | python python\real_esrgan_x4plus\real_esrgan_x4plus.py |
 | real_esrgan_general_x4v3  | python python\real_esrgan_general_x4v3\real_esrgan_general_x4v3.py |
 | inception_v3  | python python\inception_v3\inception_v3.py |
@@ -55,15 +57,4 @@ python python\stable_diffusion_v2_1\stable_diffusion_v2_1.py --prompt "spectacul
 | aotgan  | python python\aotgan\aotgan.py |
 | | |
 
-*. Before running Stable Diffusion app, please download Stable Diffusion models from following AI-Hub website and save them to path: samples\python\stable_diffusion_v1_5\models & samples\python\stable_diffusion_v2_1\models.<br>
-
-There're 3 models for each Stable Diffusion need to be downloaded: TextEncoderQuantizable, UnetQuantizable, VaeDecoderQuantizable <br>
-
-Choose runtime: Qualcomm® AI Engine Direct<br>
-Choose device: Snapdragon® X Elite<br>
-
-Models:<br>
-[stable_diffusion_v1_5](https://aihub.qualcomm.com/compute/models/stable_diffusion_v1_5_w8a16_quantized)<br>
-[stable_diffusion_v2_1](https://aihub.qualcomm.com/compute/models/stable_diffusion_v2_1_quantized)<br>
-
 *More models will be supported soon!*