Skip to content

Commit f15a972

Browse files
geoandgeoand
authored
Merge pull request #2020 from orionpapadakis/update-gpullama3-version
Update GPULlama3 version
2 parents 36e0ef6 + f3b22ba commit f15a972

File tree

4 files changed

+133
-33
lines changed

4 files changed

+133
-33
lines changed

‎docs/modules/ROOT/pages/gpullama3-chat-model.adoc‎

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,23 +12,42 @@ This extension allows Quarkus applications to use locally hosted Llama3 and othe
1212
1313
=== Java Version and TornadoVM
1414
15-
GPULlama3.java requires *Java 21 or later* due to its use of the https://openjdk.org/jeps/448[Java Vector API] and TornadoVM integration.
15+
GPULlama3.java requires TornadoVM and *Java 21 or later* due to its use of the https://openjdk.org/jeps/448[Java Vector API].
1616
17-
Install TornadoVM locally as follows:
17+
Download and Install TornadoVM locally as follows:
18+
19+
*Linux (x86_64)*
20+
21+
[source,shell]
22+
----
23+
wget https://github.com/beehive-lab/TornadoVM/releases/download/v2.1.0/tornadovm-2.1.0-opencl-linux-amd64.zip
24+
unzip tornadovm-2.1.0-opencl-linux-amd64.zip
25+
# Replace <path-to-sdk> manually with the absolute path of the extracted folder
26+
export TORNADO_SDK="<path-to-sdk>/tornadovm-2.1.0-opencl"
27+
export PATH=$TORNADO_SDK/bin:$PATH
28+
29+
tornado --devices
30+
tornado --version
31+
----
32+
33+
*macOS (Apple Silicon)*
1834
1935
[source,shell]
2036
----
21-
cd ~
22-
git clone [email protected]:beehive-lab/TornadoVM.git
23-
cd ~/TornadoVM
24-
./bin/tornadovm-installer --jdk jdk21 --backend opencl
25-
source setvars.sh
37+
wget https://github.com/beehive-lab/TornadoVM/releases/download/v2.1.0/tornadovm-2.1.0-opencl-mac-aarch64.zip
38+
unzip tornadovm-2.1.0-opencl-mac-aarch64.zip
39+
# Replace <path-to-sdk> manually with the absolute path of the extracted folder
40+
export TORNADO_SDK="<path-to-sdk>/tornadovm-2.1.0-opencl"
41+
export PATH=$TORNADO_SDK/bin:$PATH
42+
43+
tornado --devices
44+
tornado --version
2645
----
2746
2847
The above steps:
2948
3049
- Set the `TORNADOVM_SDK` environment variable to the TornadoVM SDK path.
31-
- Create a `tornado-argfile` under `~/TornadoVM` containing the JVM arguments required to enable TornadoVM.
50+
- `TORNADO_SDK` contains the `tornado-argfile` with all the JVM arguments required to enable TornadoVM.
3251
- ⚠️ The `tornado-argfile` should be used for *building* and *running* the Quarkus application (see section Building & Running the Quarkus Application).
3352
3453
== Using GPULlama3.java

‎integration-tests/gpu-llama3/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
<maven.compiler.release>21</maven.compiler.release>
1515
<quarkus.version>3.18.0</quarkus.version>
1616
<!-- TornadoVM argfile path -->
17-
<tornado.argfile>${env.TORNADO_SDK}/../../../tornado-argfile</tornado.argfile>
17+
<tornado.argfile>${env.TORNADO_SDK}/tornado-argfile</tornado.argfile>
1818
</properties>
1919
<dependencies>
2020
<dependency>

‎model-providers/gpu-llama3/runtime/pom.xml‎

Lines changed: 104 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -74,30 +74,111 @@
7474
</runnerParentFirstArtifacts>
7575
</configuration>
7676
</plugin>
77-
<plugin>
78-
<artifactId>maven-compiler-plugin</artifactId>
79-
<configuration>
80-
<!--
81-
In order for vector support to work on JDK 21+, we need to build the module with JDK 23,
82-
because in that version Vector is no longer preview, so we build target and runtime versions
83-
don't need to match (remember that preview features only work on the same runtime version as the target
84-
-->
85-
<source>21</source>
86-
<target>21</target>
87-
<release combine.self="override" /> <!-- make sure -release is not passed to javac-->
88-
<compilerArgs>
89-
<arg>--add-modules=jdk.incubator.vector</arg>
90-
</compilerArgs>
91-
<annotationProcessorPaths>
92-
<path>
93-
<groupId>io.quarkus</groupId>
94-
<artifactId>quarkus-extension-processor</artifactId>
95-
<version>${quarkus.version}</version>
96-
</path>
97-
</annotationProcessorPaths>
98-
</configuration>
99-
</plugin>
10077
</plugins>
10178
</build>
10279

80+
<profiles>
81+
<!-- Java 21: Enable preview features to match gpu-llama3 compilation -->
82+
<profile>
83+
<id>jdk21-with-preview</id>
84+
<activation>
85+
<jdk>21</jdk>
86+
</activation>
87+
<build>
88+
<plugins>
89+
<plugin>
90+
<artifactId>maven-compiler-plugin</artifactId>
91+
<configuration>
92+
<!--
93+
Target Java 21 with preview features to match gpu-llama3 dependency
94+
-->
95+
<source>21</source>
96+
<target>21</target>
97+
<release combine.self="override" /> <!-- make sure -release is not passed to javac-->
98+
<compilerArgs>
99+
<arg>--enable-preview</arg>
100+
<arg>--add-modules=jdk.incubator.vector</arg>
101+
</compilerArgs>
102+
<annotationProcessorPaths>
103+
<path>
104+
<groupId>io.quarkus</groupId>
105+
<artifactId>quarkus-extension-processor</artifactId>
106+
<version>${quarkus.version}</version>
107+
</path>
108+
</annotationProcessorPaths>
109+
</configuration>
110+
</plugin>
111+
</plugins>
112+
</build>
113+
</profile>
114+
115+
<!-- Java 22-23: Compile to Java 21 target for compatibility -->
116+
<profile>
117+
<id>jdk22-23-with-preview</id>
118+
<activation>
119+
<jdk>[22,23]</jdk>
120+
</activation>
121+
<build>
122+
<plugins>
123+
<plugin>
124+
<artifactId>maven-compiler-plugin</artifactId>
125+
<configuration>
126+
<!--
127+
Compile with JDK 22-23 but target Java 21 for compatibility
128+
-->
129+
<source>21</source>
130+
<target>21</target>
131+
<release combine.self="override" /> <!-- make sure -release is not passed to javac-->
132+
<compilerArgs>
133+
<arg>--enable-preview</arg>
134+
<arg>--add-modules=jdk.incubator.vector</arg>
135+
</compilerArgs>
136+
<annotationProcessorPaths>
137+
<path>
138+
<groupId>io.quarkus</groupId>
139+
<artifactId>quarkus-extension-processor</artifactId>
140+
<version>${quarkus.version}</version>
141+
</path>
142+
</annotationProcessorPaths>
143+
</configuration>
144+
</plugin>
145+
</plugins>
146+
</build>
147+
</profile>
148+
149+
<!-- Java 24+: Use current JDK's preview features -->
150+
<profile>
151+
<id>jdk24-plus</id>
152+
<activation>
153+
<jdk>[24,)</jdk>
154+
</activation>
155+
<build>
156+
<plugins>
157+
<plugin>
158+
<artifactId>maven-compiler-plugin</artifactId>
159+
<configuration>
160+
<!--
161+
Use JDK 24+ where preview features are supported for current version
162+
-->
163+
<source>24</source>
164+
<target>24</target>
165+
<release combine.self="override" /> <!-- make sure -release is not passed to javac-->
166+
<compilerArgs>
167+
<arg>--enable-preview</arg>
168+
<arg>--add-modules=jdk.incubator.vector</arg>
169+
</compilerArgs>
170+
<annotationProcessorPaths>
171+
<path>
172+
<groupId>io.quarkus</groupId>
173+
<artifactId>quarkus-extension-processor</artifactId>
174+
<version>${quarkus.version}</version>
175+
</path>
176+
</annotationProcessorPaths>
177+
</configuration>
178+
</plugin>
179+
</plugins>
180+
</build>
181+
</profile>
182+
</profiles>
183+
103184
</project>

‎pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@
4848
<quarkus-neo4j.version>5.7.0</quarkus-neo4j.version>
4949
<jlama.version>0.8.4</jlama.version>
5050
<google-auth-library-oauth2-http.version>1.37.1</google-auth-library-oauth2-http.version>
51-
<gpu-llama3.version>0.2.2</gpu-llama3.version>
51+
<gpu-llama3.version>0.3.1</gpu-llama3.version>
5252
<smallrye-certificate-generator.version>0.9.2</smallrye-certificate-generator.version>
5353
</properties>
5454
<dependencyManagement>

0 commit comments

Comments
 (0)