-
Notifications
You must be signed in to change notification settings - Fork 26
Description
Describe the bug
It looks like com.databricks.jdbc.api.impl.DatabricksPreparedStatement#setTimestamp(int, java.sql.Timestamp, java.util.Calendar) method truncates microseconds part from the timestamp when setting the query parameter, while Databricks itself support microseconds in timestamps.
This is critical because we are using timestamps we are getting from the DB to query next chunks of data. So this behaviour makes our system re-read the same records in loop.
To Reproduce
- Create a table with timestamp column
- Execute smth like:
try (Connection conn = DriverManager.getConnection(url)) {
// Build a timestamp with microsecond precision
Timestamp originalTs = Timestamp.valueOf("2025-11-21 23:30:49.185645");
// Ensure nanos are exactly what we expect (Spark uses microsecond precision)
originalTs.setNanos(185_645_000);
// Calendar matching Kafka Connect style (timestamp with time zone awareness)
Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
// Update existing row using PreparedStatement#setTimestamp with Calendar
try (PreparedStatement ps = conn.prepareStatement(updateSql)) {
ps.setTimestamp(1, originalTs, calendar);
LOGGER.info("Executing update with PreparedStatement#setTimestamp and Calendar");
int updated = ps.executeUpdate();
}
}
with LogLevel=5
3. Check logs
Expected behavior
Timestamp parameter is set as is.
Actual behavior
Timestamp parameter is truncated to milliseconds precision:
[main] INFO com.tealium.jdbc.DatabricksDriverTest - Original Timestamp before setTimestamp: 2025-11-21 23:30:49.185645 (nanos = 185645000)
[main] INFO com.tealium.jdbc.DatabricksDriverTest - Executing update with PreparedStatement#setTimestamp and Calendar
[main] INFO com.tealium.jdbc.DatabricksDriverTest - Round-tripped JDBC Timestamp: 2025-11-21 23:30:49.185 (nanos = 185000000)
2025-11-28 15:19:41 FINE com.databricks.jdbc.dbclient.impl.thrift.DatabricksThriftServiceClient#executeStatement - public DatabricksResultSet executeStatement(String sql = {UPDATE vzh_test SET ts = ? WHERE id = 2}, Compute cluster = {SQL Warehouse with warehouse ID {...}}, Map<Integer, ImmutableSqlParameter> parameters = {{1=SqlParameter{value=2025-11-21 23:30:49.185, type=TIMESTAMP, cardinal=1}}}, StatementType statementType = {UPDATE}, IDatabricksSession session)
Client Environment (please complete the following information):
- OS: MacOS
- Java version: Java 17
- Java vendor: graalvm
- Driver Version 1.0.11-oss
Additional context
Simba's version seems to process this case correctly without truncating microseconds part:
Nov 28 16:26:12.329 DEBUG 1 com.databricks.client.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.executeHelperParameter: The spark parameters for parameterized query are [TSparkParameter(ordinal:1, type:TIMESTAMP, value:<TSparkParameterValue stringValue:2025-11-21 23:30:49.185645>)]