Skip to content

Commit 3f2099c

Browse files
gopalldbclaude
gopalldb
and
claude
authored
Add EnableGeoSpatialSupport connection parameter to gate geospatial types (#1094)
## Description Adds a new connection parameter EnableGeoSpatialSupport to independently gate native GEOMETRY and GEOGRAPHY data type support in the JDBC driver. Key changes: - Added EnableGeoSpatialSupport parameter to DatabricksJdbcUrlParams (default: 0) - Implemented isGeoSpatialSupportEnabled() in connection context with prerequisite check for EnableComplexDatatypeSupport - Updated ArrowStreamResult.getObject() to convert geospatial types to STRING when flag is disabled - Updated MetadataResultSetBuilder to report geospatial columns as VARCHAR when flag is disabled - Added telemetry tracking for the new parameter - Added comprehensive unit tests for flag behavior and type conversion logic When EnableGeoSpatialSupport=0 (default): - GEOMETRY and GEOGRAPHY columns are returned as STRING type - Backward compatible with existing applications When EnableGeoSpatialSupport=1 AND EnableComplexDatatypeSupport=1: - GEOMETRY and GEOGRAPHY columns are returned as native DatabricksGeometry and DatabricksGeography objects This change provides opt-in support for geospatial types while maintaining backward compatibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- Provide a brief summary of the changes made and the issue they aim to address.--> ## Testing <!-- Describe how the changes have been tested--> Added tests ## Additional Notes to the Reviewer <!-- Share any additional context or insights that may help the reviewer understand the changes better. This could include challenges faced, limitations, or compromises made during the development process. Also, mention any areas of the code that you would like the reviewer to focus on specifically. --> --------- Co-authored-by: Claude <[email protected]>
1 parent 7fdb141 commit 3f2099c

File tree

13 files changed

+1025
-9
lines changed

13 files changed

+1025
-9
lines changed

‎NEXT_CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
- Note: Large batches are chunked for execution. If a chunk fails, previous chunks remain committed (no transaction rollback). Consider using staging tables for critical workflows.
1414
- Added a new config attribute `DisableOauthRefreshToken` to control whether refresh tokens are requested in OAuth exchanges. By default, the driver does not include the `offline_access` scope. If `offline_access` is explicitly provided by the user, it is preserved and not removed.
1515
- Added Feature-flag integration for SQL Exec API rollout
16+
- Add a gating flag for enabling GeoSpatial support: `EnableGeoSpatialSupport`. By default, it will be disabled
1617
- Call statements will return result sets in response
1718

1819
### Updated

‎src/main/java/com/databricks/jdbc/api/impl/DatabricksConnectionContext.java‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -915,6 +915,13 @@ public boolean isComplexDatatypeSupportEnabled() {
915915
return getParameter(DatabricksJdbcUrlParams.ENABLE_COMPLEX_DATATYPE_SUPPORT).equals("1");
916916
}
917917

918+
@Override
919+
public boolean isGeoSpatialSupportEnabled() {
920+
// Geospatial support requires complex datatype support to be enabled
921+
return isComplexDatatypeSupportEnabled()
922+
&& getParameter(DatabricksJdbcUrlParams.ENABLE_GEOSPATIAL_SUPPORT).equals("1");
923+
}
924+
918925
@Override
919926
public boolean isRequestTracingEnabled() {
920927
return getParameter(DatabricksJdbcUrlParams.ENABLE_REQUEST_TRACING).equals("1");

‎src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSet.java‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -492,6 +492,16 @@ private static boolean isComplexType(String typeName) {
492492
|| typeName.startsWith(GEOGRAPHY);
493493
}
494494

495+
/**
496+
* Checks if the given type name represents a geospatial type (GEOMETRY or GEOGRAPHY).
497+
*
498+
* @param typeName The type name to check
499+
* @return true if the type name starts with GEOMETRY or GEOGRAPHY, false otherwise
500+
*/
501+
private static boolean isGeospatialType(String typeName) {
502+
return typeName.startsWith(GEOMETRY) || typeName.startsWith(GEOGRAPHY);
503+
}
504+
495505
@Override
496506
public Object getObject(int columnIndex) throws SQLException {
497507
checkIfClosed();

‎src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSetMetaData.java‎

Lines changed: 40 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,18 @@ public DatabricksResultSetMetaData(
9595
columnTypeName = ColumnInfoTypeName.TIMESTAMP;
9696
columnInfo.setTypeText(TIMESTAMP);
9797
}
98+
99+
// Check if we need to convert geospatial types to string when geospatial support is
100+
// disabled
101+
String typeText = columnInfo.getTypeText();
102+
if (!ctx.isGeoSpatialSupportEnabled() && isGeospatialType(columnTypeName)) {
103+
LOGGER.debug(
104+
"Geospatial support is disabled, converting {} to STRING in metadata",
105+
columnTypeName);
106+
columnTypeName = ColumnInfoTypeName.STRING;
107+
typeText = "STRING";
108+
}
109+
98110
int columnType = DatabricksTypeUtil.getColumnType(columnTypeName);
99111
int[] precisionAndScale = getPrecisionAndScale(columnInfo, columnType);
100112
int precision = precisionAndScale[0];
@@ -106,8 +118,7 @@ public DatabricksResultSetMetaData(
106118
.columnType(columnType)
107119
.columnTypeText(
108120
metadataResultSetBuilder.stripTypeName(
109-
columnInfo
110-
.getTypeText())) // store base type eg. DECIMAL instead of DECIMAL(7,2)
121+
typeText)) // store base type eg. DECIMAL instead of DECIMAL(7,2)
111122
.typePrecision(precision)
112123
.typeScale(scale)
113124
.displaySize(DatabricksTypeUtil.getDisplaySize(columnTypeName, precision, scale))
@@ -211,16 +222,31 @@ public DatabricksResultSetMetaData(
211222
.columnTypeClassName("java.lang.String")
212223
.columnType(Types.OTHER)
213224
.columnTypeText(VARIANT);
214-
} else if (isGeometryColumn(arrowMetadata, columnIndex)) {
225+
} else if (isGeometryColumn(arrowMetadata, columnIndex)
226+
&& ctx.isGeoSpatialSupportEnabled()) {
227+
// Only set GEOMETRY type if geospatial support is enabled
215228
columnBuilder
216229
.columnTypeClassName(GEOMETRY_CLASS_NAME)
217230
.columnType(Types.OTHER)
218231
.columnTypeText(GEOMETRY);
219-
} else if (isGeographyColumn(arrowMetadata, columnIndex)) {
232+
} else if (isGeographyColumn(arrowMetadata, columnIndex)
233+
&& ctx.isGeoSpatialSupportEnabled()) {
234+
// Only set GEOGRAPHY type if geospatial support is enabled
220235
columnBuilder
221236
.columnTypeClassName(GEOGRAPHY_CLASS_NAME)
222237
.columnType(Types.OTHER)
223238
.columnTypeText(GEOGRAPHY);
239+
} else if ((isGeometryColumn(arrowMetadata, columnIndex)
240+
|| isGeographyColumn(arrowMetadata, columnIndex))
241+
&& !ctx.isGeoSpatialSupportEnabled()) {
242+
// Convert geospatial types to STRING when support is disabled
243+
LOGGER.debug(
244+
"Geospatial support is disabled, converting column {} to STRING in Thrift metadata",
245+
columnInfo.getName());
246+
columnBuilder
247+
.columnTypeClassName("java.lang.String")
248+
.columnType(Types.VARCHAR)
249+
.columnTypeText("STRING");
224250
}
225251
columnsBuilder.add(columnBuilder.build());
226252
columnNameToIndexMap.putIfAbsent(columnInfo.getName(), ++currIndex);
@@ -672,6 +698,16 @@ private boolean isGeographyColumn(List<String> arrowMetadata, int index) {
672698
&& arrowMetadata.get(index).contains(GEOGRAPHY);
673699
}
674700

701+
/**
702+
* Checks if the given column type is a geospatial type.
703+
*
704+
* @param type the column type to check
705+
* @return true if the type is GEOMETRY or GEOGRAPHY, false otherwise
706+
*/
707+
private boolean isGeospatialType(ColumnInfoTypeName type) {
708+
return type == ColumnInfoTypeName.GEOMETRY || type == ColumnInfoTypeName.GEOGRAPHY;
709+
}
710+
675711
private ImmutableDatabricksColumn.Builder getColumnBuilder() {
676712
return ImmutableDatabricksColumn.builder()
677713
.isAutoIncrement(false)

‎src/main/java/com/databricks/jdbc/api/impl/arrow/ArrowStreamResult.java‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,24 @@ public Object getObject(int columnIndex) throws DatabricksSQLException {
142142
// Handle complex type conversion when complex datatype support is disabled
143143
boolean isComplexDatatypeSupportEnabled =
144144
this.session.getConnectionContext().isComplexDatatypeSupportEnabled();
145+
boolean isGeoSpatialSupportEnabled =
146+
this.session.getConnectionContext().isGeoSpatialSupportEnabled();
147+
148+
// Check if we need to convert geospatial types to string when geospatial support is disabled
149+
// This check must come before the general complex type check
150+
if (!isGeoSpatialSupportEnabled && isGeospatialType(requiredType)) {
151+
LOGGER.debug("Geospatial support is disabled, converting {} to STRING", requiredType);
152+
153+
Object result =
154+
chunkIterator.getColumnObjectAtCurrentRow(
155+
columnIndex, ColumnInfoTypeName.STRING, "STRING", columnInfos.get(columnIndex));
156+
if (result == null) {
157+
return null;
158+
}
159+
// Return raw string for geospatial types when support is disabled
160+
return result;
161+
}
162+
145163
if (!isComplexDatatypeSupportEnabled && isComplexType(requiredType)) {
146164
LOGGER.debug("Complex datatype support is disabled, converting complex type to STRING");
147165

@@ -174,6 +192,17 @@ public static boolean isComplexType(ColumnInfoTypeName type) {
174192
|| type == ColumnInfoTypeName.GEOGRAPHY;
175193
}
176194

195+
/**
196+
* Checks if the given type is a geospatial type (GEOMETRY or GEOGRAPHY).
197+
*
198+
* @param type The type to check
199+
* @return true if the type is a geospatial type, false otherwise
200+
*/
201+
@VisibleForTesting
202+
public static boolean isGeospatialType(ColumnInfoTypeName type) {
203+
return type == ColumnInfoTypeName.GEOMETRY || type == ColumnInfoTypeName.GEOGRAPHY;
204+
}
205+
177206
/** {@inheritDoc} */
178207
@Override
179208
public long getCurrentRow() {

‎src/main/java/com/databricks/jdbc/api/internal/IDatabricksConnectionContext.java‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -300,6 +300,12 @@ public interface IDatabricksConnectionContext {
300300
/** Returns true if driver return complex data type java objects natively as opposed to string */
301301
boolean isComplexDatatypeSupportEnabled();
302302

303+
/**
304+
* Returns true if driver returns GEOMETRY and GEOGRAPHY types natively. Requires
305+
* isComplexDatatypeSupportEnabled() to be true
306+
*/
307+
boolean isGeoSpatialSupportEnabled();
308+
303309
/** Returns the size for HTTP connection pool */
304310
int getHttpConnectionPoolSize();
305311

‎src/main/java/com/databricks/jdbc/common/DatabricksJdbcUrlParams.java‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,10 @@ public enum DatabricksJdbcUrlParams {
124124
"EnableComplexDatatypeSupport",
125125
"flag to enable native support of complex data types as java objects",
126126
"0"),
127+
ENABLE_GEOSPATIAL_SUPPORT(
128+
"EnableGeoSpatialSupport",
129+
"flag to enable native support of GEOMETRY and GEOGRAPHY data types. Requires EnableComplexDatatypeSupport=1",
130+
"0"),
127131
ROWS_FETCHED_PER_BLOCK(
128132
"RowsFetchedPerBlock",
129133
"The maximum number of rows that a query returns at a time.",

‎src/main/java/com/databricks/jdbc/dbclient/impl/common/MetadataResultSetBuilder.java‎

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
package com.databricks.jdbc.dbclient.impl.common;
22

33
import static com.databricks.jdbc.common.MetadataResultConstants.*;
4+
import static com.databricks.jdbc.common.util.DatabricksTypeUtil.GEOGRAPHY;
5+
import static com.databricks.jdbc.common.util.DatabricksTypeUtil.GEOMETRY;
46
import static com.databricks.jdbc.common.util.DatabricksTypeUtil.INTERVAL;
57
import static com.databricks.jdbc.common.util.DatabricksTypeUtil.MEASURE;
68
import static com.databricks.jdbc.common.util.WildcardUtil.isNullOrEmpty;
@@ -193,8 +195,11 @@ List<List<Object>> getRows(
193195
if (typeVal == null) { // safety check
194196
object = null;
195197
} else {
196-
// Check if complex datatype support is disabled and this is a complex type
197-
if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
198+
// Check if geospatial support is disabled and this is a geospatial type
199+
if (!ctx.isGeoSpatialSupportEnabled() && isGeospatialType(typeVal)) {
200+
object = Types.VARCHAR;
201+
} else if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
202+
// Check if complex datatype support is disabled and this is a complex type
198203
object = Types.VARCHAR;
199204
} else {
200205
object = getCode(stripBaseTypeName(typeVal));
@@ -238,7 +243,10 @@ List<List<Object>> getRows(
238243
}
239244
} catch (SQLException e) {
240245
if (mappedColumn.getColumnName().equals(DATA_TYPE_COLUMN.getColumnName())) {
241-
if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
246+
// Check if geospatial support is disabled and this is a geospatial type
247+
if (!ctx.isGeoSpatialSupportEnabled() && isGeospatialType(typeVal)) {
248+
object = Types.VARCHAR;
249+
} else if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
242250
object = Types.VARCHAR;
243251
} else {
244252
object = getCode(stripBaseTypeName(typeVal));
@@ -552,6 +560,20 @@ private boolean isComplexType(String typeVal) {
552560
|| baseType.contains(STRUCT_TYPE);
553561
}
554562

563+
/**
564+
* Checks if the given type string represents a geospatial type (GEOMETRY or GEOGRAPHY).
565+
*
566+
* @param typeVal The type string to check
567+
* @return true if the type is a geospatial type, false otherwise
568+
*/
569+
private boolean isGeospatialType(String typeVal) {
570+
if (typeVal == null) {
571+
return false;
572+
}
573+
String baseType = stripBaseTypeName(typeVal);
574+
return baseType.contains(GEOMETRY) || baseType.contains(GEOGRAPHY);
575+
}
576+
555577
int getCode(String s) {
556578
switch (s) {
557579
case "STRING":
@@ -897,8 +919,11 @@ List<List<Object>> getThriftRows(List<List<Object>> rows, List<ResultColumn> col
897919
if (typeVal == null) { // safety check
898920
object = null;
899921
} else {
900-
// Check if complex datatype support is disabled and this is a complex type
901-
if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
922+
// Check if geospatial support is disabled and this is a geospatial type
923+
if (!ctx.isGeoSpatialSupportEnabled() && isGeospatialType(typeVal)) {
924+
object = Types.VARCHAR;
925+
} else if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
926+
// Check if complex datatype support is disabled and this is a complex type
902927
object = Types.VARCHAR;
903928
} else {
904929
object = getCode(stripBaseTypeName(typeVal));

‎src/main/java/com/databricks/jdbc/model/telemetry/DriverConnectionParameters.java‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,9 @@ public class DriverConnectionParameters {
9191
@JsonProperty("enable_complex_datatype_support")
9292
boolean enableComplexDatatypeSupport;
9393

94+
@JsonProperty("enable_geospatial_support")
95+
boolean enableGeoSpatialSupport;
96+
9497
@JsonProperty("azure_workspace_resource_id")
9598
String azureWorkspaceResourceId;
9699

@@ -278,6 +281,11 @@ public DriverConnectionParameters setEnableComplexDatatypeSupport(
278281
return this;
279282
}
280283

284+
public DriverConnectionParameters setEnableGeoSpatialSupport(boolean enableGeoSpatialSupport) {
285+
this.enableGeoSpatialSupport = enableGeoSpatialSupport;
286+
return this;
287+
}
288+
281289
public DriverConnectionParameters setAzureWorkspaceResourceId(String azureWorkspaceResourceId) {
282290
this.azureWorkspaceResourceId = azureWorkspaceResourceId;
283291
return this;
@@ -379,6 +387,7 @@ public String toString() {
379387
.add("googleCredentialFilePath", googleCredentialFilePath)
380388
.add("allowedVolumeIngestionPaths", allowedVolumeIngestionPaths)
381389
.add("enableComplexDatatypeSupport", enableComplexDatatypeSupport)
390+
.add("enableGeoSpatialSupport", enableGeoSpatialSupport)
382391
.add("azureWorkspaceResourceId", azureWorkspaceResourceId)
383392
.add("azureTenantId", azureTenantId)
384393
.add("stringColumnLength", stringColumnLength)

‎src/main/java/com/databricks/jdbc/telemetry/TelemetryHelper.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,7 @@ private static DriverConnectionParameters buildDriverConnectionParameters(
198198
.setSocketTimeout(connectionContext.getSocketTimeout())
199199
.setStringColumnLength(connectionContext.getDefaultStringColumnLength())
200200
.setEnableComplexDatatypeSupport(connectionContext.isComplexDatatypeSupportEnabled())
201+
.setEnableGeoSpatialSupport(connectionContext.isGeoSpatialSupportEnabled())
201202
.setAzureWorkspaceResourceId(connectionContext.getAzureWorkspaceResourceId())
202203
.setAzureTenantId(connectionContext.getAzureTenantId())
203204
.setSslTrustStoreType(connectionContext.getSSLTrustStoreType())

0 commit comments

Comments
 (0)