Files
dbx/agents/metadata-constraint-coverage.tsv
T
jischeng c3893bf314 feat(spark): add Apache Spark database driver with catalog support
* feat(spark): add Apache Spark database driver

Spark Thrift Server speaks the HiveServer2 protocol, so the Spark driver
reuses the hive-jdbc agent runtime. Registers spark (agentKey=spark,
defaultPort=10015) across the manifest, DatabaseType enum, agent catalog,
SQL dialects, connection dialog, and a SparkAgent built on AbstractJdbcAgent.

* feat(spark): add catalog support and official logo

SparkAgent now supports Spark 3.4+ multi-catalog (Paimon, Lance, Iceberg)
via catalog=<name> in url_params, mirroring the StarRocks catalog flow:
- afterConnect switches to the configured catalog via USE <catalog>
- listDatabases returns the catalog name when SHOW DATABASES is empty
  (catalogs like Lance expose tables at the catalog root with no databases)
- listTables uses SHOW TABLES IN <catalog>[.<schema>] uniformly, working
  for both catalog-root tables (Lance) and catalog.schema tables (Paimon)
- setSchemaSQL qualifies USE with the catalog to prevent resetting to
  spark_catalog on schema switches
- buildJdbcUrl omits database from the URL path

Also replaces the placeholder lightning icon with the official Apache Spark
logo (spark-logo.png from spark.apache.org).

* fix(spark): use catalog-qualified DESCRIBE for column retrieval

getColumnsFromDescribe used USE <schema> + DESCRIBE <table>, which
generated USE `catalog`.`catalog` when the sidebar passed the catalog
name as the schema (Lance case where the catalog is the only node).
This failed and fell back to JDBC metadata which also returned nothing,
resulting in an empty DDL (CREATE TABLE ... ()).

Now uses DESCRIBE `catalog`.`table` directly when a catalog is
configured, bypassing USE entirely. Also fixes setSchemaSQL to treat
schema == catalog as catalog-only (no .schema suffix).

* fix(ci): add spark to metadata coverage matrix and BRIDGE_REQUIRED_TYPES

- agents/metadata-constraint-coverage.tsv: register spark as
  intentional-fallback (SHOW TABLES/SHOW DATABASES, like Hive/TDengine)
- packages/node-core/src/diagnostics.ts: add spark to
  BRIDGE_REQUIRED_TYPES (mcpMode=bridge, matches manifest)

* fix(ci): add spark to DBX_CONNECTION_TYPE_DESCRIPTION in mcp-server

The driver-manifest test asserts every manifest dbType appears in the
MCP server's connection type description string.
2026-07-07 14:36:22 +08:00

25 lines
3.0 KiB
Plaintext

driver strategy scope reason
access intentional-fallback java-jdbc-metadata Uses Access JDBC metadata; no portable stable server-side paging API, common constraints filter locally.
bigquery native-pushdown java-sql Uses dataset INFORMATION_SCHEMA.TABLES with type, normal-name filter, stable order, and literal LIMIT/OFFSET.
cassandra intentional-fallback java-cql Uses Cassandra system_schema; portable LIKE/fuzzy paging is not safe across versions, common constraints filter locally.
dameng native-pushdown java-sql Uses ALL_OBJECTS with type, filter, stable order, and LIMIT/OFFSET with legacy fallback.
databend native-pushdown java-sql Uses system procedures for routine pushdown and shared table metadata constraints for table-like objects.
db2 native-pushdown java-sql Uses SYSCAT metadata with type, filter, stable order, and OFFSET/FETCH.
gbase8a native-pushdown java-sql Uses information_schema with type, filter, stable order, and LIMIT/OFFSET.
gbase8s native-pushdown java-sql Uses systables with type, filter, stable order, and SKIP/FIRST.
goldendb native-pushdown java-sql Uses information_schema with type, filter, stable order, and LIMIT/OFFSET.
h2 native-pushdown java-sql Uses INFORMATION_SCHEMA with type, filter, stable order, and LIMIT/OFFSET.
hive intentional-fallback java-sql Uses Hive metadata paths where stable server-side fuzzy paging is not portable, common constraints filter locally.
informix native-pushdown java-sql Uses systables/sysprocedures with type, filter, stable order, and SKIP/FIRST.
kingbase native-pushdown java-sql Uses sys_catalog/information_schema with type, filter, stable order, and LIMIT/OFFSET for both compatibility modes.
kylin intentional-fallback java-jdbc-metadata Uses JDBC metadata from Kylin driver; no portable server-side paging API, common constraints filter locally.
neo4j intentional-fallback java-graph-metadata Uses JDBC metadata with Cypher fallback; label discovery cannot guarantee relational-style filtered paging, common constraints filter locally.
oceanbase-oracle native-pushdown java-sql Uses Oracle-compatible metadata SQL with type, filter, stable order, and ROWNUM paging.
oracle-go native-pushdown native-go Uses Oracle-compatible metadata SQL with type, filter, stable order, and ROWNUM paging, with legacy profile guard.
snowflake native-pushdown java-sql Uses INFORMATION_SCHEMA with type, filter, stable order, and literal LIMIT/OFFSET.
spark intentional-fallback java-show-metadata Uses SHOW TABLES/SHOW DATABASES; stable server-side fuzzy paging is not portable, common constraints filter locally.
sundb native-pushdown java-sql Uses information_schema with type, filter, stable order, and LIMIT/OFFSET.
tdengine intentional-fallback java-show-metadata Uses SHOW STABLES/TABLES; stable server-side fuzzy paging is not portable, common constraints filter locally.
trino native-pushdown java-sql Uses information_schema.tables with type, filter, stable order, and literal LIMIT/OFFSET.
xugu native-pushdown native-go Uses ALL_TABLES/ALL_VIEWS metadata with type, filter, stable order, and ROWNUM paging.