[feat][PES][datasource] add support for Oscar database in metadata query service#1070
Open
v-kkhuang wants to merge 7 commits into
Open
[feat][PES][datasource] add support for Oscar database in metadata query service#1070v-kkhuang wants to merge 7 commits into
v-kkhuang wants to merge 7 commits into
Conversation
…abase properties with common metadata(优化HiveMetaService.queryTableProps：表名为空时回退查询database，返回库参数及常用元数据location/owner/ownerType/name/description) Co-Authored-By: Claude <noreply@anthropic.com>
…B_DEFAULT__ to trigger database properties query(优化HiveMetaService.queryTableProps：改用占位表名__DB_DEFAULT__触发库参数查询，部分终端不支持空表名；并在落库查询前校验占位表真实不存在，避免误判) Co-Authored-By: Claude <noreply@anthropic.com>
…lds in ES mappings(优化EsMetaService.queryColumns：递归展开ES mappings嵌套object字段，含type停止递归取叶子字段，含properties则用其进入下一层，层级用|分隔；FIELD_PROPS改为公开常量) Co-Authored-By: Claude <noreply@anthropic.com>
… query across full stack(新增existsTable表是否存在判断接口，打通metadata-query全链路)
- pes-common: MetadataDbService新增existsTable声明；AbstractDbMetaService新增existsTable实现及默认queryExistsTable(throw WarnException)
- server: MetadataQueryService/Impl补齐existsTableByDsId/ByDsName/ByDsNameAndEnvId，反射调用existsTable
- restful: MetadataCoreRestful(/metadatamanager/exists/{id}/db/{db}/table/{table})与MetadataQueryRestful(/metadataQuery/existsTable)新增existsTable接口，返回exists
- hive: HiveMetaService实现queryExistsTable，通过getTable异常类型判断，InvalidTableException判为表不存在返回false，其他异常直接抛出
Co-Authored-By: Claude <noreply@anthropic.com>
…mary keys from SYSCAT.KEYCOLUSE(修复DB2 getColumns对带引号小写表名如DB2INST1."db2hive04"查询报错) - getColumns拼SQL时对table加双引号引用，保留建表时原始大小写，避免未引用标识符被DB2折叠为大写导致找不到表；对普通大写表名同样兼容 - getPrimaryKeys改用直接查SYSCAT.KEYCOLUSE按真实表名大小写匹配主键，不再依赖JDBC getPrimaryKeys(底层SYSIBM.SQLPRIMARYKEYS对带引号小写表名不可靠)；同时参数化消除schema/table拼接注入隐患 Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Background/Problem:
Currently, the Linkis metadata query service does not support Oscar database (神通数据库), which is a widely-used domestic database in China. Users who need to query Oscar database metadata through Linkis cannot do so, limiting the system's applicability in domestic enterprise environments.
Purpose of Change:
To address this problem, this PR adds full support for Oscar database in the Linkis metadata query service, including database connection, database listing, table listing, and column querying capabilities. Additionally, it includes optimizations for ElasticSearch and Hive metadata query services to improve metadata retrieval functionality.
Value/Impact:
After the change, enterprise users can integrate Oscar database with Linkis-based data development platforms, expanding the system's support for domestic databases and improving query efficiency for ES nested fields and Hive database properties.
Related issues/PRs
Related issues: close #1069
Related pr:none
Brief change log
Checklist