Commit Graph

218 Commits

Author SHA1 Message Date
jerryjzhang
cd863705a4 [improvement][chat]Opt log infos.
Some checks are pending
supersonic CentOS CI / build (21) (push) Waiting to run
supersonic mac CI / build (21) (push) Waiting to run
supersonic ubuntu CI / build (21) (push) Waiting to run
supersonic windows CI / build (21) (push) Waiting to run
2025-01-08 20:46:44 +08:00
jerryjzhang
61e22c2104 [improvement][headless]Add sql field in the OntologyQuery. 2025-01-08 19:43:32 +08:00
jerryjzhang
4e653c1fb1 [improvement][headless]Expression replacement logic supports more complex sql.
Some checks failed
supersonic CentOS CI / build (21) (push) Has been cancelled
supersonic mac CI / build (21) (push) Has been cancelled
supersonic ubuntu CI / build (21) (push) Has been cancelled
supersonic windows CI / build (21) (push) Has been cancelled
2025-01-05 16:09:42 +08:00
jerryjzhang
4738b9d01b [improvement][headless]Release brand new version of Translator module.
Some checks are pending
supersonic CentOS CI / build (21) (push) Waiting to run
supersonic mac CI / build (21) (push) Waiting to run
supersonic ubuntu CI / build (21) (push) Waiting to run
supersonic windows CI / build (21) (push) Waiting to run
2025-01-05 00:00:18 +08:00
Jun Zhang
be59b051fc [fix][headless]Fix logic bug in s2sql parsing. (#1996)
Some checks are pending
supersonic CentOS CI / build (21) (push) Waiting to run
supersonic mac CI / build (21) (push) Waiting to run
supersonic ubuntu CI / build (21) (push) Waiting to run
supersonic windows CI / build (21) (push) Waiting to run
2025-01-04 14:58:48 +08:00
jerryjzhang
a46e89af56 [improvement][headless]Simplify query sql and fix demo.
Some checks failed
supersonic CentOS CI / build (21) (push) Has been cancelled
supersonic mac CI / build (21) (push) Has been cancelled
supersonic ubuntu CI / build (21) (push) Has been cancelled
supersonic windows CI / build (21) (push) Has been cancelled
2024-12-31 09:13:56 +08:00
jerryjzhang
a2f54d4c80 [improvement][headless]Remove unnecessary Database class
Some checks are pending
supersonic CentOS CI / build (21) (push) Waiting to run
supersonic mac CI / build (21) (push) Waiting to run
supersonic ubuntu CI / build (21) (push) Waiting to run
supersonic windows CI / build (21) (push) Waiting to run
2024-12-29 21:58:39 +08:00
jerryjzhang
24eaffb2d5 [improvement][headless]Adapt to new master version. 2024-12-29 19:44:37 +08:00
jerryjzhang
739514ddfa [fix][chat&headless]Adapt to new master. 2024-12-29 10:45:18 +08:00
jerryjzhang
6486257c9e [improvement][chat&headless]Remove deprecated system time fields. 2024-12-29 03:22:42 +08:00
jerryjzhang
920d8f280a [improvement][headless]Add more checks to accept() of QueryParsers. 2024-12-27 11:57:23 +08:00
jerryjzhang
d834e98a66 [improvement][chat]Introduce AllFieldMapper to increase parsing robustness when normal pipeline fails. 2024-12-26 23:28:59 +08:00
jerryjzhang
94267f6028 [improvement][chat]Introduce AllFieldMapper to increase parsing robustness when normal pipeline fails.
[improvement][chat]Introduce `AllFieldMapper` to increase parsing robustness when normal pipeline fails.
2024-12-26 23:20:43 +08:00
jerryjzhang
68963b9ec9 [improvement][project]Adjust files based on code style. 2024-12-26 09:12:12 +08:00
jerryjzhang
5de5b0a5e2 [fix][headless]Fix issue in determining mysql version to support with statement. 2024-12-22 21:40:58 +08:00
jerryjzhang
214d90772d [improvement][headless]Move discovery of query models from SemanticNode to SqlQueryParser.
[improvement][headless]Move discovery of query models from SemanticNode to `SqlQueryParser`.
2024-12-22 21:15:23 +08:00
jerryjzhang
d8b8c4e6b9 [improvement][headless]Move discovery of query models from SemanticNode to QueryParser. 2024-12-22 17:45:32 +08:00
jerryjzhang
10a510409f [fix][headless]Fix issue in determining mysql version to support with statement. 2024-12-21 22:50:55 +08:00
jerryjzhang
19c1d306df [fix][headless]Fix issue in determining mysql version to support with statement. 2024-12-21 22:50:22 +08:00
jerryjzhang
4cb2256351 [improvement][headless]Merge function of QueryConverter abstraction to QueryParser. 2024-12-19 21:45:24 +08:00
jerryjzhang
91856ddebd [improvement][chat]Inject schema info into the prompt of LLMSqlCorrector. 2024-12-19 09:55:32 +08:00
jerryjzhang
6fcd105249 [improvement][chat]Inject schema info into the prompt of LLMSqlCorrector. 2024-12-18 12:04:06 +08:00
jerryjzhang
1d0f5612b7 [fix][headless]Fix NEP problem. 2024-12-18 09:05:32 +08:00
wwsheng009
b57eed47e2 SAP HANA DATABASE Source support improvement[优化SAPhana数据库的支持] (#1959)
* (improvement)(database) update the support for sap hana database source

* (fix)(common) add the default timeout for ZhipuAiEmbeddingModel,avoid the program error
2024-12-17 21:52:19 +08:00
jerryjzhang
aa3b8997bd [improvement][headless]Merge function of QueryConverter abstraction to QueryParser. 2024-12-17 19:45:37 +08:00
jerryjzhang
84944fa341 [improvement][headless]Support count(1) as metric calculation expression. 2024-12-17 16:36:12 +08:00
jerryjzhang
b4c19533a4 [improvement][headless]bizName of dimension and metric shall not be the same as the field name in the database table. 2024-12-17 15:18:08 +08:00
jerryjzhang
831fbfe475 [improvement][headless]Refactor translator module to make code logic cleaner and more readable. 2024-12-15 21:03:57 +08:00
Jun Zhang
62b64c74a3 Merge branch 'tencentmusic:master' into aibi 2024-12-15 18:26:41 +08:00
jerryjzhang
ed5c129a4a [improvement][headless]Enhance translation of derived metrics and refactor translator code. 2024-12-15 17:25:33 +08:00
jerryjzhang
f9cc395e49 [improvement][headless]Refactor translator module to make code logic cleaner and more readable. 2024-12-15 08:48:48 +08:00
wwsheng009
2e28a4c7a5 Sap hanadb support,SAP HANA数据库适配支持 (#1958)
Some checks failed
supersonic CentOS CI / build (21) (push) Has been cancelled
supersonic mac CI / build (21) (push) Has been cancelled
supersonic ubuntu CI / build (21) (push) Has been cancelled
supersonic windows CI / build (21) (push) Has been cancelled
2024-12-13 09:06:21 +08:00
jerryjzhang
9f8793bfe2 [improvement][headless]Remove fixed TimeEnums fields. 2024-12-12 00:27:09 +08:00
Jun Zhang
a8868ba9ab Merge branch 'tencentmusic:master' into master 2024-12-11 22:48:14 +08:00
jerryjzhang
8c44c9f42f [improvement][headless]Remove fixed TimeEnums fields. 2024-12-11 21:33:43 +08:00
jerryjzhang
f97ac1da83 [improvement][headless]Introduce DerivedMetricConverter and optimize metric creation in S2VisitsDemo. 2024-12-11 17:51:02 +08:00
mislayming
07f6be51c7 SpringBoot3升级 (#1947)
Some checks failed
supersonic CentOS CI / build (21) (push) Has been cancelled
supersonic mac CI / build (21) (push) Has been cancelled
supersonic ubuntu CI / build (21) (push) Has been cancelled
supersonic windows CI / build (21) (push) Has been cancelled
2024-12-11 17:09:20 +08:00
siriusbo
3db9a0dcec [improvement][headless-core] 提升join准确性,修复join条件错误问题 (#1945)
Some checks are pending
supersonic CentOS CI / build (11) (push) Waiting to run
supersonic CentOS CI / build (21) (push) Waiting to run
supersonic CentOS CI / build (8) (push) Waiting to run
supersonic mac CI / build (11) (push) Waiting to run
supersonic mac CI / build (21) (push) Waiting to run
supersonic mac CI / build (8) (push) Waiting to run
supersonic ubuntu CI / build (11) (push) Waiting to run
supersonic ubuntu CI / build (21) (push) Waiting to run
supersonic ubuntu CI / build (8) (push) Waiting to run
supersonic windows CI / build (11) (push) Waiting to run
supersonic windows CI / build (21) (push) Waiting to run
supersonic windows CI / build (8) (push) Waiting to run
2024-12-09 22:13:27 +08:00
Jun Zhang
e55f43c737 [improvement]Use QueryWrapper in place of hard-coded SQLs (#1944)
Some checks are pending
supersonic CentOS CI / build (11) (push) Waiting to run
supersonic CentOS CI / build (21) (push) Waiting to run
supersonic CentOS CI / build (8) (push) Waiting to run
supersonic mac CI / build (11) (push) Waiting to run
supersonic mac CI / build (21) (push) Waiting to run
supersonic mac CI / build (8) (push) Waiting to run
supersonic ubuntu CI / build (11) (push) Waiting to run
supersonic ubuntu CI / build (21) (push) Waiting to run
supersonic ubuntu CI / build (8) (push) Waiting to run
supersonic windows CI / build (11) (push) Waiting to run
supersonic windows CI / build (21) (push) Waiting to run
supersonic windows CI / build (8) (push) Waiting to run
* [improvement][launcher]Use API to get element ID avoiding hard-code.

* [fix][launcher]Fix mysql scripts.

* [improvement][launcher]Support DuckDB database and refactor translator code structure.

* [improvement][headless-fe] Revamped the interaction for semantic modeling routing and successfully implemented the switching between dimension and dataset management.

* [improvement][Headless] Add table ddl in Dbschema

* [improvement][Headless] Add get database by type

* [improvement][Headless] Supports automatic batch creation of models based on db table names.

* [improvement][Headless] Supports getting domain by bizName

* [improvement][launcher]Refactor unit tests and demo data.

* [fix][launcher]Change default vector dimension to 512.

* [improvement](Dict) add dimValueAliasMap info for KnowledgeBaseService

* [improvement][headless]Use QueryWrapper to replace hard-code SQL in mapper xml.

* [improvement][chat]Introduce ChatMemory to delegate ChatMemoryDO.

* [fix][common]Fix embedding store sys configs.

* [fix][common]Fix postgres schema, using varchar instead of char.

* [improvement][launcher]Change supersonic docker deployment from mysql to postgres.

* [Fix][launcher]Fix a number of issues related to semantic modeling.

* [Fix][headless]Fix the evaluation logic of agg type.

* [fix][assembly]Fix Dockerfile and add docker compose run script.

* [fix][chat]Fix "multiple assignments to same column "similar_queries".

* [improvement][headless]Use LamdaQueryWrapper to avoid hard-coded column names.

* [improvement][headless]Refactor headless infra to support advanced semantic modelling.

* [improvement][headless]Change class name `Dim` to `Dimension`.

* [improvement][chat]Introduce `TimeFieldMapper` to always map time field.

* [fix][headless]Remove unnecessary dimension existence check.

* [fix][chat]Fix adjusted filters don't take effect.

---------
2024-12-08 13:32:29 +08:00
Jun Zhang
0fc29304a8 [improvement][launcher]Refactor unit tests and demo data. (#1935) 2024-12-01 21:08:26 +08:00
Jun Zhang
593597fe26 Refactor translator module (#1932)
* [improvement][Chat] Support agent permission management #1143

* [improvement][chat]Iterate LLM prompts of parsing and correction.

* [improvement][headless]Clean code logic of headless core.

* (fix) (chat) 记忆管理更新不生效 (#1912)

* [improvement][headless-fe] Added null-check conditions to the data formatting function.

* [improvement][headless]Clean code logic of headless translator.

* [improvement][headless-fe] Added permissions management for agents.

* [improvement][headless-fe] Unified the assistant's permission settings interaction to match the system style.

* [improvement](Dict)Support returns dict task list of dimensions by page

* [improvement][headless-fe] Revised the interaction for semantic modeling routing and implemented the initial version of metric management switching.

* [improvement][launcher]Set system property `s2.test` in junit tests in order to facilitate conditional breakpoints.

* [improvement][headless] add validateAndQuery interface in SqlQueryApiController

* [improvement][launcher]Use API to get element ID avoiding hard-code.

* [improvement][launcher]Support DuckDB database and refactor translator code structure.

---------

Co-authored-by: lxwcodemonkey <jolunoluo@tencent.com>
Co-authored-by: tristanliu <tristanliu@tencent.com>
Co-authored-by: daikon12 <1059907724@qq.com>
Co-authored-by: lexluo09 <39718951+lexluo09@users.noreply.github.com>
2024-11-30 00:27:33 +08:00
Jun Zhang
cb183b7ac8 [improvement][Chat] Support agent permission management (#1923)
* [improvement][Chat] Support agent permission management #1143

* [improvement][chat]Iterate LLM prompts of parsing and correction.

* [improvement][headless-fe] Added null-check conditions to the data formatting function.

* [improvement][headless]Clean code logic of headless translator.

---------

Co-authored-by: lxwcodemonkey <jolunoluo@tencent.com>
Co-authored-by: tristanliu <tristanliu@tencent.com>
2024-11-23 09:09:04 +08:00
Jun Zhang
e8c9855163 [improvement][launcher]Clean code logic of s2demos and remove tag-related constructs. 2024-11-16 17:23:35 +08:00
LXW
87729956e8 (improvement)(Headless) Refactor the SemanticModeller to rule first and then llm, and automatically infer field types in the rule method. (#1900)
Co-authored-by: lxwcodemonkey
2024-11-11 00:10:58 +08:00
jerryjzhang
1e5bf7909e [improvement][headless]Deprecate and remove entity-related abstraction and logic.#1876 2024-11-04 09:53:18 +08:00
jipeli
0cb1faaf53 (improvement)(Headless) Fixed duplicate fields error (#1870) 2024-10-31 22:46:46 +08:00
jipeli
858feb9c3c (improvement)(Headless) define the measure without agg funciton by the model (#1864) 2024-10-30 22:13:13 +08:00
jerryjzhang
53ddc67262 [improvement][chat]Sort parses in NL2SQLParser right after rule-based parsing. 2024-10-29 23:31:02 +08:00
jerryjzhang
b3d4440781 [improvement][chat]Support user feedback to multiple candidate semantic parses.#1847 2024-10-28 17:31:33 +08:00
lexluo09
ee4b218a1e [improvement][chat] Fix the issue of HDFS path concatenation (#1820) 2024-10-17 14:12:12 +08:00