Commit Graph

281 Commits

Author SHA1 Message Date
jerryjzhang
28d5f38ffb [improvement][headless]Refactor headless infra to support advanced semantic modelling. 2024-12-04 14:56:34 +08:00
jerryjzhang
3e903b862e [Fix][headless]Fix the evaluation logic of agg type. 2024-12-02 23:39:53 +08:00
jerryjzhang
0ce79cbfc0 [Fix][launcher]Fix a number of issues related to semantic modeling. 2024-12-02 21:38:15 +08:00
Jun Zhang
0fc29304a8 [improvement][launcher]Refactor unit tests and demo data. (#1935) 2024-12-01 21:08:26 +08:00
lxwcodemonkey
6ecc5a9362 [improvement][Headless] Supports automatic batch creation of models based on db table names. 2024-11-30 22:22:56 +08:00
lxwcodemonkey
d22a7d1cc6 [improvement][Headless] Add table ddl in Dbschema 2024-11-30 20:56:58 +08:00
Jun Zhang
593597fe26 Refactor translator module (#1932)
* [improvement][Chat] Support agent permission management #1143

* [improvement][chat]Iterate LLM prompts of parsing and correction.

* [improvement][headless]Clean code logic of headless core.

* (fix) (chat) 记忆管理更新不生效 (#1912)

* [improvement][headless-fe] Added null-check conditions to the data formatting function.

* [improvement][headless]Clean code logic of headless translator.

* [improvement][headless-fe] Added permissions management for agents.

* [improvement][headless-fe] Unified the assistant's permission settings interaction to match the system style.

* [improvement](Dict)Support returns dict task list of dimensions by page

* [improvement][headless-fe] Revised the interaction for semantic modeling routing and implemented the initial version of metric management switching.

* [improvement][launcher]Set system property `s2.test` in junit tests in order to facilitate conditional breakpoints.

* [improvement][headless] add validateAndQuery interface in SqlQueryApiController

* [improvement][launcher]Use API to get element ID avoiding hard-code.

* [improvement][launcher]Support DuckDB database and refactor translator code structure.

---------

Co-authored-by: lxwcodemonkey <jolunoluo@tencent.com>
Co-authored-by: tristanliu <tristanliu@tencent.com>
Co-authored-by: daikon12 <1059907724@qq.com>
Co-authored-by: lexluo09 <39718951+lexluo09@users.noreply.github.com>
2024-11-30 00:27:33 +08:00
jerryjzhang
a058dc8b6e [improvement][launcher]Support DuckDB database and refactor translator code structure. 2024-11-30 00:23:16 +08:00
jerryjzhang
860fd5d299 [improvement][headless]Clean code logic of headless translator. 2024-11-25 21:12:03 +08:00
daikon12
c22e3ef2e8 [improvement](Dict)Support returns dict task list of dimensions by page 2024-11-24 16:20:57 +08:00
Jun Zhang
cb183b7ac8 [improvement][Chat] Support agent permission management (#1923)
* [improvement][Chat] Support agent permission management #1143

* [improvement][chat]Iterate LLM prompts of parsing and correction.

* [improvement][headless-fe] Added null-check conditions to the data formatting function.

* [improvement][headless]Clean code logic of headless translator.

---------

Co-authored-by: lxwcodemonkey <jolunoluo@tencent.com>
Co-authored-by: tristanliu <tristanliu@tencent.com>
2024-11-23 09:09:04 +08:00
jerryjzhang
d7586a5d3b [improvement][headless]Clean code logic of headless translator. 2024-11-20 22:23:53 +08:00
Jun Zhang
e8c9855163 [improvement][launcher]Clean code logic of s2demos and remove tag-related constructs. 2024-11-16 17:23:35 +08:00
LXW
87729956e8 (improvement)(Headless) Refactor the SemanticModeller to rule first and then llm, and automatically infer field types in the rule method. (#1900)
Co-authored-by: lxwcodemonkey
2024-11-11 00:10:58 +08:00
jerryjzhang
14a19a901f [improvement][project]Simplify code logic in multiple modules. 2024-11-10 14:31:12 +08:00
Jun Zhang
e0e167fd40 [improvement][chat]Refactor code logic in rule-based parsing. 2024-11-09 15:49:08 +08:00
lexluo09
d4a9d5a7e6 [fix][chat] The terms in the descriptor of the term no longer undergo descriptor mapping (#1897) 2024-11-09 14:16:55 +08:00
Jun Zhang
c9c6dc4e44 Merge a number of fixes and improvements (#1896) 2024-11-09 00:23:02 +08:00
jerryjzhang
9a05b5cce6 [improvement][headless]Deprecate and remove entity-related abstraction and logic.#1876 2024-11-04 10:17:30 +08:00
jerryjzhang
1e5bf7909e [improvement][headless]Deprecate and remove entity-related abstraction and logic.#1876 2024-11-04 09:53:18 +08:00
lexluo09
a43e47e491 [improvement][headless] Determine if the SQL is supported by agg for more complex scenarios (#1872) 2024-11-01 10:37:26 +08:00
daikon
4c929d59a8 (improvement)(knowledge) support DimValueAliasReq single (#1871) 2024-10-31 23:52:31 +08:00
lexluo09
cf359f3e2f [improvement][chat] Optimize and modify the mapper method for terminology (#1866) 2024-10-31 11:18:35 +08:00
LXW
838745d415 (improvement)(Headless) Implement SemanticModeller through SPI (#1865)
Co-authored-by: lxwcodemonkey
2024-10-30 23:32:38 +08:00
jerryjzhang
53ddc67262 [improvement][chat]Sort parses in NL2SQLParser right after rule-based parsing. 2024-10-29 23:31:02 +08:00
jerryjzhang
cbb76550c7 [improvement][chat]Optimize NL2SQL parsing logic. 2024-10-29 20:33:32 +08:00
LXW
ec6d7ba93d (improvement)(Headless) Support setting fields as foreign keys when creating data-model (#1857)
Co-authored-by: lxwcodemonkey
2024-10-29 18:42:31 +08:00
lexluo09
0383683ac0 [improvement][chat] In the search interface, modelId and modelName are changed to dataSetId and dataSetName (#1855) 2024-10-29 16:01:02 +08:00
jerryjzhang
826354cdf1 [improvement][launcher]Add DictConf for dimensions in S2VisitsDemo. 2024-10-29 14:38:58 +08:00
jerryjzhang
400b9f86f0 [improvement][chat]Modify core workflow of NL2SQLParser, always invoking rule-based parsers first.#1729 2024-10-29 13:12:56 +08:00
jerryjzhang
14badcd4ae [improvement][chat]Introduce separate ChatParseResp in Chat module. 2024-10-28 12:44:39 +08:00
jerryjzhang
eb28d832bc [improvement][chat]Support user feedback to semantic parse info.#1729 2024-10-28 02:07:54 +08:00
jerryjzhang
b69ee81d58 [improvement][chat]Move generation of semantic text info and rewrite of error message to dedicated ResultProcessor. 2024-10-27 22:34:16 +08:00
jerryjzhang
bb363a0286 [improvement][headless-chat]Incorporate Request into Context objects, removing unnecessary copy. 2024-10-27 18:47:34 +08:00
LXW
bd82b0904b (improvement)(Headless) Add integration testing for building data-model by LLM (#1848)
Co-authored-by: lxwcodemonkey
2024-10-27 18:16:59 +08:00
lexluo09
473329d398 [improvement][chat] Filter at the lowest level in the Map based on the dataSetId (#1834) 2024-10-20 21:51:57 +08:00
LXW
1d84e00887 (improvement)(Headless) Supports batch creation of models by specifying db table names (#1833)
Co-authored-by: lxwcodemonkey
2024-10-20 20:17:11 +08:00
LXW
1f208ffcb7 (improvement)(Common) Move ChatModel management to common modules and add chatApp for data-model builder (#1831)
Co-authored-by: lxwcodemonkey
2024-10-20 18:42:08 +08:00
jerryjzhang
0cce0a76b4 [feature][headless-chat]Introduce ChatApp to support more flexible chat model config.#1739 2024-10-12 11:58:34 +08:00
jerryjzhang
2717a1603c [feature][headless-chat]Introduce ChatApp to support more flexible chat model config.#1739 2024-10-12 11:51:37 +08:00
lexluo09
72d01bb419 [improvement][chat] Optimize the MemoryReviewTask task by adding individual exception handling (#1788) 2024-10-11 23:09:13 +08:00
daikon
0a86a63937 opt queryDictValue (#1787) 2024-10-11 22:27:46 +08:00
LXW
fbf0ea0627 (improvement)(Headless) Support building data-model by LLM #1319 (#1784)
Co-authored-by: lxwcodemonkey
2024-10-11 14:34:17 +08:00
jipeli
50b0036d0f [improvement][headless] Fix permission exception loss (#1764) (#1782) 2024-10-11 12:00:41 +08:00
jerryjzhang
248f4f83f6 [feature][chat]Refactor chat model config related codes.#1739 2024-10-09 17:27:07 +08:00
daikon
ee6b8a1044 (improvement)(knowledge) add keyValue parameter for queryDictValue (#1749) 2024-10-08 00:10:34 +08:00
lexluo09
71a9954be5 (improvement)[build] Use Spotless to customize the code formatting (#1750) 2024-10-04 00:05:04 +08:00
lexluo09
8693e7f8a7 [improvement][headless] In the translator, adopt Calcite to support multiple WITH structures and optimize the Calcite code. (#1747) 2024-09-30 18:53:27 +08:00
lexluo09
de2385cd2c [improvement][headless] Backend uniformly supports other types of databases. (#1745) 2024-09-30 15:02:58 +08:00
feelshana
e351a1c541 (fix)(headless)fix the issue that the time schema in LllM is not matched with the sql which modifiedy by sqlAdaptor when the dateformat updated;also fix the issue that the model update is not effected on dimension(#1720)(1719) (#1725) 2024-09-28 09:36:49 +08:00