Commit Graph

287 Commits

Author SHA1 Message Date
lexluo09
bbad302efd (improvement)(chat) Add the pluginList parameter to QueryContext. (#609) 2024-01-09 15:23:07 +08:00
lexluo09
ad8079e058 (improvement)(chat) The system settings module supports setting the number of returned parses. (#606) 2024-01-08 19:35:28 +08:00
lexluo09
8eef11f342 (improvement)(chat) Fix the thread safety issue in parallel insertion of nature. (#603) 2024-01-08 17:08:38 +08:00
lexluo09
b55b4c130e (improvement)(chat) Add queryFilters setting to queryContext. (#601) 2024-01-05 12:04:55 +08:00
lexluo09
602b9547b8 (improvement)(chat) Make it compatible with the case when the agent is empty. (#596) 2024-01-04 21:39:56 +08:00
lexluo09
ade96c3adc (improvement)(chat) Clean up unused code and optimize some code and make the tests pass (#595) 2024-01-04 18:22:40 +08:00
lexluo09
023e84c420 (improvement)(chat) Split chat into three modules: server, api, and core. (#594) 2024-01-04 16:56:49 +08:00
LXW
0858c13365 (improvement)(headless) Update server and core, server calls core one-way (#592)
* (improvement)(headless) Update server and core, server calls core one-way
* (improvement)(Auth) When obtaining the user information, determine whether the user is a system admin.
---------

Co-authored-by: jolunoluo
2024-01-04 12:08:12 +08:00
LXW
52fea5311d (improvement)(headless) Merge server and core in headless-api (#590)
(improvement)(headless) Merge server and core in headless-api

---------

Co-authored-by: jolunoluo
2024-01-02 20:08:23 +08:00
lexluo09
d72166944c (improvement)(chat) Add 'Few-shot Examples' display to the Chat chart. (#589) 2024-01-02 18:07:15 +08:00
LXW
e7f13572d7 (improvement)(headless) Modify the headless module structure to api, core and server (#588)
Co-authored-by: jolunoluo
2024-01-02 16:43:28 +08:00
codescracker
af1c560cc4 (fix)(chat):fix typo in s2sql and add prompt to output. (#581)
* 1.refactor the retrieval module. 2.refactor the http service module. 3.upgrade text2sql output format the parse for absolute time related expression in query.

* fix bug.

* upgrade the config module, now support config llm suppoted by langchain.

* fix conflicts.

* update text2sql config reload to be compitable with new config format.

* modify default config.

* 1.add self-consistency feature for text2sql. 2.upgrade llm api call from sync to async. 3.refactor text2sql module. 4. refactor semantical retriever modules.

* merege with upstream master

* add general retrieve service.

* add api service for sql_agent for crud opereations of few-shots examples.

* modify requirements

* add auto-cot feature

* 1. output log to a fixed log file.  2.allow few-shots examples tied to data model, and add strategy that extend examples when retrieved examples tied to a data model is not enough. 3. fix misformat in s2ql args.

* add prior_ext to output.

* fix type in in s2sql

* add prompt to output.

---------

Co-authored-by: shaoweigong <shaoweigong@tencent.com>
2024-01-02 16:35:33 +08:00
LXW
af6c8cdbda (feature)(headless) Add a new headless-api module and Change old headless-api module to headless-common (#582)
* (improvement)(headless) Add headless-api module

* (improvement)(headless) Change old headless-api module to headless-common

---------

Co-authored-by: jolunoluo <jolunoluo@tencent.com>
2023-12-28 22:14:57 +08:00
lexluo09
6c8ebdfe1a (improvement)(chat) Modify the default configuration related to text2sql numbers. (#575) 2023-12-27 20:39:08 +08:00
codescracker
b706c4efb4 Add feature to s2sql that allow few-shots example tied to data model. (#571)
* 1.refactor the retrieval module. 2.refactor the http service module. 3.upgrade text2sql output format the parse for absolute time related expression in query.

* fix bug.

* upgrade the config module, now support config llm suppoted by langchain.

* fix conflicts.

* update text2sql config reload to be compitable with new config format.

* modify default config.

* 1.add self-consistency feature for text2sql. 2.upgrade llm api call from sync to async. 3.refactor text2sql module. 4. refactor semantical retriever modules.

* merege with upstream master

* add general retrieve service.

* add api service for sql_agent for crud opereations of few-shots examples.

* modify requirements

* add auto-cot feature

* 1. output log to a fixed log file.  2.allow few-shots examples tied to data model, and add strategy that extend examples when retrieved examples tied to a data model is not enough. 3. fix misformat in s2ql args.

* add prior_ext to output.

---------

Co-authored-by: shaoweigong <shaoweigong@tencent.com>
2023-12-27 19:39:50 +08:00
LXW
40c86810bb (improvement)(chat) Modify query type rules in QueryTypeParser (#570)
Co-authored-by: jolunoluo
2023-12-25 20:54:04 +08:00
LXW
e0955c0618 (improvement)(chat) Replace the table name in correct sql with a string concatenated by the model name in FromCorrector (#566)
Co-authored-by: jolunoluo
2023-12-22 19:24:57 +08:00
lexluo09
0a1f446fb8 (improvement)(chat) After a failed query, the absence of a returned queryId causes the recommendation question call to fail. (#562) 2023-12-21 22:10:48 +08:00
lexluo09
7b580b7c94 (improvement)(chat) Introduce a separate log file to record key outputs in the pipeline. (#559) 2023-12-20 22:45:20 +08:00
LXW
2631352c30 (improvement)(common) Add JsonUtil configuration to avoid deserialization failure when enumeration values change (#558)
Co-authored-by: jolunoluo
2023-12-20 22:11:36 +08:00
Scott
f7914ff6f4 (fix):fill values into param options in webservice query, some refactor (#557) 2023-12-20 19:56:06 +08:00
lexluo09
150d67f903 (improvement)(semantic) Change 'semantic' to 'headless' (#555) 2023-12-20 17:08:16 +08:00
lexluo09
1d9b6d6877 [improvement](chat) optimize functionCall result format (#549) 2023-12-19 15:46:17 +08:00
LXW
d8930e8906 (improvement)(chat) Optimize the conditions for chatDemo loading (#548)
Co-authored-by: jolunoluo
2023-12-19 14:57:30 +08:00
LXW
7cb7697353 (improvement)(chat) Modify the order of demo loaders (#542)
Co-authored-by: jolunoluo
2023-12-18 23:15:23 +08:00
LXW
e7d52f87f0 (improvement)(python) run.py read files in utf-8 format (#540)
Co-authored-by: jolunoluo
2023-12-18 22:07:27 +08:00
lexluo09
e08435902a [improvement](chat) fix SqlGeneration format issues (#538) 2023-12-18 22:04:40 +08:00
LXW
d7f1f06daf (improvement)(chat) Remove candidateParses in ParseResp (#536)
Co-authored-by: jolunoluo
2023-12-18 21:10:53 +08:00
LXW
d7fafa361d (improvement)(chat) Switching metric supports default aggregation method of metric (#534)
Co-authored-by: jolunoluo
2023-12-18 18:51:21 +08:00
Scott
0c69651ef3 (fix):remove parantheses in generated sql (#532) 2023-12-18 17:57:06 +08:00
LXW
61641ecb00 (improvement)(chat) The history record only retains the query result of the first parse (#526)
Co-authored-by: jolunoluo
2023-12-18 10:20:50 +08:00
lexluo09
fe75b3e393 [improvement](chat) Add persistence to InMemoryEmbeddingStore, fix the issue of PythonServiceS2EmbeddingStore being empty. (#524) 2023-12-17 11:04:29 +08:00
lexluo09
59c21ea19a [improvement](chat) support filterCondition in InMemoryS2EmbeddingStore (#523) 2023-12-16 20:04:01 +08:00
lexluo09
276b224c13 [improvement](chat) Support sqlGenerationMode in python interface (#521) 2023-12-16 17:24:24 +08:00
lexluo09
f03da53d6f [improvement](chat) Support frontend configuration for S2SQL generation method. (#520) 2023-12-16 17:10:34 +08:00
lexluo09
9201550027 [improvement](chat) Support new four methods of generating SQL using SqlGeneration large models. (#519) 2023-12-16 14:44:56 +08:00
LXW
4dae84034e (improvement)(chat) Rename SimilarMetricProcessor, SimilarQueryProcessor to MetricRecommendProcessor, QueryRecommendProcessor (#513)
Co-authored-by: jolunoluo
2023-12-15 14:56:44 +08:00
LXW
e6eac03ec6 (improvement)(chat) Rename DrillDownDimensionProcessor to DimensionRecommendProcessor (#512)
Co-authored-by: jolunoluo
2023-12-15 11:48:41 +08:00
LXW
e9a479e2df (improvement)(chat) Add DrillDownDimensionProcessor and SimilarQueryProcessor to obtain recommended dimensions and similar queries (#511)
Co-authored-by: jolunoluo
2023-12-15 10:27:33 +08:00
lexluo09
3d30632b41 [improvement](python) Remove unused Python-related files (#507) 2023-12-14 15:19:52 +08:00
LXW
fda5a577d6 (improvement)(chat) rename QueryResponder to ExecuteResultProcessor and add ResultProcessor (#504)
Co-authored-by: jolunoluo
2023-12-13 22:16:42 +08:00
LXW
f89be48e98 (improvement)(semantic) save metric default query config (#503)
Co-authored-by: jolunoluo
2023-12-13 21:29:15 +08:00
lexluo09
340cb2c835 [improvement](chat) Resolve the issue of an empty result in schemaLinkParse. (#500) 2023-12-13 11:18:15 +08:00
jerryjzhang
86c2f96942 [improvement][fix]Fix codestyle 2023-12-12 15:57:37 +08:00
lexluo09
73899e3174 [improvement](chat) Change llmparser to pyllm, retrieve LLMProxy from environment variables, defaulting to JavaLLMProxy. (#497) 2023-12-12 14:29:35 +08:00
Scott
49bb2c6d8b fix:use correct sql generated mode in java code (#496) 2023-12-11 21:46:57 +08:00
Scott
f3f60af231 (fix):fix python service interact (#494) 2023-12-11 19:31:36 +08:00
jerryjzhang
3cdfcae01c [improvement][chat]Add javadoc to Mapper related classes 2023-12-11 18:07:26 +08:00
lexluo09
0c6efada43 [improvement](chat) Merge HanlpDictMapper and FuzzyNameMapper into KeywordMapper. (#493)
Co-authored-by: lexluo <lexluo@tencent.com>
2023-12-11 17:22:57 +08:00
codescracker
d79f73eab6 add auto-CoT feature (#483)
* 1.refactor the retrieval module. 2.refactor the http service module. 3.upgrade text2sql output format the parse for absolute time related expression in query.

* fix bug.

* upgrade the config module, now support config llm suppoted by langchain.

* fix conflicts.

* update text2sql config reload to be compitable with new config format.

* modify default config.

* 1.add self-consistency feature for text2sql. 2.upgrade llm api call from sync to async. 3.refactor text2sql module. 4. refactor semantical retriever modules.

* merege with upstream master

* add general retrieve service.

* add api service for sql_agent for crud opereations of few-shots examples.

* modify requirements

* add auto-cot feature

---------

Co-authored-by: shaoweigong <shaoweigong@tencent.com>
2023-12-11 16:07:49 +08:00