[feature](project) delete package-lock.json (#44)

Co-authored-by: williamhliu <williamhliu@tencent.com>

.gitignore

@@ -14,4 +14,5 @@ assembly/runtime/*
 *.umi/
 /assembly/deploy
 /runtime
-**/.flattened-pom.xml
+**/.flattened-pom.xml
+__pycache__/

CHANGELOG.md

@@ -4,7 +4,17 @@
 - "Breaking Changes" describes any changes that may break existing functionality or cause
   compatibility issues with previous versions.
 
+## SuperSonic [0.7.3] - 2023-08-29
 
+### Added
+- meet checkstyle code requirements
+- save parseInfo after parsing
+- add time statistics
+- add agent
+
+### Updated
+- dsl where condition is used for front-end display
+- dsl remove context inheritance
 
 ## SuperSonic [0.7.2] - 2023-08-12
 

README.md

@@ -10,35 +10,39 @@ English | [中文](README_CN.md)
 
 The emergence of Large Language Model (LLM) like ChatGPT is reshaping the way information is retrieved. In the field of data analytics, both academia and industry are primarily focused on leveraging LLM to convert natural language queries into SQL queries. While some works show promising results, they are still not applicable to real-world scenarios.
 
-From our perspective, the key to filling the real-world gap lies in two aspects: 
+From our perspective, the key to filling the real-world gap lies in three aspects: 
-1. Utilize a combination of rule-based and model-based semantic parsers to deal with different scenarios.
+1. Complement the LLM-based semantic parser with rule-based semantic parsers to improve **efficiency**(in terms of latency and cost).
-2. Introduce a semantic model layer encapsulating the underlying data complexity(joins, formulas, etc) to simplify semantic parsing.
+2. Augment semantic parsing with schema mappers(as a kind of preprocessor) and semantic correctors(as a kind of postprocessor) to improve **accuracy** and **stability**.
+3. Introduce a semantic layer encapsulating underlying data context(joins, formulas, etc) to reduce **complexity**.
 
 With these ideas in mind, we develop SuperSonic as a practical reference implementation and use it to power our real-world products. Additionally, to facilitate further development of data chatbot, we decide to open source SuperSonic as an extensible framework.
 
 ## Out-of-the-box Features
 
-- Built-in graphical interface for business users to enter data queries 
+- Built-in CUI(Chat User Interface) for *business users* to enter data queries
-- Built-in graphical interface for analytics engineers to manage semantic models
+- Built-in GUI(Graphical User Interface) for *analytics engineers* to build semantic models
+- Built-in GUI for *system administrators* to manage chat plugins and agents
 - Support input auto-completion as well as query recommendation
 - Support multi-turn conversation and history context management 
-- Support three-level permission control: domain-level, column-level and row-level 
+- Support four-level permission control: domain-level, model-level, column-level and row-level
 
 ## Extensible Components
 
-The high-level architecture and main process flow is shown in below diagram:
+The high-level architecture and main process flow is as follows:
 
-<img src="./docs/images/supersonic_components.png" height="80%" width="80%" align="center"/> 
+<img src="./docs/images/supersonic_components.png" height="65%" width="65%" align="center"/> 
 
-- **Chat Interface:** accepts natural language queries and answer results with appropriate visualization charts. It supports input auto-completion as well as multi-turn conversation.
+- **Knowledge Base:** extracts schema information periodically from the semantic models and build dictionary and index to facilitate schema mapping.
 
-- **Modeling Interface:** empowers analytics engineers to visually define and maintain semantic models. The configurations related to access permission and chat conversation can also be set on the UI.
+- **Schema Mapper:** identifies references to schema elements(metrics/dimensions/entities/values) in user queries. It matches the query text against the knowledge base.
 
-- **Schema Mapper Chain:** identifies references to schema elements(metrics/dimensions/entities/values) in user queries. It matches the query text against a knowledge base constructed from the semantic models.
+- **Semantic Parser:** understands user queries and extracts semantic information. It consists of a combination of rule-based and model-based parsers, each of which deals with specific scenarios.
 
-- **Semantic Parser Chain:** understands user queries and extract semantic information. It consists of a combination of rule-based and model-based parsers, each of which deals with specific scenarios.
+- **Semantic Corrector:** checks validity of extracted semantic information and performs correction and optimization if needed.
 
-- **Semantic Query:** performs execution according to extracted semantic information. It generates SQL queries and executes them against physical data models.
+- **Semantic Layer:** performs execution according to extracted semantic information. It generates SQL queries and executes them against physical data models.
+
+- **Chat Plugin:** extends functionality with third-party tools. The LLM is going to select the most suitable one, given all configured plugins with function description and sample questions.
 
 ## Quick Demo
 
@@ -46,11 +50,11 @@ SuperSonic comes with sample semantic models as well as chat conversations that
 
 - Download the latest prebuilt binary from the [release page](https://github.com/tencentmusic/supersonic/releases)
 - Run script "bin/start-standalone.sh" to start a standalone server
-- Visit http://localhost:9080 in browser to start exploration
+- Visit http://localhost:9080 in the browser to start exploration
 
 ## How to Build
 
-SuperSonic can be deployed in two modes: standalone (intended for quick demo) and distributed (intended for production). 
+SuperSonic can be deployed in two modes: standalone (for a quick demo) and distributed (for production use). 
 
 ### Build for Standalone Mode 
  
@@ -59,3 +63,13 @@ Pull the source code and run script "assembly/bin/build-standalone.sh" to build
 ### Build for Distributed Mode 
 
 Pull the source code and run scripts "assembly/bin/build-chat.sh" and "assembly/bin/build-semantic.sh" separately to build packages.
+
+### Build for Local Development
+
+Pull the source code and run script "assembly/bin/build-ide.sh" and run bootstrap class "StandaloneLauncher" in the IDE.
+
+## WeChat Contact
+
+Please join the chat group to suggest feedbacks or ideas:
+
+<img src="./docs/images/wechat_contact.jpeg" height="30%" width="30%" align="center"/> 

README_CN.md

@@ -8,35 +8,39 @@
 
 大型语言模型（LLMs）如ChatGPT的出现正在重塑信息检索的方式。在数据分析领域，学术界和工业界主要关注利用深度学习模型将自然语言查询转换为SQL查询。虽然一些工作显示出有前景的结果，但它们还并不适用于实际场景。
 
-在我们看来，为了在实际场景发挥价值，有两个关键点：
+在我们看来，为了在实际场景发挥价值，有三个关键点：
-1. 将基于规则和基于模型的语义解析器相结合，发挥各自优势，以便处理不同的场景。
+1. 在基于大模型语义解析器基础上，增加基于规则的解析器，提升语义解析的**效率**。
-2. 引入语义模型层来封装数据底层的复杂性（关联、公式等），从而简化语义解析的求解空间。
+2. 加入模式映射器和语义修正器，来增强语义解析能力，提升语义解析的**准确性**和**稳定性**。
+3. 引入语义模型层，封装底层数据的上下文（关联、公式等），降低语义解析的**复杂性**。
 
 为了验证上述想法，我们开发了超音数项目，并将其应用在实际的内部产品中。与此同时，我们将超音数作为一个可扩展的框架开源，希望能够促进数据问答对话领域的进一步发展。
 
 ## 开箱即用的特性
 
-- 内置图形界面以便业务用户输入数据查询。
+- 内置对话界面以便*业务用户*输入数据查询。
-- 内置图形界面以便分析工程师管理语义模型。
+- 内置图形界面以便*分析工程师*构建语义模型。
+- 内置图形界面以便*系统管理员*管理问答插件和助理。
 - 支持文本输入的联想和查询问题的推荐。
 - 支持多轮对话，根据语境自动切换上下文。
-- 支持三级权限控制：主题域级、列级、行级。
+- 支持四级权限控制：主题域级、模型级、列级、行级。
 
 ## 易于扩展的组件
 
 超音数的整体架构和主流程如下图所示：
 
-<img src="./docs/images/supersonic_components.png" height="80%" width="80%" align="center"/> 
+<img src="./docs/images/supersonic_components.png" height="65%" width="65%" align="center"/> 
 
-- **问答对话界面(chat interface)**：接受用户查询并选择合适的可视化图表呈现结果，支持输入联想和多轮对话。
+- **知识库(Knowledge Base)：** 定期从语义模型中提取相关的模式信息，构建词典和索引，以便后续的模式映射。
 
-- **语义建模界面(modeling interface)**：使分析工程师能够通过可视化方式定义和维护语义模型，与访问权限和聊天对话相关的配置也可以在用户界面上设置。
+- **模式映射器(Schema Mapper)：** 将自然语言文本在知识库中进行匹配，为后续的语义解析提供相关信息。
 
-- **模式映射器(schema mapper chain)**：基于语义模型构建知识库，然后将自然语言文本在知识库中进行匹配，为后续的语义解析提供相关信息。
+- **语义解析器(Semantic Parser)：** 理解用户查询并抽取语义信息，其由一组基于规则和基于模型的解析器组成，每个解析器可应对不同的特定场景。
 
-- **语义解析器(semantic parser chain)**：理解用户查询并抽取语义信息，其由一组基于规则和基于模型的解析器组成，每个解析器可应对不同的特定场景。
+- **语义修正器(Semantic Corrector)：** 检查语义信息的合法性，对不合法的信息做修正和优化处理。
 
-- **语义查询(semantic query)**: 根据语义信息生成物理SQL执行查询。
+- **语义模型层(Semantic Layer)：** 根据语义信息生成物理SQL执行查询。
+
+- **问答插件(Chat Plugin)：** 通过第三方工具扩展功能。给定所有配置的插件及其功能描述和示例问题，大语言模型将选择最合适的插件。
 
 ## 快速体验
 
@@ -52,8 +56,18 @@
 
 ### Standalone模式构建
 
-下载源码包，运行脚本"assembly/bin/build-standalone.sh"，将所有服务一起编译打包
+下载源码包，运行脚本"assembly/bin/build-standalone.sh"，将所有服务一起编译打包。
 
 ### Distributed模式构建
 
-下载源码包，分别运行脚本"assembly/bin/build-chat.sh"、"assembly/bin/build-semantic.sh"，为问答层服务和语义层服务编译打包
+下载源码包，分别运行脚本"assembly/bin/build-chat.sh"、"assembly/bin/build-semantic.sh"，为问答层服务和语义层服务编译打包。
+
+### 本地开发构建
+
+下载源码包，运行脚本"assembly/bin/build-ide.sh"，然后在本地IDE运行启动类"StandaloneLauncher"。
+
+## 微信联系方式
+
+欢迎加入微信群反馈建议：
+
+<img src="./docs/images/wechat_contact.jpeg" height="30%" width="30%" align="center"/> 

assembly/bin/build-chat.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 sbinDir=$(cd "$(dirname "$0")"; pwd)
-baseDir=$(readlink -f $sbinDir/../)
+baseDir=$(cd "$sbinDir/.." && pwd -P)
 runtimeDir=$baseDir/runtime
 buildDir=$baseDir/build
 

assembly/bin/build-ide.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 sbinDir=$(cd "$(dirname "$0")"; pwd)
-baseDir=$(readlink -f $sbinDir/../)
+baseDir=$(cd "$sbinDir/.." && pwd -P)
 buildDir=$baseDir/build
 
 cd $baseDir/bin

assembly/bin/build-semantic.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 sbinDir=$(cd "$(dirname "$0")"; pwd)
-baseDir=$(readlink -f $sbinDir/../)
+baseDir=$(cd "$sbinDir/.." && pwd -P)
 runtimeDir=$baseDir/runtime
 buildDir=$baseDir/build
 

assembly/bin/build-standalone.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 sbinDir=$(cd "$(dirname "$0")"; pwd)
-baseDir=$(readlink -f $sbinDir/../)
+baseDir=$(cd "$sbinDir/.." && pwd -P)
 runtimeDir=$baseDir/runtime
 buildDir=$baseDir/build
 

assembly/bin/start-chat.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 sbinDir=$(cd "$(dirname "$0")"; pwd)
-baseDir=$(readlink -f $sbinDir/../)
+baseDir=$(cd "$sbinDir/.." && pwd -P)
 runtimeDir=$baseDir/../runtime
 buildDir=$baseDir/build
 

assembly/bin/start-semantic.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 sbinDir=$(cd "$(dirname "$0")"; pwd)
-baseDir=$(readlink -f $sbinDir/../)
+baseDir=$(cd "$sbinDir/.." && pwd -P)
 runtimeDir=$baseDir/../runtime
 buildDir=$baseDir/build
 

assembly/bin/start-standalone.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 sbinDir=$(cd "$(dirname "$0")"; pwd)
-baseDir=$(readlink -f $sbinDir/../)
+baseDir=$(cd "$sbinDir/.." && pwd -P)
 runtimeDir=$baseDir/../runtime
 buildDir=$baseDir/build
 
@@ -29,4 +29,4 @@ rm -fr  ${buildDir}/supersonic-webapp
 #start standalone service
 sh ${runtimeDir}/supersonic-standalone/bin/service.sh restart
 #start llm service
-sh ${runtimeDir}/supersonic-standalone/llm/bin/service.sh restart
+sh ${runtimeDir}/supersonic-standalone/llm/bin/service.sh restart

auth/api/src/main/java/com/tencent/supersonic/auth/api/authentication/config/AuthenticationConfig.java

@@ -19,7 +19,7 @@ public class AuthenticationConfig {
     @Value("${authentication.token.secret:secret}")
     private String tokenSecret;
 
-    @Value("${authentication.token.http.header.key:Auth}")
+    @Value("${authentication.token.http.header.key:Authorization}")
     private String tokenHttpHeaderKey;
 
 

auth/api/src/main/java/com/tencent/supersonic/auth/api/authorization/request/QueryAuthResReq.java

@@ -10,8 +10,6 @@ import lombok.ToString;
 @ToString
 public class QueryAuthResReq {
 
-    private String user;
-
     private List<String> departmentIds = new ArrayList<>();
 
     private List<AuthRes> resources;

auth/api/src/main/java/com/tencent/supersonic/auth/api/authorization/service/AuthService.java

@@ -1,10 +1,11 @@
 package com.tencent.supersonic.auth.api.authorization.service;
 
+import com.tencent.supersonic.auth.api.authentication.pojo.User;
 import com.tencent.supersonic.auth.api.authorization.pojo.AuthGroup;
 import com.tencent.supersonic.auth.api.authorization.request.QueryAuthResReq;
 import com.tencent.supersonic.auth.api.authorization.response.AuthorizedResourceResp;
 import java.util.List;
-import javax.servlet.http.HttpServletRequest;
+
 
 public interface AuthService {
 
@@ -14,5 +15,5 @@ public interface AuthService {
 
     void removeAuthGroup(AuthGroup group);
 
-    AuthorizedResourceResp queryAuthorizedResources(QueryAuthResReq req, HttpServletRequest request);
+    AuthorizedResourceResp queryAuthorizedResources(QueryAuthResReq req, User user);
 }

auth/authentication/src/main/java/com/tencent/supersonic/auth/authentication/config/TppConfig.java

@@ -1,21 +0,0 @@
-package com.tencent.supersonic.auth.authentication.config;
-
-
-import lombok.Data;
-import org.springframework.beans.factory.annotation.Value;
-import org.springframework.context.annotation.Configuration;
-
-@Data
-@Configuration
-public class TppConfig {
-
-    @Value(value = "${auth.app.secret:}")
-    private String appSecret;
-
-    @Value(value = "${auth.app.key:}")
-    private String appKey;
-
-    @Value(value = "${auth.oa.url:}")
-    private String tppOaUrl;
-
-}

auth/authentication/src/main/java/com/tencent/supersonic/auth/authentication/persistence/repository/Impl/UserRepositoryImpl.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.auth.authentication.persistence.repository.Impl;
+package com.tencent.supersonic.auth.authentication.persistence.repository.impl;
 
 
 import com.tencent.supersonic.auth.authentication.persistence.dataobject.UserDO;

auth/authentication/src/main/java/com/tencent/supersonic/auth/authentication/rest/UserController.java

@@ -11,12 +11,12 @@ import java.util.Set;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 import lombok.extern.slf4j.Slf4j;
+import org.springframework.web.bind.annotation.RestController;
+import org.springframework.web.bind.annotation.RequestMapping;
 import org.springframework.web.bind.annotation.GetMapping;
-import org.springframework.web.bind.annotation.PathVariable;
 import org.springframework.web.bind.annotation.PostMapping;
 import org.springframework.web.bind.annotation.RequestBody;
-import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.PathVariable;
-import org.springframework.web.bind.annotation.RestController;
 
 @RestController
 @RequestMapping("/api/auth/user")

auth/authentication/src/main/java/com/tencent/supersonic/auth/authentication/utils/UserTokenUtils.java

@@ -9,7 +9,6 @@ import static com.tencent.supersonic.auth.api.authentication.constant.UserConsta
 import static com.tencent.supersonic.auth.api.authentication.constant.UserConstants.TOKEN_USER_ID;
 import static com.tencent.supersonic.auth.api.authentication.constant.UserConstants.TOKEN_USER_NAME;
 import static com.tencent.supersonic.auth.api.authentication.constant.UserConstants.TOKEN_USER_PASSWORD;
-
 import com.tencent.supersonic.auth.api.authentication.config.AuthenticationConfig;
 import com.tencent.supersonic.auth.api.authentication.pojo.User;
 import com.tencent.supersonic.auth.api.authentication.pojo.UserWithPassword;
@@ -22,9 +21,11 @@ import java.util.Date;
 import java.util.HashMap;
 import java.util.Map;
 import javax.servlet.http.HttpServletRequest;
+import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.StringUtils;
 import org.springframework.stereotype.Component;
 
+@Slf4j
 @Component
 public class UserTokenUtils {
 
@@ -68,7 +69,9 @@ public class UserTokenUtils {
     public UserWithPassword getUserWithPassword(HttpServletRequest request) {
         String token = request.getHeader(authenticationConfig.getTokenHttpHeaderKey());
         if (StringUtils.isBlank(token)) {
-            throw new AccessException("token is blank, get user failed");
+            String message = "token is blank, get user failed";
+            log.warn("{}, uri: {}", message, request.getServletPath());
+            throw new AccessException(message);
         }
         final Claims claims = getClaims(token);
         Long userId = Long.parseLong(claims.getOrDefault(TOKEN_USER_ID, 0).toString());

auth/authorization/src/main/java/com/tencent/supersonic/auth/authorization/application/AuthServiceImpl.java

@@ -2,27 +2,27 @@ package com.tencent.supersonic.auth.authorization.application;
 
 import com.google.common.base.Strings;
 import com.google.gson.Gson;
+import com.tencent.supersonic.auth.api.authentication.pojo.User;
 import com.tencent.supersonic.auth.api.authentication.service.UserService;
-import com.tencent.supersonic.auth.api.authorization.pojo.AuthGroup;
 import com.tencent.supersonic.auth.api.authorization.pojo.AuthRes;
 import com.tencent.supersonic.auth.api.authorization.pojo.AuthResGrp;
-import com.tencent.supersonic.auth.api.authorization.pojo.AuthRule;
 import com.tencent.supersonic.auth.api.authorization.pojo.DimensionFilter;
 import com.tencent.supersonic.auth.api.authorization.request.QueryAuthResReq;
 import com.tencent.supersonic.auth.api.authorization.response.AuthorizedResourceResp;
 import com.tencent.supersonic.auth.api.authorization.service.AuthService;
-import java.util.ArrayList;
+import com.tencent.supersonic.auth.api.authorization.pojo.AuthGroup;
-import java.util.List;
+import com.tencent.supersonic.auth.api.authorization.pojo.AuthRule;
-import java.util.Map;
-import java.util.Objects;
-import java.util.Set;
-import java.util.stream.Collectors;
-import javax.servlet.http.HttpServletRequest;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.StringUtils;
 import org.springframework.jdbc.core.JdbcTemplate;
 import org.springframework.stereotype.Service;
 import org.springframework.util.CollectionUtils;
+import java.util.List;
+import java.util.Objects;
+import java.util.Set;
+import java.util.Map;
+import java.util.ArrayList;
+import java.util.stream.Collectors;
 
 @Service
 @Slf4j
@@ -33,7 +33,7 @@ public class AuthServiceImpl implements AuthService {
     private UserService userService;
 
     public AuthServiceImpl(JdbcTemplate jdbcTemplate,
-            UserService userService) {
+                           UserService userService) {
         this.jdbcTemplate = jdbcTemplate;
         this.userService = userService;
     }
@@ -78,12 +78,12 @@ public class AuthServiceImpl implements AuthService {
 
 
     @Override
-    public AuthorizedResourceResp queryAuthorizedResources(QueryAuthResReq req, HttpServletRequest request) {
+    public AuthorizedResourceResp queryAuthorizedResources(QueryAuthResReq req, User user) {
-        Set<String> userOrgIds = userService.getUserAllOrgId(req.getUser());
+        Set<String> userOrgIds = userService.getUserAllOrgId(user.getName());
         if (!CollectionUtils.isEmpty(userOrgIds)) {
             req.setDepartmentIds(new ArrayList<>(userOrgIds));
         }
-        List<AuthGroup> groups = getAuthGroups(req);
+        List<AuthGroup> groups = getAuthGroups(req, user.getName());
         AuthorizedResourceResp resource = new AuthorizedResourceResp();
         Map<String, List<AuthGroup>> authGroupsByModelId = groups.stream()
                 .collect(Collectors.groupingBy(AuthGroup::getModelId));
@@ -130,14 +130,14 @@ public class AuthServiceImpl implements AuthService {
         return resource;
     }
 
-    private List<AuthGroup> getAuthGroups(QueryAuthResReq req) {
+    private List<AuthGroup> getAuthGroups(QueryAuthResReq req, String userName) {
         List<AuthGroup> groups = load().stream()
                 .filter(group -> {
                     if (!Objects.equals(group.getModelId(), req.getModelId())) {
                         return false;
                     }
                     if (!CollectionUtils.isEmpty(group.getAuthorizedUsers()) && group.getAuthorizedUsers()
-                            .contains(req.getUser())) {
+                            .contains(userName)) {
                         return true;
                     }
                     for (String departmentId : req.getDepartmentIds()) {
@@ -148,7 +148,7 @@ public class AuthServiceImpl implements AuthService {
                     }
                     return false;
                 }).collect(Collectors.toList());
-        log.info("user:{} department:{} authGroups:{}", req.getUser(), req.getDepartmentIds(), groups);
+        log.info("user:{} department:{} authGroups:{}", userName, req.getDepartmentIds(), groups);
         return groups;
     }
 

auth/authorization/src/main/java/com/tencent/supersonic/auth/authorization/rest/AuthController.java

@@ -1,11 +1,14 @@
 package com.tencent.supersonic.auth.authorization.rest;
 
-import com.tencent.supersonic.auth.api.authorization.pojo.AuthGroup;
+import com.tencent.supersonic.auth.api.authentication.pojo.User;
+import com.tencent.supersonic.auth.api.authentication.utils.UserHolder;
 import com.tencent.supersonic.auth.api.authorization.request.QueryAuthResReq;
 import com.tencent.supersonic.auth.api.authorization.response.AuthorizedResourceResp;
 import com.tencent.supersonic.auth.api.authorization.service.AuthService;
+import com.tencent.supersonic.auth.api.authorization.pojo.AuthGroup;
 import java.util.List;
 import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
 import lombok.extern.slf4j.Slf4j;
 import org.springframework.web.bind.annotation.GetMapping;
 import org.springframework.web.bind.annotation.PostMapping;
@@ -62,12 +65,13 @@ public class AuthController {
      * 查询有权限访问的受限资源id
      *
      * @param req
-     * @param request
      * @return
      */
     @PostMapping("/queryAuthorizedRes")
     public AuthorizedResourceResp queryAuthorizedResources(@RequestBody QueryAuthResReq req,
-            HttpServletRequest request) {
+                                                           HttpServletRequest request,
-        return authService.queryAuthorizedResources(req, request);
+                                                           HttpServletResponse response) {
+        User user = UserHolder.findUser(request, response);
+        return authService.queryAuthorizedResources(req, user);
     }
 }

chat/api/src/main/java/com/tencent/supersonic/chat/api/component/SemanticCorrector.java

@@ -0,0 +1,9 @@
+package com.tencent.supersonic.chat.api.component;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import net.sf.jsqlparser.JSQLParserException;
+
+public interface SemanticCorrector {
+    CorrectionInfo corrector(CorrectionInfo correctionInfo) throws JSQLParserException;
+
+}

chat/api/src/main/java/com/tencent/supersonic/chat/api/component/SemanticLayer.java

@@ -6,14 +6,15 @@ import com.tencent.supersonic.chat.api.pojo.ModelSchema;
 import com.tencent.supersonic.common.pojo.enums.AuthType;
 import com.tencent.supersonic.semantic.api.model.request.PageDimensionReq;
 import com.tencent.supersonic.semantic.api.model.request.PageMetricReq;
-import com.tencent.supersonic.semantic.api.model.response.DimensionResp;
 import com.tencent.supersonic.semantic.api.model.response.DomainResp;
-import com.tencent.supersonic.semantic.api.model.response.MetricResp;
+import com.tencent.supersonic.semantic.api.model.response.DimensionResp;
 import com.tencent.supersonic.semantic.api.model.response.ModelResp;
+import com.tencent.supersonic.semantic.api.model.response.MetricResp;
 import com.tencent.supersonic.semantic.api.model.response.QueryResultWithSchemaResp;
 import com.tencent.supersonic.semantic.api.query.request.QueryDslReq;
 import com.tencent.supersonic.semantic.api.query.request.QueryMultiStructReq;
 import com.tencent.supersonic.semantic.api.query.request.QueryStructReq;
+
 import java.util.List;
 
 /**
@@ -31,22 +32,13 @@ import java.util.List;
 public interface SemanticLayer {
 
     QueryResultWithSchemaResp queryByStruct(QueryStructReq queryStructReq, User user);
-
     QueryResultWithSchemaResp queryByMultiStruct(QueryMultiStructReq queryMultiStructReq, User user);
-
     QueryResultWithSchemaResp queryByDsl(QueryDslReq queryDslReq, User user);
-
     List<ModelSchema> getModelSchema();
-
     List<ModelSchema> getModelSchema(List<Long> ids);
-
     ModelSchema getModelSchema(Long model, Boolean cacheEnable);
-
     PageInfo<DimensionResp> getDimensionPage(PageDimensionReq pageDimensionCmd);
-
     PageInfo<MetricResp> getMetricPage(PageMetricReq pageMetricCmd);
-
     List<DomainResp> getDomainList(User user);
-
     List<ModelResp> getModelList(AuthType authType, Long domainId, User user);
 }

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/ChatContext.java

@@ -6,6 +6,7 @@ import lombok.Data;
 public class ChatContext {
 
     private Integer chatId;
+    private Integer agentId;
     private String queryText;
     private SemanticParseInfo parseInfo = new SemanticParseInfo();
     private String user;

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/CorrectionInfo.java

@@ -0,0 +1,21 @@
+package com.tencent.supersonic.chat.api.pojo;
+
+import com.tencent.supersonic.chat.api.pojo.request.QueryFilters;
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@AllArgsConstructor
+@NoArgsConstructor
+@Builder
+public class CorrectionInfo {
+
+    private QueryFilters queryFilters;
+
+    private SemanticParseInfo parseInfo;
+
+    private String sql;
+
+}

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/ModelSchema.java

@@ -32,6 +32,7 @@ public class ModelSchema {
                 break;
             case VALUE:
                 element = dimensionValues.stream().filter(e -> e.getId() == elementID).findFirst();
+                break;
             default:
         }
 

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/SchemaElement.java

@@ -1,18 +1,19 @@
 package com.tencent.supersonic.chat.api.pojo;
 
 import com.google.common.base.Objects;
+
 import java.io.Serializable;
 import java.util.List;
-import lombok.Builder;
+
 import lombok.Data;
 import lombok.Getter;
+import lombok.Builder;
 import lombok.NoArgsConstructor;
 
 @Data
 @Getter
 @Builder
 @NoArgsConstructor
-//@AllArgsConstructor
 public class SchemaElement implements Serializable {
 
     private Long model;
@@ -23,11 +24,8 @@ public class SchemaElement implements Serializable {
     private SchemaElementType type;
     private List<String> alias;
 
-//    public SchemaElement() {
-//    }
-
     public SchemaElement(Long model, Long id, String name, String bizName,
-            Long useCnt, SchemaElementType type, List<String> alias) {
+                         Long useCnt, SchemaElementType type, List<String> alias) {
         this.model = model;
         this.id = id;
         this.name = name;

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/SemanticParseInfo.java

@@ -1,23 +1,26 @@
 package com.tencent.supersonic.chat.api.pojo;
 
 
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+import java.util.LinkedHashSet;
+import java.util.ArrayList;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Comparator;
+
 import com.tencent.supersonic.chat.api.pojo.request.QueryFilter;
+import com.tencent.supersonic.chat.api.pojo.response.EntityInfo;
 import com.tencent.supersonic.common.pojo.DateConf;
 import com.tencent.supersonic.common.pojo.Order;
 import com.tencent.supersonic.common.pojo.enums.AggregateTypeEnum;
-import java.util.ArrayList;
-import java.util.Comparator;
-import java.util.HashMap;
-import java.util.LinkedHashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.TreeSet;
 import lombok.Data;
 
 @Data
 public class SemanticParseInfo {
 
+    private Integer id;
     private String queryMode;
     private SchemaElement model;
     private Set<SchemaElement> metrics = new TreeSet<>(new SchemaNameLengthComparator());
@@ -33,7 +36,7 @@ public class SemanticParseInfo {
     private double score;
     private List<SchemaElementMatch> elementMatches = new ArrayList<>();
     private Map<String, Object> properties = new HashMap<>();
-
+    private EntityInfo entityInfo;
     public Long getModelId() {
         return model != null ? model.getId() : 0L;
     }
@@ -43,7 +46,6 @@ public class SemanticParseInfo {
     }
 
     private static class SchemaNameLengthComparator implements Comparator<SchemaElement> {
-
         @Override
         public int compare(SchemaElement o1, SchemaElement o2) {
             int len1 = o1.getName().length();

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/request/ExecuteQueryReq.java

@@ -9,8 +9,11 @@ import lombok.Data;
 public class ExecuteQueryReq {
 
     private User user;
+    private Integer agentId;
     private Integer chatId;
     private String queryText;
+    private Long queryId;
+    private Integer parseId;
     private SemanticParseInfo parseInfo;
     private boolean saveAnswer = true;
 }

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/request/QueryReq.java

@@ -12,4 +12,5 @@ public class QueryReq {
     private User user;
     private QueryFilters queryFilters;
     private boolean saveAnswer = true;
+    private Integer agentId;
 }

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/response/AggregateInfo.java

@@ -6,6 +6,5 @@ import lombok.Data;
 
 @Data
 public class AggregateInfo {
-
     private List<MetricInfo> metricInfos = new ArrayList<>();
 }

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/response/ParseResp.java

@@ -1,12 +1,13 @@
 package com.tencent.supersonic.chat.api.pojo.response;
 
 import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
-import java.util.List;
-import lombok.AllArgsConstructor;
-import lombok.Builder;
 import lombok.Data;
 import lombok.Getter;
+import lombok.Builder;
 import lombok.NoArgsConstructor;
+import lombok.AllArgsConstructor;
+
+import java.util.List;
 
 @Data
 @Getter
@@ -14,9 +15,9 @@ import lombok.NoArgsConstructor;
 @NoArgsConstructor
 @AllArgsConstructor
 public class ParseResp {
-
     private Integer chatId;
     private String queryText;
+    private Long queryId;
     private ParseState state;
     private List<SemanticParseInfo> selectedParses;
     private List<SemanticParseInfo> candidateParses;

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/response/SearchResp.java

@@ -3,6 +3,7 @@ package com.tencent.supersonic.chat.api.pojo.response;
 import java.util.List;
 import lombok.Data;
 
+
 @Data
 public class SearchResp {
 

chat/api/src/main/java/com/tencent/supersonic/chat/api/pojo/response/ShowCaseResp.java

@@ -0,0 +1,18 @@
+package com.tencent.supersonic.chat.api.pojo.response;
+
+
+import lombok.Data;
+
+import java.util.List;
+import java.util.Map;
+
+@Data
+public class ShowCaseResp {
+
+    private Map<Long, List<QueryResp>> showCaseMap;
+
+    private int pageSize;
+
+    private int current;
+
+}

chat/core/pom.xml

@@ -40,12 +40,6 @@
             <scope>compile</scope>
         </dependency>
 
-        <dependency>
-            <groupId>com.github.plexpt</groupId>
-            <artifactId>chatgpt</artifactId>
-            <version>4.1.2</version>
-        </dependency>
-
         <dependency>
             <groupId>org.junit.jupiter</groupId>
             <artifactId>junit-jupiter</artifactId>

chat/core/src/main/java/com/tencent/supersonic/chat/agent/Agent.java

@@ -0,0 +1,49 @@
+package com.tencent.supersonic.chat.agent;
+
+
+import com.alibaba.fastjson.JSONObject;
+import com.google.common.collect.Lists;
+import com.tencent.supersonic.chat.agent.tool.AgentToolType;
+import com.tencent.supersonic.common.pojo.RecordInfo;
+import java.util.Objects;
+import lombok.Data;
+import org.springframework.util.CollectionUtils;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Data
+public class Agent extends RecordInfo {
+
+    private Integer id;
+    private Integer enableSearch;
+    private String name;
+    private String description;
+
+    //0 offline, 1 online
+    private Integer status;
+    private List<String> examples;
+    private String agentConfig;
+    public List<String> getTools(AgentToolType type) {
+        Map map = JSONObject.parseObject(agentConfig, Map.class);
+        if (CollectionUtils.isEmpty(map) || map.get("tools") == null) {
+            return Lists.newArrayList();
+        }
+        List<Map> toolList = (List) map.get("tools");
+        return toolList.stream()
+                .filter(tool -> {
+                            if (Objects.isNull(type)) {
+                                return true;
+                            }
+                            return type.name().equals(tool.get("type"));
+                        }
+                )
+                .map(JSONObject::toJSONString)
+                .collect(Collectors.toList());
+    }
+
+    public boolean enableSearch() {
+        return enableSearch != null && enableSearch == 1;
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/agent/AgentConfig.java

@@ -0,0 +1,18 @@
+package com.tencent.supersonic.chat.agent;
+
+import com.google.common.collect.Lists;
+import com.tencent.supersonic.chat.agent.tool.AgentTool;
+import lombok.AllArgsConstructor;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import java.util.List;
+
+
+@Data
+@NoArgsConstructor
+@AllArgsConstructor
+public class AgentConfig {
+
+    List<AgentTool> tools = Lists.newArrayList();
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/agent/tool/AgentTool.java

@@ -0,0 +1,16 @@
+package com.tencent.supersonic.chat.agent.tool;
+
+import lombok.AllArgsConstructor;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@NoArgsConstructor
+@AllArgsConstructor
+public class AgentTool {
+
+    private String name;
+
+    private AgentToolType type;
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/agent/tool/AgentToolType.java

@@ -0,0 +1,8 @@
+package com.tencent.supersonic.chat.agent.tool;
+
+public enum AgentToolType {
+    RULE,
+    DSL,
+    PLUGIN,
+    INTERPRET
+}

chat/core/src/main/java/com/tencent/supersonic/chat/agent/tool/DslTool.java

@@ -0,0 +1,14 @@
+package com.tencent.supersonic.chat.agent.tool;
+
+import lombok.Data;
+
+import java.util.List;
+
+@Data
+public class DslTool extends AgentTool {
+
+    private List<Long> modelIds;
+
+    private List<String> exampleQuestions;
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/agent/tool/MetricInterpretTool.java

@@ -0,0 +1,16 @@
+package com.tencent.supersonic.chat.agent.tool;
+
+import com.tencent.supersonic.chat.parser.llm.interpret.MetricOption;
+import lombok.Data;
+
+import java.util.List;
+
+
+@Data
+public class MetricInterpretTool extends AgentTool {
+
+    private Long modelId;
+
+    private List<MetricOption> metricOptions;
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/agent/tool/PluginTool.java

@@ -0,0 +1,13 @@
+package com.tencent.supersonic.chat.agent.tool;
+
+
+import lombok.Data;
+
+import java.util.List;
+
+@Data
+public class PluginTool extends AgentTool {
+
+    private List<Long> plugins;
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/agent/tool/RuleQueryTool.java

@@ -0,0 +1,20 @@
+package com.tencent.supersonic.chat.agent.tool;
+
+
+import lombok.Data;
+import org.apache.commons.collections.CollectionUtils;
+
+import java.util.List;
+
+@Data
+public class RuleQueryTool extends AgentTool {
+
+    private List<Long> modelIds;
+
+    private List<String> queryModes;
+
+    public boolean isContainsAllModel() {
+        return CollectionUtils.isNotEmpty(modelIds) && modelIds.contains(-1L);
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/aspect/TimeCostAspect.java

@@ -0,0 +1,99 @@
+/*
+//package com.tencent.supersonic.chat.aspect;
+//
+//import lombok.extern.slf4j.Slf4j;
+//import org.aspectj.lang.JoinPoint;
+//import org.aspectj.lang.ProceedingJoinPoint;
+//import org.aspectj.lang.annotation.*;
+//import org.springframework.stereotype.Component;
+//
+//import java.util.HashMap;
+//import java.util.Map;
+//
+//@Aspect
+//@Component
+//@Slf4j
+//public class TimeCostAspect {
+//
+//    ThreadLocal<Long> startTime = new ThreadLocal<>();
+//
+//    ThreadLocal<Map<String, Long>> map = new ThreadLocal<>();
+//
+//    @Pointcut("execution(public * com.tencent.supersonic.chat.mapper.HanlpDictMapper.*(*))")
+//    //@Pointcut("execution(* public com.tencent.supersonic.chat.parser.*.*(..))")
+//    //@Pointcut("execution(* com.tencent.supersonic.chat.mapper.*Mapper.map(..)) ")
+//    //@Pointcut("execution(* com.tencent.supersonic.chat.mapper.HanlpDictMapper.map(..)) ")
+//    //@Pointcut("execution(* com.tencent.supersonic.chat.parser.rule.QueryModeParser.*(..)) ")
+//    public void point() {
+//    }
+//
+//    @Around("point()")
+//    public void doAround(ProceedingJoinPoint joinPoint) throws Throwable {
+//        long start = System.currentTimeMillis();
+//        try {
+//            log.info("切面开始");
+//            Object result = joinPoint.proceed();
+//            log.info("切面开始");
+//            if (result == null) {
+//                //如果切到了 没有返回类型的void方法，这里直接返回
+//                //return null;
+//            }
+//            long end = System.currentTimeMillis();
+//            log.info("===================");
+//            String targetClassName = joinPoint.getSignature().getDeclaringTypeName();
+//            String MethodName = joinPoint.getSignature().getName();
+//            String typeStr = joinPoint.getSignature().getDeclaringType().toString().split(" ")[0];
+//            log.info("类/接口:" + targetClassName + "(" + typeStr + ")");
+//            log.info("方法:" + MethodName);
+//            Long total = end - start;
+//            log.info("耗时: " + total + " ms!");
+//            map.get().put(targetClassName + "_" + MethodName, total);
+//            //return result;
+//        } catch (Throwable e) {
+//            long end = System.currentTimeMillis();
+//            log.info("====around " + joinPoint + "\tUse time : " + (end - start) + " ms with exception : "
+//                    + e.getMessage());
+//            throw e;
+//        }
+//    }
+//
+////    //对Controller下面的方法执行前进行切入，初始化开始时间
+////    @Before(value = "execution(* com.appleyk.controller.*.*(..))")
+////    public void beforMehhod(JoinPoint jp) {
+////        startTime.set(System.currentTimeMillis());
+////    }
+////
+////    //对Controller下面的方法执行后进行切入，统计方法执行的次数和耗时情况
+////    //注意，这里的执行方法统计的数据不止包含Controller下面的方法，也包括环绕切入的所有方法的统计信息
+////    @AfterReturning(value = "execution(* com.appleyk.controller.*.*(..))")
+////    public void afterMehhod(JoinPoint jp) {
+////        long end = System.currentTimeMillis();
+////        long total =  end - startTime.get();
+////        String methodName = jp.getSignature().getName();
+////        log.info("连接点方法为：" + methodName + ",执行总耗时为：" +total+"ms");
+////
+////        //重新new一个map
+////        Map<String, Long> map = new HashMap<>();
+//////从map2中将最后的 连接点方法给移除了，替换成最终的，避免连接点方法多次进行叠加计算
+////        //由于map2受ThreadLocal的保护，这里不支持remove，因此，需要单开一个map进行数据交接
+////        for(Map.Entry<String, Long> entry:map2.get().entrySet()){
+////            if(entry.getKey().equals(methodName)){
+////                map.put(methodName, total);
+////
+////            }else{
+////                map.put(entry.getKey(), entry.getValue());
+////            }
+////        }
+////
+////        for (Map.Entry<String, Long> entry :map1.get().entrySet()) {
+////            for(Map.Entry<String, Long> entry2 :map.entrySet()){
+////                if(entry.getKey().equals(entry2.getKey())){
+////                    System.err.println(entry.getKey()+",被调用次数："+entry.getValue()+",综合耗时："+entry2.getValue()+"ms");
+////                }
+////            }
+////
+////        }
+////    }
+//
+//}
+*/

chat/core/src/main/java/com/tencent/supersonic/chat/config/AggregatorConfig.java

@@ -3,11 +3,9 @@ package com.tencent.supersonic.chat.config;
 import lombok.Data;
 import org.springframework.beans.factory.annotation.Value;
 import org.springframework.context.annotation.Configuration;
-
-@Configuration
 @Data
+@Configuration
 public class AggregatorConfig {
-
     @Value("${metric.aggregator.ratio.enable:true}")
     private Boolean enableRatio;
 }

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/BaseSemanticCorrector.java

@@ -0,0 +1,33 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.component.SemanticCorrector;
+import com.tencent.supersonic.chat.api.pojo.SchemaElement;
+import com.tencent.supersonic.chat.api.pojo.SemanticSchema;
+import com.tencent.supersonic.common.util.ContextUtils;
+import com.tencent.supersonic.knowledge.service.SchemaService;
+import com.tencent.supersonic.semantic.api.model.enums.TimeDimensionEnum;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public abstract class BaseSemanticCorrector implements SemanticCorrector {
+    public static final String DATE_FIELD = "数据日期";
+    protected Map<String, String> getFieldToBizName(Long modelId) {
+
+        SemanticSchema semanticSchema = ContextUtils.getBean(SchemaService.class).getSemanticSchema();
+
+        List<SchemaElement> dbAllFields = new ArrayList<>();
+        dbAllFields.addAll(semanticSchema.getMetrics());
+        dbAllFields.addAll(semanticSchema.getDimensions());
+
+        Map<String, String> result = dbAllFields.stream()
+                .filter(entry -> entry.getModel().equals(modelId))
+                .collect(Collectors.toMap(SchemaElement::getName, a -> a.getBizName(), (k1, k2) -> k1));
+        result.put(DATE_FIELD, TimeDimensionEnum.DAY.getName());
+        return result;
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/DateFieldCorrector.java

@@ -0,0 +1,27 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.chat.parser.llm.dsl.DSLDateHelper;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserSelectHelper;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserUpdateHelper;
+import java.util.List;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.util.CollectionUtils;
+
+@Slf4j
+public class DateFieldCorrector extends BaseSemanticCorrector {
+
+    @Override
+    public CorrectionInfo corrector(CorrectionInfo correctionInfo) {
+
+        String sql = correctionInfo.getSql();
+        List<String> whereFields = SqlParserSelectHelper.getWhereFields(sql);
+        if (CollectionUtils.isEmpty(whereFields) || !whereFields.contains(DATE_FIELD)) {
+            String currentDate = DSLDateHelper.getCurrentDate(correctionInfo.getParseInfo().getModelId());
+            sql = SqlParserUpdateHelper.addWhere(sql, DATE_FIELD, currentDate);
+        }
+        correctionInfo.setSql(sql);
+        return correctionInfo;
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/FieldCorrector.java

@@ -0,0 +1,17 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserUpdateHelper;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class FieldCorrector extends BaseSemanticCorrector {
+
+    @Override
+    public CorrectionInfo corrector(CorrectionInfo correctionInfo) {
+        String replaceFields = SqlParserUpdateHelper.replaceFields(correctionInfo.getSql(),
+                getFieldToBizName(correctionInfo.getParseInfo().getModelId()));
+        correctionInfo.setSql(replaceFields);
+        return correctionInfo;
+    }
+}

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/FieldValueCorrector.java

@@ -0,0 +1,48 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.chat.parser.llm.dsl.DSLParseResult;
+import com.tencent.supersonic.chat.query.llm.dsl.LLMReq;
+import com.tencent.supersonic.chat.query.llm.dsl.LLMReq.ElementValue;
+import com.tencent.supersonic.common.pojo.Constants;
+import com.tencent.supersonic.common.util.JsonUtil;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserUpdateHelper;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.util.CollectionUtils;
+
+@Slf4j
+public class FieldValueCorrector extends BaseSemanticCorrector {
+
+    @Override
+    public CorrectionInfo corrector(CorrectionInfo correctionInfo) {
+
+        Object context = correctionInfo.getParseInfo().getProperties().get(Constants.CONTEXT);
+        if (Objects.isNull(context)) {
+            return correctionInfo;
+        }
+
+        DSLParseResult dslParseResult = JsonUtil.toObject(JsonUtil.toString(context), DSLParseResult.class);
+        if (Objects.isNull(dslParseResult) || Objects.isNull(dslParseResult.getLlmReq())) {
+            return correctionInfo;
+        }
+        LLMReq llmReq = dslParseResult.getLlmReq();
+        List<ElementValue> linking = llmReq.getLinking();
+        if (CollectionUtils.isEmpty(linking)) {
+            return correctionInfo;
+        }
+
+        Map<String, Set<String>> fieldValueToFieldNames = linking.stream().collect(
+                Collectors.groupingBy(ElementValue::getFieldValue,
+                        Collectors.mapping(ElementValue::getFieldName, Collectors.toSet())));
+
+        String sql = SqlParserUpdateHelper.replaceValueFields(correctionInfo.getSql(), fieldValueToFieldNames);
+        correctionInfo.setSql(sql);
+        return correctionInfo;
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/FunctionCorrector.java

@@ -0,0 +1,16 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserUpdateHelper;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class FunctionCorrector extends BaseSemanticCorrector {
+
+    @Override
+    public CorrectionInfo corrector(CorrectionInfo correctionInfo) {
+        String replaceFunction = SqlParserUpdateHelper.replaceFunction(correctionInfo.getSql());
+        correctionInfo.setSql(replaceFunction);
+        return correctionInfo;
+    }
+}

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/QueryFilterAppend.java

@@ -0,0 +1,48 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.chat.api.pojo.request.QueryFilters;
+import com.tencent.supersonic.common.pojo.Constants;
+import com.tencent.supersonic.common.util.StringUtil;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserUpdateHelper;
+import java.util.Objects;
+import java.util.stream.Collectors;
+import lombok.extern.slf4j.Slf4j;
+import net.sf.jsqlparser.JSQLParserException;
+import net.sf.jsqlparser.expression.Expression;
+import net.sf.jsqlparser.parser.CCJSqlParserUtil;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang3.StringUtils;
+
+@Slf4j
+public class QueryFilterAppend extends BaseSemanticCorrector {
+
+    @Override
+    public CorrectionInfo corrector(CorrectionInfo correctionInfo) throws JSQLParserException {
+        String queryFilter = getQueryFilter(correctionInfo.getQueryFilters());
+        String sql = correctionInfo.getSql();
+
+        if (StringUtils.isNotEmpty(queryFilter)) {
+            log.info("add queryFilter to sql :{}", queryFilter);
+            Expression expression = CCJSqlParserUtil.parseCondExpression(queryFilter);
+            sql = SqlParserUpdateHelper.addWhere(sql, expression);
+        }
+        correctionInfo.setSql(sql);
+        return correctionInfo;
+    }
+
+    private String getQueryFilter(QueryFilters queryFilters) {
+        if (Objects.isNull(queryFilters) || CollectionUtils.isEmpty(queryFilters.getFilters())) {
+            return null;
+        }
+        return queryFilters.getFilters().stream()
+                .map(filter -> {
+                    String bizNameWrap = StringUtil.getSpaceWrap(filter.getBizName());
+                    String operatorWrap = StringUtil.getSpaceWrap(filter.getOperator().getValue());
+                    String valueWrap = StringUtil.getCommaWrap(filter.getValue().toString());
+                    return bizNameWrap + operatorWrap + valueWrap;
+                })
+                .collect(Collectors.joining(Constants.AND_UPPER));
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/SelectFieldAppendCorrector.java

@@ -0,0 +1,39 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserSelectHelper;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserUpdateHelper;
+import com.tencent.supersonic.semantic.api.model.enums.TimeDimensionEnum;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.Set;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.util.CollectionUtils;
+
+@Slf4j
+public class SelectFieldAppendCorrector extends BaseSemanticCorrector {
+
+    @Override
+    public CorrectionInfo corrector(CorrectionInfo correctionInfo) {
+        String sql = correctionInfo.getSql();
+        if (SqlParserSelectHelper.hasAggregateFunction(sql)) {
+            return correctionInfo;
+        }
+        Set<String> selectFields = new HashSet<>(SqlParserSelectHelper.getSelectFields(sql));
+        Set<String> whereFields = new HashSet<>(SqlParserSelectHelper.getWhereFields(sql));
+
+        if (CollectionUtils.isEmpty(selectFields) || CollectionUtils.isEmpty(whereFields)) {
+            return correctionInfo;
+        }
+
+        whereFields.addAll(SqlParserSelectHelper.getOrderByFields(sql));
+        whereFields.removeAll(selectFields);
+        whereFields.remove(TimeDimensionEnum.DAY.getName());
+        whereFields.remove(TimeDimensionEnum.WEEK.getName());
+        whereFields.remove(TimeDimensionEnum.MONTH.getName());
+
+        String replaceFields = SqlParserUpdateHelper.addFieldsToSelect(sql, new ArrayList<>(whereFields));
+        correctionInfo.setSql(replaceFields);
+        return correctionInfo;
+    }
+}

chat/core/src/main/java/com/tencent/supersonic/chat/corrector/TableNameCorrector.java

@@ -0,0 +1,21 @@
+package com.tencent.supersonic.chat.corrector;
+
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserUpdateHelper;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class TableNameCorrector extends BaseSemanticCorrector {
+
+    public static final String TABLE_PREFIX = "t_";
+
+    @Override
+    public CorrectionInfo corrector(CorrectionInfo correctionInfo) {
+        Long modelId = correctionInfo.getParseInfo().getModelId();
+        String sqlOutput = correctionInfo.getSql();
+        String replaceTable = SqlParserUpdateHelper.replaceTable(sqlOutput, TABLE_PREFIX + modelId);
+        correctionInfo.setSql(replaceTable);
+        return correctionInfo;
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/EntityMapper.java

@@ -2,19 +2,19 @@ package com.tencent.supersonic.chat.mapper;
 
 
 import com.tencent.supersonic.chat.api.component.SchemaMapper;
-import com.tencent.supersonic.chat.api.pojo.ModelSchema;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import com.tencent.supersonic.chat.api.pojo.SchemaElement;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
 import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
+import com.tencent.supersonic.chat.api.pojo.SchemaElement;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
+import com.tencent.supersonic.chat.api.pojo.ModelSchema;
 import com.tencent.supersonic.chat.service.SemanticService;
 import com.tencent.supersonic.common.util.ContextUtils;
-import java.util.List;
-import java.util.stream.Collectors;
 import lombok.extern.slf4j.Slf4j;
 import org.springframework.beans.BeanUtils;
 import org.springframework.util.CollectionUtils;
+import java.util.List;
+import java.util.stream.Collectors;
 
 
 @Slf4j
@@ -33,7 +33,7 @@ public class EntityMapper implements SchemaMapper {
                 continue;
             }
             List<SchemaElementMatch> valueSchemaElements = schemaElementMatchList.stream().filter(schemaElementMatch ->
-                            SchemaElementType.VALUE.equals(schemaElementMatch.getElement().getType()))
+                    SchemaElementType.VALUE.equals(schemaElementMatch.getElement().getType()))
                     .collect(Collectors.toList());
             for (SchemaElementMatch schemaElementMatch : valueSchemaElements) {
                 if (!entity.getId().equals(schemaElementMatch.getElement().getId())) {
@@ -51,7 +51,7 @@ public class EntityMapper implements SchemaMapper {
     }
 
     private boolean checkExistSameEntitySchemaElements(SchemaElementMatch valueSchemaElementMatch,
-            List<SchemaElementMatch> schemaElementMatchList) {
+                                                  List<SchemaElementMatch> schemaElementMatchList) {
         List<SchemaElementMatch> entitySchemaElements = schemaElementMatchList.stream().filter(schemaElementMatch ->
                         SchemaElementType.ENTITY.equals(schemaElementMatch.getElement().getType()))
                 .collect(Collectors.toList());

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/FuzzyNameMapper.java

@@ -2,12 +2,12 @@ package com.tencent.supersonic.chat.mapper;
 
 import com.hankcs.hanlp.seg.common.Term;
 import com.tencent.supersonic.chat.api.component.SchemaMapper;
-import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import com.tencent.supersonic.chat.api.pojo.SchemaElement;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
 import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
+import com.tencent.supersonic.chat.api.pojo.SchemaElement;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
 import com.tencent.supersonic.chat.api.pojo.SemanticSchema;
+import com.tencent.supersonic.chat.api.pojo.QueryContext;
 import com.tencent.supersonic.common.util.ContextUtils;
 import com.tencent.supersonic.knowledge.service.SchemaService;
 import com.tencent.supersonic.knowledge.utils.HanlpHelper;
@@ -43,13 +43,13 @@ public class FuzzyNameMapper implements SchemaMapper {
         log.debug("after db mapper,mapInfo:{}", queryContext.getMapInfo());
     }
 
-    private void detectAndAddToSchema(QueryContext queryContext, List<Term> terms, List<SchemaElement> Models,
+    private void detectAndAddToSchema(QueryContext queryContext, List<Term> terms, List<SchemaElement> models,
             SchemaElementType schemaElementType) {
         try {
 
-            Map<String, Set<SchemaElement>> ModelResultSet = getResultSet(queryContext, terms, Models);
+            Map<String, Set<SchemaElement>> modelResultSet = getResultSet(queryContext, terms, models);
 
-            addToSchemaMapInfo(ModelResultSet, queryContext.getMapInfo(), schemaElementType);
+            addToSchemaMapInfo(modelResultSet, queryContext.getMapInfo(), schemaElementType);
 
         } catch (Exception e) {
             log.error("detectAndAddToSchema error", e);
@@ -57,20 +57,21 @@ public class FuzzyNameMapper implements SchemaMapper {
     }
 
     private Map<String, Set<SchemaElement>> getResultSet(QueryContext queryContext, List<Term> terms,
-            List<SchemaElement> Models) {
+            List<SchemaElement> models) {
 
         String queryText = queryContext.getRequest().getQueryText();
 
         MapperHelper mapperHelper = ContextUtils.getBean(MapperHelper.class);
+        Set<Long> modelIds = mapperHelper.getModelIds(queryContext.getRequest());
 
         Double metricDimensionThresholdConfig = getThreshold(queryContext, mapperHelper);
 
-        Map<String, Set<SchemaElement>> nameToItems = getNameToItems(Models);
+        Map<String, Set<SchemaElement>> nameToItems = getNameToItems(models);
 
         Map<Integer, Integer> regOffsetToLength = terms.stream().sorted(Comparator.comparing(Term::length))
                 .collect(Collectors.toMap(Term::getOffset, term -> term.word.length(), (value1, value2) -> value2));
 
-        Map<String, Set<SchemaElement>> ModelResultSet = new HashMap<>();
+        Map<String, Set<SchemaElement>> modelResultSet = new HashMap<>();
         for (Integer startIndex = 0; startIndex <= queryText.length() - 1; ) {
             for (Integer endIndex = startIndex; endIndex <= queryText.length(); ) {
                 endIndex = mapperHelper.getStepIndex(regOffsetToLength, endIndex);
@@ -86,8 +87,12 @@ public class FuzzyNameMapper implements SchemaMapper {
                             || mapperHelper.getSimilarity(detectSegment, name) < metricDimensionThresholdConfig) {
                         continue;
                     }
-                    Set<SchemaElement> preSchemaElements = ModelResultSet.putIfAbsent(detectSegment,
+                    if (!CollectionUtils.isEmpty(modelIds)) {
-                            schemaElements);
+                        schemaElements = schemaElements.stream()
+                                .filter(schemaElement -> modelIds.contains(schemaElement.getModel()))
+                                .collect(Collectors.toSet());
+                    }
+                    Set<SchemaElement> preSchemaElements = modelResultSet.putIfAbsent(detectSegment, schemaElements);
                     if (Objects.nonNull(preSchemaElements)) {
                         preSchemaElements.addAll(schemaElements);
                     }
@@ -95,7 +100,7 @@ public class FuzzyNameMapper implements SchemaMapper {
             }
             startIndex = mapperHelper.getStepIndex(regOffsetToLength, startIndex);
         }
-        return ModelResultSet;
+        return modelResultSet;
     }
 
     private Double getThreshold(QueryContext queryContext, MapperHelper mapperHelper) {
@@ -103,9 +108,9 @@ public class FuzzyNameMapper implements SchemaMapper {
         Double metricDimensionThresholdConfig = mapperHelper.getMetricDimensionThresholdConfig();
         Double metricDimensionMinThresholdConfig = mapperHelper.getMetricDimensionMinThresholdConfig();
 
-        Map<Long, List<SchemaElementMatch>> ModelElementMatches = queryContext.getMapInfo()
+        Map<Long, List<SchemaElementMatch>> modelElementMatches = queryContext.getMapInfo()
                 .getModelElementMatches();
-        boolean existElement = ModelElementMatches.entrySet().stream()
+        boolean existElement = modelElementMatches.entrySet().stream()
                 .anyMatch(entry -> entry.getValue().size() >= 1);
 
         if (!existElement) {
@@ -114,13 +119,13 @@ public class FuzzyNameMapper implements SchemaMapper {
             metricDimensionThresholdConfig = halfThreshold >= metricDimensionMinThresholdConfig ? halfThreshold
                     : metricDimensionMinThresholdConfig;
             log.info("ModelElementMatches:{} , not exist Element metricDimensionThresholdConfig reduce by half:{}",
-                    ModelElementMatches, metricDimensionThresholdConfig);
+                    modelElementMatches, metricDimensionThresholdConfig);
         }
         return metricDimensionThresholdConfig;
     }
 
-    private Map<String, Set<SchemaElement>> getNameToItems(List<SchemaElement> Models) {
+    private Map<String, Set<SchemaElement>> getNameToItems(List<SchemaElement> models) {
-        return Models.stream().collect(
+        return models.stream().collect(
                 Collectors.toMap(SchemaElement::getName, a -> {
                     Set<SchemaElement> result = new HashSet<>();
                     result.add(a);

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/HanlpDictMapper.java

@@ -9,7 +9,7 @@ import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
 import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
 import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
 import com.tencent.supersonic.chat.service.SemanticService;
-import com.tencent.supersonic.chat.utils.NatureHelper;
+import com.tencent.supersonic.knowledge.utils.NatureHelper;
 import com.tencent.supersonic.common.util.ContextUtils;
 import com.tencent.supersonic.knowledge.dictionary.DictWordType;
 import com.tencent.supersonic.knowledge.dictionary.MapResult;
@@ -21,6 +21,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.Objects;
 import java.util.Optional;
+import java.util.Set;
 import java.util.stream.Collectors;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.collections.CollectionUtils;
@@ -37,10 +38,13 @@ public class HanlpDictMapper implements SchemaMapper {
         for (Term term : terms) {
             log.info("word:{},nature:{},frequency:{}", term.word, term.nature.toString(), term.getFrequency());
         }
-        Long modelId = queryContext.getRequest().getModelId();
 
         QueryMatchStrategy matchStrategy = ContextUtils.getBean(QueryMatchStrategy.class);
-        Map<MatchText, List<MapResult>> matchResult = matchStrategy.match(queryText, terms, modelId);
+        MapperHelper mapperHelper = ContextUtils.getBean(MapperHelper.class);
+        Set<Long> detectModelIds = mapperHelper.getModelIds(queryContext.getRequest());
+
+        Map<MatchText, List<MapResult>> matchResult = matchStrategy.match(queryContext.getRequest(), terms,
+                detectModelIds);
 
         List<MapResult> matches = getMatches(matchResult);
 
@@ -51,6 +55,7 @@ public class HanlpDictMapper implements SchemaMapper {
         convertTermsToSchemaMapInfo(matches, queryContext.getMapInfo(), terms);
     }
 
+
     private void convertTermsToSchemaMapInfo(List<MapResult> mapResults, SchemaMapInfo schemaMap, List<Term> terms) {
         if (CollectionUtils.isEmpty(mapResults)) {
             return;

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/MapperHelper.java

@@ -1,10 +1,16 @@
 package com.tencent.supersonic.chat.mapper;
 
 import com.hankcs.hanlp.algorithm.EditDistance;
-import com.tencent.supersonic.chat.utils.NatureHelper;
+import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
+import com.tencent.supersonic.chat.service.AgentService;
+import com.tencent.supersonic.common.util.ContextUtils;
+import com.tencent.supersonic.knowledge.utils.NatureHelper;
+import java.util.HashSet;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
 import lombok.Data;
 import lombok.extern.slf4j.Slf4j;
 import org.springframework.beans.factory.annotation.Value;
@@ -19,7 +25,7 @@ import org.springframework.stereotype.Service;
 @Slf4j
 public class MapperHelper {
 
-    @Value("${one.detection.size:6}")
+    @Value("${one.detection.size:8}")
     private Integer oneDetectionSize;
     @Value("${one.detection.max.size:20}")
     private Integer oneDetectionMaxSize;
@@ -64,7 +70,7 @@ public class MapperHelper {
      */
     public boolean existDimensionValues(List<String> natures) {
         for (String nature : natures) {
-            if (NatureHelper.isDimensionValueClassId(nature)) {
+            if (NatureHelper.isDimensionValueModelId(nature)) {
                 return true;
             }
         }
@@ -84,5 +90,24 @@ public class MapperHelper {
                 detectSegment.length());
     }
 
+    public Set<Long> getModelIds(QueryReq request) {
+
+        Long modelId = request.getModelId();
+
+        AgentService agentService = ContextUtils.getBean(AgentService.class);
+
+        Set<Long> detectModelIds = agentService.getDslToolsModelIds(request.getAgentId(), null);
+        if (Objects.nonNull(detectModelIds)) {
+            detectModelIds = detectModelIds.stream().filter(entry -> entry > 0).collect(Collectors.toSet());
+        }
+        if (Objects.nonNull(modelId) && modelId > 0 && Objects.nonNull(detectModelIds)) {
+            if (detectModelIds.contains(modelId)) {
+                Set<Long> result = new HashSet<>();
+                result.add(modelId);
+                return result;
+            }
+        }
+        return detectModelIds;
+    }
 
 }

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/MatchStrategy.java

@@ -1,15 +1,17 @@
 package com.tencent.supersonic.chat.mapper;
 
 import com.hankcs.hanlp.seg.common.Term;
+import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
 import com.tencent.supersonic.knowledge.dictionary.MapResult;
 import java.util.List;
 import java.util.Map;
+import java.util.Set;
 
 /**
  * match strategy
  */
 public interface MatchStrategy {
 
-    Map<MatchText, List<MapResult>> match(String text, List<Term> terms, Long detectModelId);
+    Map<MatchText, List<MapResult>> match(QueryReq queryReq, List<Term> terms, Set<Long> detectModelId);
 
 }

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/QueryFilterMapper.java

@@ -3,25 +3,25 @@ package com.tencent.supersonic.chat.mapper;
 import com.google.common.collect.Lists;
 import com.tencent.supersonic.chat.api.component.SchemaMapper;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import com.tencent.supersonic.chat.api.pojo.SchemaElement;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
 import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
+import com.tencent.supersonic.chat.api.pojo.SchemaElement;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
 import com.tencent.supersonic.chat.api.pojo.request.QueryFilter;
 import com.tencent.supersonic.chat.api.pojo.request.QueryFilters;
 import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
 import com.tencent.supersonic.common.pojo.Constants;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.util.CollectionUtils;
 import java.util.List;
 import java.util.Map;
 import java.util.stream.Collectors;
-import lombok.extern.slf4j.Slf4j;
-import org.springframework.util.CollectionUtils;
 
 @Slf4j
 public class QueryFilterMapper implements SchemaMapper {
 
-    private Long FREQUENCY = 9999999L;
+    private Long frequency = 9999999L;
-    private double SIMILARITY = 1.0;
+    private double similarity = 1.0;
 
     @Override
     public void map(QueryContext queryContext) {
@@ -49,7 +49,7 @@ public class QueryFilterMapper implements SchemaMapper {
     }
 
     private List<SchemaElementMatch> addValueSchemaElementMatch(List<SchemaElementMatch> candidateElementMatches,
-            QueryFilters queryFilter) {
+                                                           QueryFilters queryFilter) {
         if (queryFilter == null || CollectionUtils.isEmpty(queryFilter.getFilters())) {
             return candidateElementMatches;
         }
@@ -65,9 +65,9 @@ public class QueryFilterMapper implements SchemaMapper {
                     .build();
             SchemaElementMatch schemaElementMatch = SchemaElementMatch.builder()
                     .element(element)
-                    .frequency(FREQUENCY)
+                    .frequency(frequency)
                     .word(String.valueOf(filter.getValue()))
-                    .similarity(SIMILARITY)
+                    .similarity(similarity)
                     .detectWord(Constants.EMPTY)
                     .build();
             candidateElementMatches.add(schemaElementMatch);
@@ -76,7 +76,7 @@ public class QueryFilterMapper implements SchemaMapper {
     }
 
     private boolean checkExistSameValueSchemaElementMatch(QueryFilter queryFilter,
-            List<SchemaElementMatch> schemaElementMatches) {
+                                                          List<SchemaElementMatch> schemaElementMatches) {
         List<SchemaElementMatch> valueSchemaElements = schemaElementMatches.stream().filter(schemaElementMatch ->
                         SchemaElementType.VALUE.equals(schemaElementMatch.getElement().getType()))
                 .collect(Collectors.toList());

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/QueryMatchStrategy.java

@@ -1,8 +1,8 @@
 package com.tencent.supersonic.chat.mapper;
 
 import com.hankcs.hanlp.seg.common.Term;
+import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
 import com.tencent.supersonic.common.pojo.Constants;
-import com.tencent.supersonic.knowledge.dictionary.DictWordType;
 import com.tencent.supersonic.knowledge.dictionary.MapResult;
 import com.tencent.supersonic.knowledge.service.SearchService;
 import java.util.ArrayList;
@@ -32,7 +32,8 @@ public class QueryMatchStrategy implements MatchStrategy {
     private MapperHelper mapperHelper;
 
     @Override
-    public Map<MatchText, List<MapResult>> match(String text, List<Term> terms, Long detectmodelId) {
+    public Map<MatchText, List<MapResult>> match(QueryReq queryReq, List<Term> terms, Set<Long> detectModelIds) {
+        String text = queryReq.getQueryText();
         if (Objects.isNull(terms) || StringUtils.isEmpty(text)) {
             return null;
         }
@@ -43,22 +44,19 @@ public class QueryMatchStrategy implements MatchStrategy {
         List<Integer> offsetList = terms.stream().sorted(Comparator.comparing(Term::getOffset))
                 .map(term -> term.getOffset()).collect(Collectors.toList());
 
-        log.debug("retryCount:{},terms:{},regOffsetToLength:{},offsetList:{},detectmodelId:{}", terms,
+        log.debug("retryCount:{},terms:{},regOffsetToLength:{},offsetList:{},detectModelIds:{}", terms,
-                regOffsetToLength, offsetList, detectmodelId);
+                regOffsetToLength, offsetList, detectModelIds);
 
-        List<MapResult> detects = detect(text, regOffsetToLength, offsetList, detectmodelId);
+        List<MapResult> detects = detect(queryReq, regOffsetToLength, offsetList, detectModelIds);
         Map<MatchText, List<MapResult>> result = new HashMap<>();
 
-        MatchText matchText = MatchText.builder()
+        result.put(MatchText.builder().regText(text).detectSegment(text).build(), detects);
-                .regText(text)
-                .detectSegment(text)
-                .build();
-        result.put(matchText, detects);
         return result;
     }
 
-    private List<MapResult> detect(String text, Map<Integer, Integer> regOffsetToLength, List<Integer> offsetList,
+    private List<MapResult> detect(QueryReq queryReq, Map<Integer, Integer> regOffsetToLength, List<Integer> offsetList,
-            Long detectmodelId) {
+            Set<Long> detectModelIds) {
+        String text = queryReq.getQueryText();
         List<MapResult> results = Lists.newArrayList();
 
         for (Integer index = 0; index <= text.length() - 1; ) {
@@ -69,7 +67,7 @@ public class QueryMatchStrategy implements MatchStrategy {
                 int offset = mapperHelper.getStepOffset(offsetList, index);
                 i = mapperHelper.getStepIndex(regOffsetToLength, i);
                 if (i <= text.length()) {
-                    List<MapResult> mapResults = detectByStep(text, detectmodelId, index, i, offset);
+                    List<MapResult> mapResults = detectByStep(queryReq, detectModelIds, index, i, offset);
                     selectMapResultInOneRound(mapResultRowSet, mapResults);
                 }
             }
@@ -106,16 +104,19 @@ public class QueryMatchStrategy implements MatchStrategy {
         return a.getName() + Constants.UNDERLINE + String.join(Constants.UNDERLINE, a.getNatures());
     }
 
-    private List<MapResult> detectByStep(String text, Long detectmodelId, Integer index, Integer i, int offset) {
+    private List<MapResult> detectByStep(QueryReq queryReq, Set<Long> detectModelIds, Integer index, Integer i,
+            int offset) {
+        String text = queryReq.getQueryText();
+        Integer agentId = queryReq.getAgentId();
         String detectSegment = text.substring(index, i);
-        Integer oneDetectionSize = mapperHelper.getOneDetectionSize();
+
         // step1. pre search
-        LinkedHashSet<MapResult> mapResults = SearchService.prefixSearch(detectSegment,
+        Integer oneDetectionMaxSize = mapperHelper.getOneDetectionMaxSize();
-                        mapperHelper.getOneDetectionMaxSize())
+        LinkedHashSet<MapResult> mapResults = SearchService.prefixSearch(detectSegment, oneDetectionMaxSize, agentId,
-                .stream().collect(Collectors.toCollection(LinkedHashSet::new));
+                detectModelIds).stream().collect(Collectors.toCollection(LinkedHashSet::new));
         // step2. suffix search
-        LinkedHashSet<MapResult> suffixMapResults = SearchService.suffixSearch(detectSegment, oneDetectionSize)
+        LinkedHashSet<MapResult> suffixMapResults = SearchService.suffixSearch(detectSegment, oneDetectionMaxSize,
-                .stream().collect(Collectors.toCollection(LinkedHashSet::new));
+                agentId, detectModelIds).stream().collect(Collectors.toCollection(LinkedHashSet::new));
 
         mapResults.addAll(suffixMapResults);
 
@@ -125,28 +126,15 @@ public class QueryMatchStrategy implements MatchStrategy {
         // step3. merge pre/suffix result
         mapResults = mapResults.stream().sorted((a, b) -> -(b.getName().length() - a.getName().length()))
                 .collect(Collectors.toCollection(LinkedHashSet::new));
-        // step4. filter by classId
+
-        if (Objects.nonNull(detectmodelId) && detectmodelId > 0) {
+        // step4. filter by similarity
-            log.debug("detectmodelId:{}, before parseResults:{}", mapResults);
-            mapResults = mapResults.stream().map(entry -> {
-                List<String> natures = entry.getNatures().stream().filter(
-                        nature -> nature.startsWith(DictWordType.NATURE_SPILT + detectmodelId) || (nature.startsWith(
-                                DictWordType.NATURE_SPILT))
-                ).collect(Collectors.toList());
-                entry.setNatures(natures);
-                return entry;
-            }).collect(Collectors.toCollection(LinkedHashSet::new));
-            log.info("after modelId parseResults:{}", mapResults);
-        }
-        // step5. filter by similarity
         mapResults = mapResults.stream()
                 .filter(term -> mapperHelper.getSimilarity(detectSegment, term.getName())
                         >= mapperHelper.getThresholdMatch(term.getNatures()))
                 .filter(term -> CollectionUtils.isNotEmpty(term.getNatures()))
                 .collect(Collectors.toCollection(LinkedHashSet::new));
 
-        log.debug("metricDimensionThreshold:{},dimensionValueThreshold:{},after isSimilarity  parseResults:{}",
+        log.info("after isSimilarity parseResults:{}", mapResults);
-                mapResults);
 
         mapResults = mapResults.stream().map(parseResult -> {
             parseResult.setOffset(offset);
@@ -154,7 +142,7 @@ public class QueryMatchStrategy implements MatchStrategy {
             return parseResult;
         }).collect(Collectors.toCollection(LinkedHashSet::new));
 
-        // step6. take only one dimension or 10 metric/dimension value per rond.
+        // step5. take only one dimension or 10 metric/dimension value per rond.
         List<MapResult> dimensionMetrics = mapResults.stream()
                 .filter(entry -> mapperHelper.existDimensionValues(entry.getNatures()))
                 .collect(Collectors.toList())
@@ -165,7 +153,7 @@ public class QueryMatchStrategy implements MatchStrategy {
         if (CollectionUtils.isNotEmpty(dimensionMetrics)) {
             return dimensionMetrics;
         } else {
-            return mapResults.stream().limit(oneDetectionSize).collect(Collectors.toList());
+            return mapResults.stream().limit(mapperHelper.getOneDetectionSize()).collect(Collectors.toList());
         }
     }
 }

chat/core/src/main/java/com/tencent/supersonic/chat/mapper/SearchMatchStrategy.java

@@ -2,12 +2,14 @@ package com.tencent.supersonic.chat.mapper;
 
 import com.google.common.collect.Lists;
 import com.hankcs.hanlp.seg.common.Term;
+import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
 import com.tencent.supersonic.knowledge.dictionary.DictWordType;
 import com.tencent.supersonic.knowledge.dictionary.MapResult;
 import com.tencent.supersonic.knowledge.service.SearchService;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
+import java.util.Set;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.stream.Collectors;
 import org.apache.commons.collections.CollectionUtils;
@@ -23,9 +25,8 @@ public class SearchMatchStrategy implements MatchStrategy {
     private static final int SEARCH_SIZE = 3;
 
     @Override
-    public Map<MatchText, List<MapResult>> match(String text, List<Term> originals,
+    public Map<MatchText, List<MapResult>> match(QueryReq queryReq, List<Term> originals, Set<Long> detectModelIds) {
-            Long detectModelId) {
+        String text = queryReq.getQueryText();
-
         Map<Integer, Integer> regOffsetToLength = originals.stream()
                 .filter(entry -> !entry.nature.toString().startsWith(DictWordType.NATURE_SPILT))
                 .collect(Collectors.toMap(Term::getOffset, value -> value.word.length(),
@@ -51,24 +52,16 @@ public class SearchMatchStrategy implements MatchStrategy {
                     String detectSegment = text.substring(detectIndex);
 
                     if (StringUtils.isNotEmpty(detectSegment)) {
-                        List<MapResult> mapResults = SearchService.prefixSearch(detectSegment);
+                        List<MapResult> mapResults = SearchService.prefixSearch(detectSegment,
-                        List<MapResult> suffixMapResults = SearchService.suffixSearch(detectSegment, SEARCH_SIZE);
+                                SearchService.SEARCH_SIZE, queryReq.getAgentId(), detectModelIds);
+                        List<MapResult> suffixMapResults = SearchService.suffixSearch(detectSegment, SEARCH_SIZE,
+                                queryReq.getAgentId(), detectModelIds);
                         mapResults.addAll(suffixMapResults);
                         // remove entity name where search
                         mapResults = mapResults.stream().filter(entry -> {
                             List<String> natures = entry.getNatures().stream()
                                     .filter(nature -> !nature.endsWith(DictWordType.ENTITY.getType()))
-                                    .filter(nature -> {
+                                    .collect(Collectors.toList());
-                                                if (Objects.isNull(detectModelId) || detectModelId <= 0) {
-                                                    return true;
-                                                }
-                                                if (nature.startsWith(DictWordType.NATURE_SPILT + detectModelId)
-                                                        && nature.startsWith(DictWordType.NATURE_SPILT)) {
-                                                    return true;
-                                                }
-                                                return false;
-                                            }
-                                    ).collect(Collectors.toList());
                             if (CollectionUtils.isEmpty(natures)) {
                                 return false;
                             }
@@ -84,4 +77,4 @@ public class SearchMatchStrategy implements MatchStrategy {
         );
         return regTextMap;
     }
-}
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/SatisfactionChecker.java

@@ -4,6 +4,7 @@ package com.tencent.supersonic.chat.parser;
 import com.tencent.supersonic.chat.api.component.SemanticQuery;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
 import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
+import com.tencent.supersonic.chat.query.llm.dsl.DslQuery;
 import lombok.extern.slf4j.Slf4j;
 
 /**
@@ -21,6 +22,9 @@ public class SatisfactionChecker {
     // check all the parse info in candidate
     public static boolean check(QueryContext queryContext) {
         for (SemanticQuery query : queryContext.getCandidateQueries()) {
+            if (query.getQueryMode().equals(DslQuery.QUERY_MODE)) {
+                continue;
+            }
             if (checkThreshold(queryContext.getRequest().getQueryText(), query.getParseInfo())) {
                 return true;
             }
@@ -29,7 +33,7 @@ public class SatisfactionChecker {
     }
 
     private static boolean checkThreshold(String queryText, SemanticParseInfo semanticParseInfo) {
-        int queryTextLength = queryText.length();
+        int queryTextLength = queryText.replaceAll(" ", "").length();
         double degree = semanticParseInfo.getScore() / queryTextLength;
         if (queryTextLength > QUERY_TEXT_LENGTH_THRESHOLD) {
             if (degree < LONG_TEXT_THRESHOLD) {

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/DSLParseResult.java

@@ -1,11 +0,0 @@
-package com.tencent.supersonic.chat.parser.llm;
-
-import com.tencent.supersonic.chat.plugin.PluginParseResult;
-import com.tencent.supersonic.chat.query.dsl.LLMResp;
-import lombok.Data;
-
-@Data
-public class DSLParseResult extends PluginParseResult {
-
-    private LLMResp llmResp;
-}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/LLMDSLParser.java

@@ -1,231 +0,0 @@
-package com.tencent.supersonic.chat.parser.llm;
-
-import com.tencent.supersonic.chat.api.component.SemanticParser;
-import com.tencent.supersonic.chat.api.pojo.ChatContext;
-import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import com.tencent.supersonic.chat.api.pojo.SchemaElement;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
-import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
-import com.tencent.supersonic.chat.api.pojo.SemanticSchema;
-import com.tencent.supersonic.chat.config.LLMConfig;
-import com.tencent.supersonic.chat.parser.SatisfactionChecker;
-import com.tencent.supersonic.chat.parser.function.ModelResolver;
-import com.tencent.supersonic.chat.plugin.Plugin;
-import com.tencent.supersonic.chat.plugin.PluginManager;
-import com.tencent.supersonic.chat.query.QueryManager;
-import com.tencent.supersonic.chat.query.dsl.DSLBuilder;
-import com.tencent.supersonic.chat.query.dsl.DSLQuery;
-import com.tencent.supersonic.chat.query.dsl.LLMReq;
-import com.tencent.supersonic.chat.query.dsl.LLMReq.ElementValue;
-import com.tencent.supersonic.chat.query.dsl.LLMResp;
-import com.tencent.supersonic.chat.query.plugin.PluginSemanticQuery;
-import com.tencent.supersonic.chat.utils.ComponentFactory;
-import com.tencent.supersonic.common.pojo.Constants;
-import com.tencent.supersonic.common.util.ContextUtils;
-import com.tencent.supersonic.common.util.DateUtils;
-import com.tencent.supersonic.common.util.JsonUtil;
-import com.tencent.supersonic.knowledge.service.SchemaService;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Objects;
-import java.util.Set;
-import java.util.stream.Collectors;
-import lombok.extern.slf4j.Slf4j;
-import org.apache.commons.lang3.StringUtils;
-import org.springframework.http.HttpEntity;
-import org.springframework.http.HttpHeaders;
-import org.springframework.http.HttpMethod;
-import org.springframework.http.MediaType;
-import org.springframework.http.ResponseEntity;
-import org.springframework.util.CollectionUtils;
-import org.springframework.web.client.RestTemplate;
-
-@Slf4j
-public class LLMDSLParser implements SemanticParser {
-
-    public static final double FUNCTION_BONUS_THRESHOLD = 201;
-
-    @Override
-    public void parse(QueryContext queryCtx, ChatContext chatCtx) {
-        final LLMConfig llmConfig = ContextUtils.getBean(LLMConfig.class);
-        if (StringUtils.isEmpty(llmConfig.getUrl()) || SatisfactionChecker.check(queryCtx)) {
-            log.info("llmConfig:{}, skip function parser, queryText:{}", llmConfig,
-                    queryCtx.getRequest().getQueryText());
-            return;
-        }
-        List<Plugin> dslPlugins = PluginManager.getPlugins().stream()
-                .filter(plugin -> DSLQuery.QUERY_MODE.equalsIgnoreCase(plugin.getType()))
-                .collect(Collectors.toList());
-
-        if (CollectionUtils.isEmpty(dslPlugins)) {
-            return;
-        }
-        Plugin plugin = dslPlugins.get(0);
-        List<Long> dslModels = plugin.getModelList();
-
-        try {
-            ModelResolver modelResolver = ComponentFactory.getModelResolver();
-            Long modelId = modelResolver.resolve(queryCtx, chatCtx, dslModels);
-            log.info("resolve modelId:{},dslModels:{}", modelId, dslModels);
-
-            if (Objects.isNull(modelId)) {
-                return;
-            }
-
-            LLMResp llmResp = requestLLM(queryCtx, modelId);
-            if (Objects.isNull(llmResp)) {
-                return;
-            }
-
-            PluginSemanticQuery semanticQuery = QueryManager.createPluginQuery(DSLQuery.QUERY_MODE);
-
-            SemanticParseInfo parseInfo = semanticQuery.getParseInfo();
-            if (Objects.nonNull(modelId) && modelId > 0) {
-                parseInfo.getElementMatches().addAll(queryCtx.getMapInfo().getMatchedElements(modelId));
-            }
-            DSLParseResult dslParseResult = new DSLParseResult();
-            dslParseResult.setRequest(queryCtx.getRequest());
-            dslParseResult.setLlmResp(llmResp);
-            dslParseResult.setPlugin(plugin);
-
-            Map<String, Object> properties = new HashMap<>();
-            properties.put(Constants.CONTEXT, dslParseResult);
-            parseInfo.setProperties(properties);
-            parseInfo.setScore(FUNCTION_BONUS_THRESHOLD);
-            parseInfo.setQueryMode(semanticQuery.getQueryMode());
-            SchemaElement Model = new SchemaElement();
-            Model.setModel(modelId);
-            Model.setId(modelId);
-            parseInfo.setModel(Model);
-            queryCtx.getCandidateQueries().add(semanticQuery);
-        } catch (Exception e) {
-            log.error("LLMDSLParser error", e);
-        }
-    }
-
-
-    private LLMResp requestLLM(QueryContext queryCtx, Long modelId) {
-        long startTime = System.currentTimeMillis();
-        String queryText = queryCtx.getRequest().getQueryText();
-        final LLMConfig llmConfig = ContextUtils.getBean(LLMConfig.class);
-
-        if (StringUtils.isEmpty(llmConfig.getUrl())) {
-            log.warn("llmConfig url is null, skip llm parser");
-            return null;
-        }
-
-        SemanticSchema semanticSchema = ContextUtils.getBean(SchemaService.class).getSemanticSchema();
-        Map<Long, String> modelIdToName = semanticSchema.getModelIdToName();
-
-        LLMReq llmReq = new LLMReq();
-        llmReq.setQueryText(queryText);
-        LLMReq.LLMSchema llmSchema = new LLMReq.LLMSchema();
-        llmSchema.setModelName(modelIdToName.get(modelId));
-        llmSchema.setDomainName(modelIdToName.get(modelId));
-        List<String> fieldNameList = getFieldNameList(queryCtx, modelId, semanticSchema);
-        fieldNameList.add(DSLBuilder.DATA_Field);
-        llmSchema.setFieldNameList(fieldNameList);
-        llmReq.setSchema(llmSchema);
-        List<ElementValue> linking = new ArrayList<>();
-        linking.addAll(getValueList(queryCtx, modelId, semanticSchema));
-        llmReq.setLinking(linking);
-        String currentDate = getCurrentDate(modelId);
-        llmReq.setCurrentDate(currentDate);
-
-        log.info("requestLLM request, modelId:{},llmReq:{}", modelId, llmReq);
-        String questUrl = llmConfig.getUrl() + llmConfig.getQueryToSqlPath();
-
-        RestTemplate restTemplate = ContextUtils.getBean(RestTemplate.class);
-
-        try {
-            HttpHeaders headers = new HttpHeaders();
-            headers.setContentType(MediaType.APPLICATION_JSON);
-            HttpEntity<String> entity = new HttpEntity<>(JsonUtil.toString(llmReq), headers);
-            ResponseEntity<LLMResp> responseEntity = restTemplate.exchange(questUrl, HttpMethod.POST, entity,
-                    LLMResp.class);
-
-            log.info("requestLLM response,cost:{}, questUrl:{} \n entity:{} \n body:{}",
-                    System.currentTimeMillis() - startTime, questUrl, entity, responseEntity.getBody());
-            return responseEntity.getBody();
-        } catch (Exception e) {
-            log.error("requestLLM error", e);
-        }
-        return null;
-    }
-
-
-    private String getCurrentDate(Long modelId) {
-        return DateUtils.getBeforeDate(4);
-//        ChatConfigFilter filter = new ChatConfigFilter();
-//        filter.setModelId(modelId);
-//
-//        List<ChatConfigResp> configResps = ContextUtils.getBean(ConfigService.class).search(filter, null);
-//        if (CollectionUtils.isEmpty(configResps)) {
-//            return
-//        }
-//        ChatConfigResp chatConfigResp = configResps.get(0);
-//        chatConfigResp.getChatDetailConfig().getChatDefaultConfig().get
-
-    }
-
-    private List<ElementValue> getValueList(QueryContext queryCtx, Long modelId, SemanticSchema semanticSchema) {
-        Map<Long, String> itemIdToName = getItemIdToName(modelId, semanticSchema);
-
-        List<SchemaElementMatch> matchedElements = queryCtx.getMapInfo().getMatchedElements(modelId);
-        if (CollectionUtils.isEmpty(matchedElements)) {
-            return new ArrayList<>();
-        }
-        Set<ElementValue> valueMatches = matchedElements.stream()
-                .filter(schemaElementMatch -> {
-                    SchemaElementType type = schemaElementMatch.getElement().getType();
-                    return SchemaElementType.VALUE.equals(type) || SchemaElementType.ID.equals(type);
-                })
-                .map(elementMatch ->
-                        {
-                            ElementValue elementValue = new ElementValue();
-                            elementValue.setFieldName(itemIdToName.get(elementMatch.getElement().getId()));
-                            elementValue.setFieldValue(elementMatch.getWord());
-                            return elementValue;
-                        }
-                )
-                .collect(Collectors.toSet());
-        return new ArrayList<>(valueMatches);
-    }
-
-    private List<String> getFieldNameList(QueryContext queryCtx, Long modelId, SemanticSchema semanticSchema) {
-        Map<Long, String> itemIdToName = getItemIdToName(modelId, semanticSchema);
-
-        List<SchemaElementMatch> matchedElements = queryCtx.getMapInfo().getMatchedElements(modelId);
-        if (CollectionUtils.isEmpty(matchedElements)) {
-            return new ArrayList<>();
-        }
-        Set<String> fieldNameList = matchedElements.stream()
-                .filter(schemaElementMatch -> {
-                    SchemaElementType elementType = schemaElementMatch.getElement().getType();
-                    return SchemaElementType.METRIC.equals(elementType) ||
-                            SchemaElementType.DIMENSION.equals(elementType) ||
-                            SchemaElementType.VALUE.equals(elementType);
-                })
-                .map(schemaElementMatch -> {
-                    SchemaElementType elementType = schemaElementMatch.getElement().getType();
-
-                    if (!SchemaElementType.VALUE.equals(elementType)) {
-                        return schemaElementMatch.getWord();
-                    }
-                    return itemIdToName.get(schemaElementMatch.getElement().getId());
-                })
-                .filter(name -> StringUtils.isNotEmpty(name) && !name.contains("%"))
-                .collect(Collectors.toSet());
-        return new ArrayList<>(fieldNameList);
-    }
-
-    private Map<Long, String> getItemIdToName(Long modelId, SemanticSchema semanticSchema) {
-        return semanticSchema.getDimensions().stream()
-                .filter(entry -> modelId.equals(entry.getModel()))
-                .collect(Collectors.toMap(SchemaElement::getId, SchemaElement::getName, (value1, value2) -> value2));
-    }
-
-}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/dsl/DSLDateHelper.java

@@ -0,0 +1,11 @@
+package com.tencent.supersonic.chat.parser.llm.dsl;
+
+import com.tencent.supersonic.common.util.DateUtils;
+
+public class DSLDateHelper {
+
+    public static String getCurrentDate(Long modelId) {
+        return DateUtils.getBeforeDate(4);
+
+    }
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/dsl/DSLParseResult.java

@@ -0,0 +1,25 @@
+package com.tencent.supersonic.chat.parser.llm.dsl;
+
+import com.tencent.supersonic.chat.agent.tool.DslTool;
+import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
+import com.tencent.supersonic.chat.query.llm.dsl.LLMReq;
+import com.tencent.supersonic.chat.query.llm.dsl.LLMResp;
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@Builder
+@AllArgsConstructor
+@NoArgsConstructor
+public class DSLParseResult {
+
+    private LLMReq llmReq;
+
+    private LLMResp llmResp;
+
+    private QueryReq request;
+
+    private DslTool dslTool;
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/dsl/LLMDslParser.java

@@ -0,0 +1,380 @@
+package com.tencent.supersonic.chat.parser.llm.dsl;
+
+import com.google.common.collect.Lists;
+import com.tencent.supersonic.chat.agent.tool.AgentToolType;
+import com.tencent.supersonic.chat.agent.tool.DslTool;
+import com.tencent.supersonic.chat.api.component.SemanticCorrector;
+import com.tencent.supersonic.chat.api.component.SemanticParser;
+import com.tencent.supersonic.chat.api.pojo.ChatContext;
+import com.tencent.supersonic.chat.api.pojo.CorrectionInfo;
+import com.tencent.supersonic.chat.api.pojo.QueryContext;
+import com.tencent.supersonic.chat.api.pojo.SchemaElement;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
+import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
+import com.tencent.supersonic.chat.api.pojo.SemanticSchema;
+import com.tencent.supersonic.chat.api.pojo.request.QueryFilter;
+import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
+import com.tencent.supersonic.chat.config.LLMConfig;
+import com.tencent.supersonic.chat.corrector.BaseSemanticCorrector;
+import com.tencent.supersonic.chat.parser.SatisfactionChecker;
+import com.tencent.supersonic.chat.parser.plugin.function.ModelResolver;
+import com.tencent.supersonic.chat.query.QueryManager;
+import com.tencent.supersonic.chat.query.llm.dsl.DslQuery;
+import com.tencent.supersonic.chat.query.llm.dsl.LLMReq;
+import com.tencent.supersonic.chat.query.llm.dsl.LLMReq.ElementValue;
+import com.tencent.supersonic.chat.query.llm.dsl.LLMResp;
+import com.tencent.supersonic.chat.query.plugin.PluginSemanticQuery;
+import com.tencent.supersonic.chat.service.AgentService;
+import com.tencent.supersonic.chat.utils.ComponentFactory;
+import com.tencent.supersonic.common.pojo.Constants;
+import com.tencent.supersonic.common.pojo.DateConf;
+import com.tencent.supersonic.common.pojo.DateConf.DateMode;
+import com.tencent.supersonic.common.util.ContextUtils;
+import com.tencent.supersonic.common.util.JsonUtil;
+import com.tencent.supersonic.common.util.jsqlparser.FilterExpression;
+import com.tencent.supersonic.common.util.jsqlparser.SqlParserSelectHelper;
+import com.tencent.supersonic.knowledge.service.SchemaService;
+import com.tencent.supersonic.semantic.api.model.enums.TimeDimensionEnum;
+import com.tencent.supersonic.semantic.api.query.enums.FilterOperatorEnum;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.commons.lang3.StringUtils;
+import org.springframework.http.HttpEntity;
+import org.springframework.http.HttpHeaders;
+import org.springframework.http.HttpMethod;
+import org.springframework.http.MediaType;
+import org.springframework.http.ResponseEntity;
+import org.springframework.util.CollectionUtils;
+import org.springframework.web.client.RestTemplate;
+
+@Slf4j
+public class LLMDslParser implements SemanticParser {
+
+    public static final double function_bonus_threshold = 201;
+
+    @Override
+    public void parse(QueryContext queryCtx, ChatContext chatCtx) {
+        QueryReq request = queryCtx.getRequest();
+        LLMConfig llmConfig = ContextUtils.getBean(LLMConfig.class);
+        if (StringUtils.isEmpty(llmConfig.getUrl()) || SatisfactionChecker.check(queryCtx)) {
+            log.info("llmConfig:{}, skip function parser, queryText:{}", llmConfig, request.getQueryText());
+            return;
+        }
+        try {
+            Long modelId = getModelId(queryCtx, chatCtx, request.getAgentId());
+            if (Objects.isNull(modelId) || modelId <= 0) {
+                return;
+            }
+
+            DslTool dslTool = getDslTool(request, modelId);
+            if (Objects.isNull(dslTool)) {
+                log.info("no dsl tool in this agent, skip dsl parser");
+                return;
+            }
+
+            LLMReq llmReq = getLlmReq(queryCtx, modelId);
+            LLMResp llmResp = requestLLM(llmReq, modelId, llmConfig);
+
+            if (Objects.isNull(llmResp)) {
+                return;
+            }
+            DSLParseResult dslParseResult = DSLParseResult.builder().request(request).dslTool(dslTool).llmReq(llmReq)
+                    .llmResp(llmResp).build();
+
+            SemanticParseInfo parseInfo = getParseInfo(queryCtx, modelId, dslTool, dslParseResult);
+
+            String correctorSql = getCorrectorSql(queryCtx, parseInfo, llmResp.getSqlOutput());
+
+            llmResp.setCorrectorSql(correctorSql);
+
+            setFilter(correctorSql, modelId, parseInfo);
+
+        } catch (Exception e) {
+            log.error("LLMDSLParser error", e);
+        }
+    }
+
+    public void setFilter(String correctorSql, Long modelId, SemanticParseInfo parseInfo) {
+
+        List<FilterExpression> expressions = SqlParserSelectHelper.getFilterExpression(correctorSql);
+        if (CollectionUtils.isEmpty(expressions)) {
+            return;
+        }
+        //set dataInfo
+        try {
+            DateConf dateInfo = getDateInfo(expressions);
+            parseInfo.setDateInfo(dateInfo);
+        } catch (Exception e) {
+            log.error("set dateInfo error :", e);
+        }
+
+        //set filter
+        try {
+            Map<String, SchemaElement> bizNameToElement = getBizNameToElement(modelId);
+            List<QueryFilter> result = getDimensionFilter(bizNameToElement, expressions);
+            parseInfo.getDimensionFilters().addAll(result);
+        } catch (Exception e) {
+            log.error("set dimensionFilter error :", e);
+        }
+    }
+
+    private List<QueryFilter> getDimensionFilter(Map<String, SchemaElement> bizNameToElement,
+            List<FilterExpression> filterExpressions) {
+        List<QueryFilter> result = Lists.newArrayList();
+        for (FilterExpression expression : filterExpressions) {
+            QueryFilter dimensionFilter = new QueryFilter();
+            dimensionFilter.setValue(expression.getFieldValue());
+            String bizName = expression.getFieldName();
+            SchemaElement schemaElement = bizNameToElement.get(bizName);
+            if (Objects.isNull(schemaElement)) {
+                continue;
+            }
+            String fieldName = schemaElement.getName();
+            dimensionFilter.setName(fieldName);
+            dimensionFilter.setBizName(bizName);
+            dimensionFilter.setElementID(schemaElement.getId());
+
+            FilterOperatorEnum operatorEnum = FilterOperatorEnum.getSqlOperator(expression.getOperator());
+            dimensionFilter.setOperator(operatorEnum);
+            result.add(dimensionFilter);
+        }
+        return result;
+    }
+
+    private DateConf getDateInfo(List<FilterExpression> filterExpressions) {
+        List<FilterExpression> dateExpressions = filterExpressions.stream()
+                .filter(expression -> {
+                    List<String> nameList = TimeDimensionEnum.getNameList();
+                    if (StringUtils.isEmpty(expression.getFieldName())) {
+                        return false;
+                    }
+                    return nameList.contains(expression.getFieldName().toLowerCase());
+                }).collect(Collectors.toList());
+        if (CollectionUtils.isEmpty(dateExpressions)) {
+            return new DateConf();
+        }
+        DateConf dateInfo = new DateConf();
+        dateInfo.setDateMode(DateMode.BETWEEN);
+        FilterExpression firstExpression = dateExpressions.get(0);
+
+        FilterOperatorEnum firstOperator = FilterOperatorEnum.getSqlOperator(firstExpression.getOperator());
+        if (FilterOperatorEnum.EQUALS.equals(firstOperator) && Objects.nonNull(firstExpression.getFieldValue())) {
+            dateInfo.setStartDate(firstExpression.getFieldValue().toString());
+            dateInfo.setEndDate(firstExpression.getFieldValue().toString());
+            dateInfo.setDateMode(DateMode.BETWEEN);
+            return dateInfo;
+        }
+        if (containOperators(firstExpression, firstOperator, FilterOperatorEnum.GREATER_THAN,
+                FilterOperatorEnum.GREATER_THAN_EQUALS)) {
+            dateInfo.setStartDate(firstExpression.getFieldValue().toString());
+            if (hasSecondDate(dateExpressions)) {
+                dateInfo.setEndDate(dateExpressions.get(1).getFieldValue().toString());
+            }
+        }
+        if (containOperators(firstExpression, firstOperator, FilterOperatorEnum.MINOR_THAN,
+                FilterOperatorEnum.MINOR_THAN_EQUALS)) {
+            dateInfo.setEndDate(firstExpression.getFieldValue().toString());
+            if (hasSecondDate(dateExpressions)) {
+                dateInfo.setStartDate(dateExpressions.get(1).getFieldValue().toString());
+            }
+        }
+        return dateInfo;
+    }
+
+    private boolean containOperators(FilterExpression expression, FilterOperatorEnum firstOperator,
+            FilterOperatorEnum... operatorEnums) {
+        return (Arrays.asList(operatorEnums).contains(firstOperator) && Objects.nonNull(expression.getFieldValue()));
+    }
+
+    private boolean hasSecondDate(List<FilterExpression> dateExpressions) {
+        return dateExpressions.size() > 1 && Objects.nonNull(dateExpressions.get(1).getFieldValue());
+    }
+
+    private String getCorrectorSql(QueryContext queryCtx, SemanticParseInfo parseInfo, String sql) {
+
+        CorrectionInfo correctionInfo = CorrectionInfo.builder()
+                .queryFilters(queryCtx.getRequest().getQueryFilters()).sql(sql)
+                .parseInfo(parseInfo).build();
+
+        List<SemanticCorrector> dslCorrections = ComponentFactory.getSqlCorrections();
+
+        dslCorrections.forEach(dslCorrection -> {
+            try {
+                dslCorrection.corrector(correctionInfo);
+                log.info("sqlCorrection:{} sql:{}", dslCorrection.getClass().getSimpleName(),
+                        correctionInfo.getSql());
+            } catch (Exception e) {
+                log.error("sqlCorrection:{} execute error,correctionInfo:{}", dslCorrection, correctionInfo, e);
+            }
+        });
+        return correctionInfo.getSql();
+    }
+
+    private SemanticParseInfo getParseInfo(QueryContext queryCtx, Long modelId, DslTool dslTool,
+            DSLParseResult dslParseResult) {
+        PluginSemanticQuery semanticQuery = QueryManager.createPluginQuery(DslQuery.QUERY_MODE);
+        SemanticParseInfo parseInfo = semanticQuery.getParseInfo();
+        parseInfo.getElementMatches().addAll(queryCtx.getMapInfo().getMatchedElements(modelId));
+
+        Map<String, Object> properties = new HashMap<>();
+        properties.put(Constants.CONTEXT, dslParseResult);
+        properties.put("type", "internal");
+        properties.put("name", dslTool.getName());
+
+        parseInfo.setProperties(properties);
+        parseInfo.setScore(function_bonus_threshold);
+        parseInfo.setQueryMode(semanticQuery.getQueryMode());
+
+        SemanticSchema semanticSchema = ContextUtils.getBean(SchemaService.class).getSemanticSchema();
+        Map<Long, String> modelIdToName = semanticSchema.getModelIdToName();
+
+        SchemaElement model = new SchemaElement();
+        model.setModel(modelId);
+        model.setId(modelId);
+        model.setName(modelIdToName.get(modelId));
+        parseInfo.setModel(model);
+        queryCtx.getCandidateQueries().add(semanticQuery);
+        return parseInfo;
+    }
+
+    private DslTool getDslTool(QueryReq request, Long modelId) {
+        AgentService agentService = ContextUtils.getBean(AgentService.class);
+        List<DslTool> dslTools = agentService.getDslTools(request.getAgentId(), AgentToolType.DSL);
+        Optional<DslTool> dslToolOptional = dslTools.stream().filter(tool -> tool.getModelIds().contains(modelId))
+                .findFirst();
+        return dslToolOptional.orElse(null);
+    }
+
+    private Long getModelId(QueryContext queryCtx, ChatContext chatCtx, Integer agentId) {
+        AgentService agentService = ContextUtils.getBean(AgentService.class);
+        Set<Long> distinctModelIds = agentService.getDslToolsModelIds(agentId, AgentToolType.DSL);
+        ModelResolver modelResolver = ComponentFactory.getModelResolver();
+        Long modelId = modelResolver.resolve(queryCtx, chatCtx, distinctModelIds);
+        log.info("resolve modelId:{},dslModels:{}", modelId, distinctModelIds);
+        return modelId;
+    }
+
+    private LLMResp requestLLM(LLMReq llmReq, Long modelId, LLMConfig llmConfig) {
+        String questUrl = llmConfig.getUrl() + llmConfig.getQueryToSqlPath();
+        long startTime = System.currentTimeMillis();
+        log.info("requestLLM request, modelId:{},llmReq:{}", modelId, llmReq);
+        RestTemplate restTemplate = ContextUtils.getBean(RestTemplate.class);
+        try {
+            HttpHeaders headers = new HttpHeaders();
+            headers.setContentType(MediaType.APPLICATION_JSON);
+            HttpEntity<String> entity = new HttpEntity<>(JsonUtil.toString(llmReq), headers);
+            ResponseEntity<LLMResp> responseEntity = restTemplate.exchange(questUrl, HttpMethod.POST, entity,
+                    LLMResp.class);
+
+            log.info("requestLLM response,cost:{}, questUrl:{} \n entity:{} \n body:{}",
+                    System.currentTimeMillis() - startTime, questUrl, entity, responseEntity.getBody());
+            return responseEntity.getBody();
+        } catch (Exception e) {
+            log.error("requestLLM error", e);
+        }
+        return null;
+    }
+
+    private LLMReq getLlmReq(QueryContext queryCtx, Long modelId) {
+        SemanticSchema semanticSchema = ContextUtils.getBean(SchemaService.class).getSemanticSchema();
+        Map<Long, String> modelIdToName = semanticSchema.getModelIdToName();
+        String queryText = queryCtx.getRequest().getQueryText();
+        LLMReq llmReq = new LLMReq();
+        llmReq.setQueryText(queryText);
+        LLMReq.LLMSchema llmSchema = new LLMReq.LLMSchema();
+        llmSchema.setModelName(modelIdToName.get(modelId));
+        llmSchema.setDomainName(modelIdToName.get(modelId));
+        List<String> fieldNameList = getFieldNameList(queryCtx, modelId, semanticSchema);
+        fieldNameList.add(BaseSemanticCorrector.DATE_FIELD);
+        llmSchema.setFieldNameList(fieldNameList);
+        llmReq.setSchema(llmSchema);
+        List<ElementValue> linking = new ArrayList<>();
+        linking.addAll(getValueList(queryCtx, modelId, semanticSchema));
+        llmReq.setLinking(linking);
+        String currentDate = DSLDateHelper.getCurrentDate(modelId);
+        llmReq.setCurrentDate(currentDate);
+        return llmReq;
+    }
+
+    private List<ElementValue> getValueList(QueryContext queryCtx, Long modelId, SemanticSchema semanticSchema) {
+        Map<Long, String> itemIdToName = getItemIdToName(modelId, semanticSchema);
+
+        List<SchemaElementMatch> matchedElements = queryCtx.getMapInfo().getMatchedElements(modelId);
+        if (CollectionUtils.isEmpty(matchedElements)) {
+            return new ArrayList<>();
+        }
+        Set<ElementValue> valueMatches = matchedElements
+                .stream()
+                .filter(elementMatch -> !elementMatch.isInherited())
+                .filter(schemaElementMatch -> {
+                    SchemaElementType type = schemaElementMatch.getElement().getType();
+                    return SchemaElementType.VALUE.equals(type) || SchemaElementType.ID.equals(type);
+                })
+                .map(elementMatch -> {
+                    ElementValue elementValue = new ElementValue();
+                    elementValue.setFieldName(itemIdToName.get(elementMatch.getElement().getId()));
+                    elementValue.setFieldValue(elementMatch.getWord());
+                    return elementValue;
+                }).collect(Collectors.toSet());
+        return new ArrayList<>(valueMatches);
+    }
+
+
+    protected Map<String, SchemaElement> getBizNameToElement(Long modelId) {
+        SemanticSchema semanticSchema = ContextUtils.getBean(SchemaService.class).getSemanticSchema();
+        List<SchemaElement> dimensions = semanticSchema.getDimensions();
+        List<SchemaElement> metrics = semanticSchema.getMetrics();
+
+        List<SchemaElement> allElements = Lists.newArrayList();
+        allElements.addAll(dimensions);
+        allElements.addAll(metrics);
+        return allElements.stream()
+                .filter(schemaElement -> schemaElement.getModel().equals(modelId))
+                .collect(Collectors.toMap(SchemaElement::getBizName, Function.identity(), (value1, value2) -> value2));
+    }
+
+
+    private List<String> getFieldNameList(QueryContext queryCtx, Long modelId, SemanticSchema semanticSchema) {
+        Map<Long, String> itemIdToName = getItemIdToName(modelId, semanticSchema);
+
+        List<SchemaElementMatch> matchedElements = queryCtx.getMapInfo().getMatchedElements(modelId);
+        if (CollectionUtils.isEmpty(matchedElements)) {
+            return new ArrayList<>();
+        }
+        Set<String> fieldNameList = matchedElements.stream()
+                .filter(schemaElementMatch -> {
+                    SchemaElementType elementType = schemaElementMatch.getElement().getType();
+                    return SchemaElementType.METRIC.equals(elementType)
+                            || SchemaElementType.DIMENSION.equals(elementType)
+                            || SchemaElementType.VALUE.equals(elementType);
+                })
+                .map(schemaElementMatch -> {
+                    SchemaElementType elementType = schemaElementMatch.getElement().getType();
+
+                    if (!SchemaElementType.VALUE.equals(elementType)) {
+                        return schemaElementMatch.getWord();
+                    }
+                    return itemIdToName.get(schemaElementMatch.getElement().getId());
+                })
+                .filter(name -> StringUtils.isNotEmpty(name) && !name.contains("%"))
+                .collect(Collectors.toSet());
+        return new ArrayList<>(fieldNameList);
+    }
+
+    private Map<Long, String> getItemIdToName(Long modelId, SemanticSchema semanticSchema) {
+        return semanticSchema.getDimensions().stream()
+                .filter(entry -> modelId.equals(entry.getModel()))
+                .collect(Collectors.toMap(SchemaElement::getId, SchemaElement::getName, (value1, value2) -> value2));
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/interpret/MetricInterpretParser.java

@@ -0,0 +1,157 @@
+package com.tencent.supersonic.chat.parser.llm.interpret;
+
+import com.alibaba.fastjson.JSONObject;
+import com.google.common.collect.Sets;
+import com.tencent.supersonic.chat.agent.Agent;
+import com.tencent.supersonic.chat.agent.tool.AgentToolType;
+import com.tencent.supersonic.chat.agent.tool.MetricInterpretTool;
+import com.tencent.supersonic.chat.api.component.SemanticLayer;
+import com.tencent.supersonic.chat.api.component.SemanticParser;
+import com.tencent.supersonic.chat.api.pojo.QueryContext;
+import com.tencent.supersonic.chat.api.pojo.ChatContext;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
+import com.tencent.supersonic.chat.api.pojo.SchemaElement;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
+import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
+import com.tencent.supersonic.chat.api.pojo.ModelSchema;
+import com.tencent.supersonic.chat.api.pojo.request.QueryFilter;
+import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
+import com.tencent.supersonic.chat.parser.SatisfactionChecker;
+import com.tencent.supersonic.chat.query.metricinterpret.MetricInterpretQuery;
+import com.tencent.supersonic.chat.query.QueryManager;
+import com.tencent.supersonic.chat.query.plugin.PluginSemanticQuery;
+import com.tencent.supersonic.chat.service.AgentService;
+import com.tencent.supersonic.chat.utils.ComponentFactory;
+import com.tencent.supersonic.common.pojo.DateConf;
+import com.tencent.supersonic.common.util.ContextUtils;
+import com.tencent.supersonic.semantic.api.model.enums.TimeDimensionEnum;
+import com.tencent.supersonic.semantic.api.query.enums.FilterOperatorEnum;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.util.CollectionUtils;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.HashMap;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class MetricInterpretParser implements SemanticParser {
+
+    @Override
+    public void parse(QueryContext queryContext, ChatContext chatContext) {
+        if (SatisfactionChecker.check(queryContext)) {
+            log.info("skip MetricInterpretParser");
+            return;
+        }
+        Map<Long, MetricInterpretTool> metricInterpretToolMap =
+                getMetricInterpretTools(queryContext.getRequest().getAgentId());
+        log.info("metric interpret tool : {}", metricInterpretToolMap);
+        if (CollectionUtils.isEmpty(metricInterpretToolMap)) {
+            return;
+        }
+        Map<Long, List<SchemaElementMatch>> elementMatches = queryContext.getMapInfo().getModelElementMatches();
+        for (Long modelId : elementMatches.keySet()) {
+            MetricInterpretTool metricInterpretTool = metricInterpretToolMap.get(modelId);
+            if (metricInterpretTool == null) {
+                continue;
+            }
+            if (CollectionUtils.isEmpty(elementMatches.get(modelId))) {
+                continue;
+            }
+            List<MetricOption> metricOptions = metricInterpretTool.getMetricOptions();
+            if (!CollectionUtils.isEmpty(metricOptions)) {
+                List<Long> metricIds = metricOptions.stream()
+                        .map(MetricOption::getMetricId).collect(Collectors.toList());
+                String name = metricInterpretTool.getName();
+                buildQuery(modelId, queryContext, metricIds, elementMatches.get(modelId), name);
+            }
+        }
+    }
+
+    private void buildQuery(Long modelId, QueryContext queryContext,
+                            List<Long> metricIds, List<SchemaElementMatch> schemaElementMatches, String toolName) {
+        PluginSemanticQuery metricInterpretQuery = QueryManager.createPluginQuery(MetricInterpretQuery.QUERY_MODE);
+        Set<SchemaElement> metrics = getMetrics(metricIds, modelId);
+        SemanticParseInfo semanticParseInfo = buildSemanticParseInfo(modelId, queryContext.getRequest(),
+                metrics, schemaElementMatches, toolName);
+        semanticParseInfo.setQueryMode(metricInterpretQuery.getQueryMode());
+        semanticParseInfo.getProperties().put("queryText", queryContext.getRequest().getQueryText());
+        metricInterpretQuery.setParseInfo(semanticParseInfo);
+        queryContext.getCandidateQueries().add(metricInterpretQuery);
+    }
+
+    public Set<SchemaElement> getMetrics(List<Long> metricIds, Long modelId) {
+        SemanticLayer semanticLayer = ComponentFactory.getSemanticLayer();
+        ModelSchema modelSchema = semanticLayer.getModelSchema(modelId, true);
+        Set<SchemaElement> metrics = modelSchema.getMetrics();
+        return metrics.stream().filter(schemaElement -> metricIds.contains(schemaElement.getId()))
+                .collect(Collectors.toSet());
+    }
+
+    private Map<Long, MetricInterpretTool> getMetricInterpretTools(Integer agentId) {
+        AgentService agentService = ContextUtils.getBean(AgentService.class);
+        Agent agent = agentService.getAgent(agentId);
+        if (agent == null) {
+            return new HashMap<>();
+        }
+        List<String> tools = agent.getTools(AgentToolType.INTERPRET);
+        if (CollectionUtils.isEmpty(tools)) {
+            return new HashMap<>();
+        }
+        List<MetricInterpretTool> metricInterpretTools = tools.stream().map(tool ->
+                        JSONObject.parseObject(tool, MetricInterpretTool.class))
+                .filter(tool -> !CollectionUtils.isEmpty(tool.getMetricOptions()))
+                .collect(Collectors.toList());
+        Map<Long, MetricInterpretTool> metricInterpretToolMap = new HashMap<>();
+        for (MetricInterpretTool metricInterpretTool : metricInterpretTools) {
+            metricInterpretToolMap.putIfAbsent(metricInterpretTool.getModelId(),
+                    metricInterpretTool);
+        }
+        return metricInterpretToolMap;
+    }
+
+    private SemanticParseInfo buildSemanticParseInfo(Long modelId, QueryReq queryReq, Set<SchemaElement> metrics,
+                                                     List<SchemaElementMatch> schemaElementMatches, String toolName) {
+        SchemaElement model = new SchemaElement();
+        model.setModel(modelId);
+        model.setId(modelId);
+        SemanticParseInfo semanticParseInfo = new SemanticParseInfo();
+        semanticParseInfo.setMetrics(metrics);
+        SchemaElement dimension = new SchemaElement();
+        dimension.setBizName(TimeDimensionEnum.DAY.getName());
+        semanticParseInfo.setDimensions(Sets.newHashSet(dimension));
+        semanticParseInfo.setElementMatches(schemaElementMatches);
+        semanticParseInfo.setModel(model);
+        semanticParseInfo.setScore(queryReq.getQueryText().length());
+        DateConf dateConf = new DateConf();
+        dateConf.setDateMode(DateConf.DateMode.RECENT);
+        dateConf.setUnit(15);
+        semanticParseInfo.setDateInfo(dateConf);
+        Map<String, Object> properties = new HashMap<>();
+        properties.put("type", "internal");
+        properties.put("name", toolName);
+        semanticParseInfo.setProperties(properties);
+        fillSemanticParseInfo(semanticParseInfo);
+        return semanticParseInfo;
+    }
+
+    private void fillSemanticParseInfo(SemanticParseInfo semanticParseInfo) {
+        List<SchemaElementMatch> schemaElementMatches = semanticParseInfo.getElementMatches();
+        if (!CollectionUtils.isEmpty(schemaElementMatches)) {
+            schemaElementMatches.stream().filter(schemaElementMatch ->
+                            SchemaElementType.VALUE.equals(schemaElementMatch.getElement().getType())
+                                    || SchemaElementType.ID.equals(schemaElementMatch.getElement().getType()))
+                    .forEach(schemaElementMatch -> {
+                        QueryFilter queryFilter = new QueryFilter();
+                        queryFilter.setValue(schemaElementMatch.getWord());
+                        queryFilter.setElementID(schemaElementMatch.getElement().getId());
+                        queryFilter.setName(schemaElementMatch.getElement().getName());
+                        queryFilter.setOperator(FilterOperatorEnum.EQUALS);
+                        queryFilter.setBizName(schemaElementMatch.getElement().getBizName());
+                        semanticParseInfo.getDimensionFilters().add(queryFilter);
+                    });
+        }
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/interpret/MetricOption.java

@@ -0,0 +1,14 @@
+package com.tencent.supersonic.chat.parser.llm.interpret;
+
+
+import lombok.AllArgsConstructor;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@AllArgsConstructor
+@NoArgsConstructor
+public class MetricOption {
+
+    private Long metricId;
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/llm/time/LLMTimeEnhancementParse.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.llm;
+package com.tencent.supersonic.chat.parser.llm.time;
 
 import com.alibaba.fastjson.JSON;
 import com.alibaba.fastjson.JSONObject;
@@ -6,8 +6,8 @@ import com.tencent.supersonic.chat.api.component.SemanticParser;
 import com.tencent.supersonic.chat.api.component.SemanticQuery;
 import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import com.tencent.supersonic.chat.utils.ChatGptHelper;
 import com.tencent.supersonic.common.pojo.DateConf;
+import com.tencent.supersonic.common.util.ChatGptHelper;
 import com.tencent.supersonic.common.util.ContextUtils;
 import lombok.extern.slf4j.Slf4j;
 
@@ -19,8 +19,8 @@ public class LLMTimeEnhancementParse implements SemanticParser {
     public void parse(QueryContext queryContext, ChatContext chatContext) {
         log.info("before queryContext:{},chatContext:{}", queryContext, chatContext);
         ChatGptHelper chatGptHelper = ContextUtils.getBean(ChatGptHelper.class);
-        String inferredTime = chatGptHelper.inferredTime(queryContext.getRequest().getQueryText());
         try {
+            String inferredTime = chatGptHelper.inferredTime(queryContext.getRequest().getQueryText());
             if (!queryContext.getCandidateQueries().isEmpty()) {
                 for (SemanticQuery query : queryContext.getCandidateQueries()) {
                     DateConf dateInfo = query.getParseInfo().getDateInfo();
@@ -43,7 +43,8 @@ public class LLMTimeEnhancementParse implements SemanticParser {
                     (Object) exception.getStackTrace());
         }
 
-        log.info("after queryContext:{},chatContext:{}", queryContext, chatContext);
+        log.info("{} after queryContext:{},chatContext:{}",
+                LLMTimeEnhancementParse.class.getSimpleName(), queryContext, chatContext);
     }
 
 

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/embedding/EmbeddingBasedParser.java

@@ -1,15 +1,14 @@
-package com.tencent.supersonic.chat.parser.embedding;
+package com.tencent.supersonic.chat.parser.plugin.embedding;
 
 import com.google.common.collect.Lists;
 import com.tencent.supersonic.chat.api.component.SemanticParser;
-import com.tencent.supersonic.chat.api.pojo.ChatContext;
-import com.tencent.supersonic.chat.api.pojo.ModelSchema;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
 import com.tencent.supersonic.chat.api.pojo.SchemaElement;
 import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
 import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
-import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
+import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
+import com.tencent.supersonic.chat.api.pojo.ModelSchema;
 import com.tencent.supersonic.chat.api.pojo.request.QueryFilter;
 import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
 import com.tencent.supersonic.chat.parser.ParseMode;
@@ -17,17 +16,18 @@ import com.tencent.supersonic.chat.plugin.Plugin;
 import com.tencent.supersonic.chat.plugin.PluginManager;
 import com.tencent.supersonic.chat.plugin.PluginParseResult;
 import com.tencent.supersonic.chat.query.QueryManager;
+import com.tencent.supersonic.chat.query.llm.dsl.DslQuery;
 import com.tencent.supersonic.chat.query.plugin.PluginSemanticQuery;
-import com.tencent.supersonic.chat.service.PluginService;
 import com.tencent.supersonic.chat.service.SemanticService;
 import com.tencent.supersonic.common.pojo.Constants;
 import com.tencent.supersonic.common.util.ContextUtils;
-import com.tencent.supersonic.semantic.api.query.enums.FilterOperatorEnum;
-import java.util.Comparator;
-import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.Set;
+import java.util.HashMap;
+import java.util.Comparator;
 import java.util.stream.Collectors;
+import com.tencent.supersonic.semantic.api.query.enums.FilterOperatorEnum;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.commons.lang3.tuple.Pair;
@@ -53,48 +53,32 @@ public class EmbeddingBasedParser implements SemanticParser {
         if (CollectionUtils.isEmpty(embeddingRetrievals)) {
             return;
         }
-        PluginService pluginService = ContextUtils.getBean(PluginService.class);
+        List<Plugin> plugins = getPluginList(queryContext);
-        List<Plugin> plugins = pluginService.getPluginList();
         Map<Long, Plugin> pluginMap = plugins.stream().collect(Collectors.toMap(Plugin::getId, p -> p));
         for (RecallRetrieval embeddingRetrieval : embeddingRetrievals) {
             Plugin plugin = pluginMap.get(Long.parseLong(embeddingRetrieval.getId()));
-            if (plugin == null) {
+            if (plugin == null || DslQuery.QUERY_MODE.equalsIgnoreCase(plugin.getType())) {
                 continue;
             }
-            Pair<Boolean, List<Long>> pair = PluginManager.resolve(plugin, queryContext);
+            Pair<Boolean, Set<Long>> pair = PluginManager.resolve(plugin, queryContext);
             log.info("embedding plugin resolve: {}", pair);
             if (pair.getLeft()) {
-                List<Long> modelList = pair.getRight();
+                Set<Long> modelList = pair.getRight();
                 if (CollectionUtils.isEmpty(modelList)) {
                     return;
                 }
-                modelList = distinctModelList(plugin, queryContext.getMapInfo(), modelList);
                 for (Long modelId : modelList) {
                     buildQuery(plugin, Double.parseDouble(embeddingRetrieval.getDistance()), modelId, queryContext,
                             queryContext.getMapInfo().getMatchedElements(modelId));
+                    if (plugin.isContainsAllModel()) {
+                        break;
+                    }
                 }
                 return;
             }
         }
     }
 
-    public List<Long> distinctModelList(Plugin plugin, SchemaMapInfo schemaMapInfo, List<Long> modelList) {
-        if (!plugin.isContainsAllModel()) {
-            return modelList;
-        }
-        boolean noElementMatch = true;
-        for (Long model : modelList) {
-            List<SchemaElementMatch> schemaElementMatches = schemaMapInfo.getMatchedElements(model);
-            if (!CollectionUtils.isEmpty(schemaElementMatches)) {
-                noElementMatch = false;
-            }
-        }
-        if (noElementMatch) {
-            return modelList.subList(0, 1);
-        }
-        return modelList;
-    }
-
     private void buildQuery(Plugin plugin, double distance, Long modelId,
             QueryContext queryContext, List<SchemaElementMatch> schemaElementMatches) {
         log.info("EmbeddingBasedParser Model: {} choose plugin: [{} {}]", modelId, plugin.getId(), plugin.getName());
@@ -114,18 +98,20 @@ public class EmbeddingBasedParser implements SemanticParser {
         if (modelId == null && !CollectionUtils.isEmpty(plugin.getModelList())) {
             modelId = plugin.getModelList().get(0);
         }
-        SchemaElement Model = new SchemaElement();
+        SchemaElement model = new SchemaElement();
-        Model.setModel(modelId);
+        model.setModel(modelId);
-        Model.setId(modelId);
+        model.setId(modelId);
         SemanticParseInfo semanticParseInfo = new SemanticParseInfo();
         semanticParseInfo.setElementMatches(schemaElementMatches);
-        semanticParseInfo.setModel(Model);
+        semanticParseInfo.setModel(model);
         Map<String, Object> properties = new HashMap<>();
         PluginParseResult pluginParseResult = new PluginParseResult();
         pluginParseResult.setPlugin(plugin);
         pluginParseResult.setRequest(queryReq);
         pluginParseResult.setDistance(distance);
         properties.put(Constants.CONTEXT, pluginParseResult);
+        properties.put("type", "plugin");
+        properties.put("name", plugin.getName());
         semanticParseInfo.setProperties(properties);
         semanticParseInfo.setScore(distance);
         fillSemanticParseInfo(semanticParseInfo);
@@ -135,9 +121,9 @@ public class EmbeddingBasedParser implements SemanticParser {
 
     private void setEntity(Long modelId, SemanticParseInfo semanticParseInfo) {
         SemanticService semanticService = ContextUtils.getBean(SemanticService.class);
-        ModelSchema ModelSchema = semanticService.getModelSchema(modelId);
+        ModelSchema modelSchema = semanticService.getModelSchema(modelId);
-        if (ModelSchema != null && ModelSchema.getEntity() != null) {
+        if (modelSchema != null && modelSchema.getEntity() != null) {
-            semanticParseInfo.setEntity(ModelSchema.getEntity());
+            semanticParseInfo.setEntity(modelSchema.getEntity());
         }
     }
 
@@ -176,4 +162,8 @@ public class EmbeddingBasedParser implements SemanticParser {
         }
     }
 
+    protected List<Plugin> getPluginList(QueryContext queryContext) {
+        return PluginManager.getPluginAgentCanSupport(queryContext.getRequest().getAgentId());
+    }
+
 }

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/embedding/EmbeddingConfig.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.embedding;
+package com.tencent.supersonic.chat.parser.plugin.embedding;
 
 import lombok.Data;
 import org.springframework.beans.factory.annotation.Value;

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/embedding/EmbeddingEntityResolver.java

@@ -1,26 +1,26 @@
-package com.tencent.supersonic.chat.parser.embedding;
+package com.tencent.supersonic.chat.parser.plugin.embedding;
 
 import com.alibaba.fastjson.JSONObject;
-import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
 import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
 import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
+import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
 import com.tencent.supersonic.chat.api.pojo.request.QueryFilter;
 import com.tencent.supersonic.chat.api.pojo.request.QueryFilters;
 import com.tencent.supersonic.chat.service.ConfigService;
-import java.util.List;
-import java.util.Objects;
-import java.util.Set;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.collections.CollectionUtils;
 import org.apache.commons.lang3.StringUtils;
 import org.springframework.stereotype.Component;
 
+import java.util.List;
+import java.util.Objects;
+import java.util.Set;
+
 @Slf4j
 @Component("EmbeddingEntityResolver")
 public class EmbeddingEntityResolver {
-
     private ConfigService configService;
 
     public EmbeddingEntityResolver(ConfigService configService) {
@@ -39,8 +39,8 @@ public class EmbeddingEntityResolver {
             }
         }
         entityId = getEntityValueFromSchemaMapInfo(modelId, queryCtx.getMapInfo(), entityElementId);
-        log.info("get entity id:{} from  schema map Info :{} ", entityId,
+        log.info("get entity id:{} from  schema map Info :{} ",
-                JSONObject.toJSONString(queryCtx.getMapInfo()));
+                entityId, JSONObject.toJSONString(queryCtx.getMapInfo()));
         if (entityId == null || entityId == 0) {
             Long entityIdFromChat = getEntityValueFromParseInfo(chatCtx.getParseInfo(), entityElementId);
             if (entityIdFromChat != null && entityIdFromChat > 0) {
@@ -95,4 +95,4 @@ public class EmbeddingEntityResolver {
         return null;
     }
 
-}
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/embedding/EmbeddingResp.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.embedding;
+package com.tencent.supersonic.chat.parser.plugin.embedding;
 
 
 import java.util.List;

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/embedding/RecallRetrieval.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.embedding;
+package com.tencent.supersonic.chat.parser.plugin.embedding;
 
 
 import lombok.Data;

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/FunctionBasedParser.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
 import com.alibaba.fastjson.JSON;
 import com.tencent.supersonic.chat.api.component.SemanticParser;
@@ -14,23 +14,21 @@ import com.tencent.supersonic.chat.plugin.PluginManager;
 import com.tencent.supersonic.chat.plugin.PluginParseConfig;
 import com.tencent.supersonic.chat.plugin.PluginParseResult;
 import com.tencent.supersonic.chat.query.QueryManager;
-import com.tencent.supersonic.chat.query.dsl.DSLQuery;
 import com.tencent.supersonic.chat.query.plugin.PluginSemanticQuery;
+import com.tencent.supersonic.chat.query.llm.dsl.DslQuery;
 import com.tencent.supersonic.chat.service.PluginService;
 import com.tencent.supersonic.chat.utils.ComponentFactory;
 import com.tencent.supersonic.common.pojo.Constants;
 import com.tencent.supersonic.common.util.ContextUtils;
-import com.tencent.supersonic.common.util.JsonUtil;
 import java.net.URI;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
-import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
+import java.util.Objects;
+import java.util.Map;
+import java.util.HashMap;
 import java.util.stream.Collectors;
+import com.tencent.supersonic.common.util.JsonUtil;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.commons.lang3.tuple.Pair;
@@ -46,11 +44,6 @@ import org.springframework.web.util.UriComponentsBuilder;
 @Slf4j
 public class FunctionBasedParser implements SemanticParser {
 
-    public static final double FUNCTION_BONUS_THRESHOLD = 200;
-
-    public static final double SKIP_DSL_LENGTH = 10;
-
-
     @Override
     public void parse(QueryContext queryCtx, ChatContext chatCtx) {
         FunctionCallInfoConfig functionCallConfig = ContextUtils.getBean(FunctionCallInfoConfig.class);
@@ -62,12 +55,21 @@ public class FunctionBasedParser implements SemanticParser {
             return;
         }
         List<PluginParseConfig> functionDOList = getFunctionDO(queryCtx.getRequest().getModelId(), queryCtx);
-        FunctionReq functionReq = FunctionReq.builder()
+        if (CollectionUtils.isEmpty(functionDOList)) {
-                .queryText(queryCtx.getRequest().getQueryText())
+            log.info("function call parser, plugin is empty, skip");
-                .pluginConfigs(functionDOList).build();
+            return;
-        FunctionResp functionResp = requestFunction(functionUrl, functionReq);
+        }
+        FunctionResp functionResp = new FunctionResp();
+        if (functionDOList.size() == 1) {
+            functionResp.setToolSelection(functionDOList.iterator().next().getName());
+        } else {
+            FunctionReq functionReq = FunctionReq.builder()
+                    .queryText(queryCtx.getRequest().getQueryText())
+                    .pluginConfigs(functionDOList).build();
+            functionResp = requestFunction(functionUrl, functionReq);
+        }
         log.info("requestFunction result:{}", functionResp.getToolSelection());
-        if (skipFunction(queryCtx, functionResp)) {
+        if (skipFunction(functionResp)) {
             return;
         }
         PluginParseResult functionCallParseResult = new PluginParseResult();
@@ -83,10 +85,10 @@ public class FunctionBasedParser implements SemanticParser {
         functionCallParseResult.setPlugin(plugin);
         log.info("QueryManager PluginQueryModes:{}", QueryManager.getPluginQueryModes());
         PluginSemanticQuery semanticQuery = QueryManager.createPluginQuery(toolSelection);
-        ModelResolver ModelResolver = ComponentFactory.getModelResolver();
+        ModelResolver modelResolver = ComponentFactory.getModelResolver();
         log.info("plugin ModelList:{}", plugin.getModelList());
-        Pair<Boolean, List<Long>> pluginResolveResult = PluginManager.resolve(plugin, queryCtx);
+        Pair<Boolean, Set<Long>> pluginResolveResult = PluginManager.resolve(plugin, queryCtx);
-        Long modelId = ModelResolver.resolve(queryCtx, chatCtx, pluginResolveResult.getRight());
+        Long modelId = modelResolver.resolve(queryCtx, chatCtx, pluginResolveResult.getRight());
         log.info("FunctionBasedParser modelId:{}", modelId);
         if ((Objects.isNull(modelId) || modelId <= 0) && !plugin.isContainsAllModel()) {
             log.info("Model is null, skip the parse, select tool: {}", toolSelection);
@@ -102,47 +104,27 @@ public class FunctionBasedParser implements SemanticParser {
         functionCallParseResult.setRequest(queryCtx.getRequest());
         Map<String, Object> properties = new HashMap<>();
         properties.put(Constants.CONTEXT, functionCallParseResult);
+        properties.put("type", "plugin");
+        properties.put("name", plugin.getName());
         parseInfo.setProperties(properties);
-        parseInfo.setScore(FUNCTION_BONUS_THRESHOLD);
+        parseInfo.setScore(queryCtx.getRequest().getQueryText().length());
         parseInfo.setQueryMode(semanticQuery.getQueryMode());
-        SchemaElement Model = new SchemaElement();
+        SchemaElement model = new SchemaElement();
-        Model.setModel(modelId);
+        model.setModel(modelId);
-        Model.setId(modelId);
+        model.setId(modelId);
-        parseInfo.setModel(Model);
+        parseInfo.setModel(model);
         queryCtx.getCandidateQueries().add(semanticQuery);
     }
 
-
+    private boolean skipFunction(FunctionResp functionResp) {
-    private Set<Long> getMatchModels(QueryContext queryCtx) {
+        return Objects.isNull(functionResp) || StringUtils.isBlank(functionResp.getToolSelection());
-        Set<Long> result = new HashSet<>();
-        Long modelId = queryCtx.getRequest().getModelId();
-        if (Objects.nonNull(modelId) && modelId > 0) {
-            result.add(modelId);
-            return result;
-        }
-        return queryCtx.getMapInfo().getMatchedModels();
-    }
-
-    private boolean skipFunction(QueryContext queryCtx, FunctionResp functionResp) {
-        if (Objects.isNull(functionResp) || StringUtils.isBlank(functionResp.getToolSelection())) {
-            return true;
-        }
-        String queryText = queryCtx.getRequest().getQueryText();
-
-        if (functionResp.getToolSelection().equalsIgnoreCase(DSLQuery.QUERY_MODE)
-                && queryText.length() < SKIP_DSL_LENGTH) {
-            log.info("queryText length is :{}, less than the threshold :{}, skip dsl.", queryText.length(),
-                    SKIP_DSL_LENGTH);
-            return true;
-        }
-        return false;
     }
 
     private List<PluginParseConfig> getFunctionDO(Long modelId, QueryContext queryContext) {
         log.info("user decide Model:{}", modelId);
-        List<Plugin> plugins = PluginManager.getPlugins();
+        List<Plugin> plugins = getPluginList(queryContext);
         List<PluginParseConfig> functionDOList = plugins.stream().filter(plugin -> {
-            if (DSLQuery.QUERY_MODE.equalsIgnoreCase(plugin.getType())) {
+            if (DslQuery.QUERY_MODE.equalsIgnoreCase(plugin.getType())) {
                 return false;
             }
             if (plugin.getParseModeConfig() == null) {
@@ -153,12 +135,12 @@ public class FunctionBasedParser implements SemanticParser {
             if (StringUtils.isBlank(pluginParseConfig.getName())) {
                 return false;
             }
-            Pair<Boolean, List<Long>> pluginResolverResult = PluginManager.resolve(plugin, queryContext);
+            Pair<Boolean, Set<Long>> pluginResolverResult = PluginManager.resolve(plugin, queryContext);
-            log.info("embedding plugin [{}-{}] resolve: {}", plugin.getId(), plugin.getName(), pluginResolverResult);
+            log.info("plugin [{}-{}] resolve: {}", plugin.getId(), plugin.getName(), pluginResolverResult);
             if (!pluginResolverResult.getLeft()) {
                 return false;
             } else {
-                List<Long> resolveModel = pluginResolverResult.getRight();
+                Set<Long> resolveModel = pluginResolverResult.getRight();
                 if (modelId != null && modelId > 0) {
                     if (plugin.isContainsAllModel()) {
                         return true;
@@ -172,20 +154,6 @@ public class FunctionBasedParser implements SemanticParser {
         return functionDOList;
     }
 
-    private List<String> getFunctionNames(Set<Long> matchedModels) {
-        List<Plugin> plugins = PluginManager.getPlugins();
-        Set<String> functionNames = plugins.stream()
-                .filter(entry -> {
-                            if (!CollectionUtils.isEmpty(entry.getModelList()) && !CollectionUtils.isEmpty(matchedModels)) {
-                                return entry.getModelList().stream().anyMatch(matchedModels::contains);
-                            }
-                            return true;
-                        }
-                ).map(Plugin::getName).collect(Collectors.toSet());
-        functionNames.add(DSLQuery.QUERY_MODE);
-        return new ArrayList<>(functionNames);
-    }
-
     public FunctionResp requestFunction(String url, FunctionReq functionReq) {
         HttpHeaders headers = new HttpHeaders();
         long startTime = System.currentTimeMillis();
@@ -205,4 +173,8 @@ public class FunctionBasedParser implements SemanticParser {
         }
         return null;
     }
+
+    protected List<Plugin> getPluginList(QueryContext queryContext) {
+        return PluginManager.getPluginAgentCanSupport(queryContext.getRequest().getAgentId());
+    }
 }

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/FunctionFiled.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
 
 import lombok.Data;

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/FunctionReq.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
 import com.tencent.supersonic.chat.plugin.PluginParseConfig;
 import java.util.List;

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/FunctionResp.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
 import lombok.Data;
 

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/HeuristicModelResolver.java

@@ -1,38 +1,40 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
-import com.tencent.supersonic.chat.api.component.SemanticQuery;
-import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
+import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
+import com.tencent.supersonic.chat.api.pojo.ChatContext;
+import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
 import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
 import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
-import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
-import com.tencent.supersonic.chat.api.pojo.SemanticParseInfo;
 import com.tencent.supersonic.chat.api.pojo.request.QueryReq;
-import java.util.HashMap;
+import com.tencent.supersonic.chat.api.component.SemanticQuery;
-import java.util.HashSet;
+import lombok.extern.slf4j.Slf4j;
-import java.util.List;
+
 import java.util.Map;
+import java.util.HashMap;
 import java.util.Objects;
+import java.util.List;
+import java.util.HashSet;
 import java.util.Set;
 import java.util.stream.Collectors;
-import lombok.extern.slf4j.Slf4j;
+
 import org.apache.commons.collections.CollectionUtils;
 
 @Slf4j
 public class HeuristicModelResolver implements ModelResolver {
 
-    protected static Long selectModelBySchemaElementCount(Map<Long, SemanticQuery> ModelQueryModes,
+    protected static Long selectModelBySchemaElementCount(Map<Long, SemanticQuery> modelQueryModes,
-            SchemaMapInfo schemaMap) {
+                                                          SchemaMapInfo schemaMap) {
-        Map<Long, ModelMatchResult> ModelTypeMap = getModelTypeMap(schemaMap);
+        Map<Long, ModelMatchResult> modelTypeMap = getModelTypeMap(schemaMap);
-        if (ModelTypeMap.size() == 1) {
+        if (modelTypeMap.size() == 1) {
-            Long ModelSelect = ModelTypeMap.entrySet().stream().collect(Collectors.toList()).get(0).getKey();
+            Long modelSelect = modelTypeMap.entrySet().stream().collect(Collectors.toList()).get(0).getKey();
-            if (ModelQueryModes.containsKey(ModelSelect)) {
+            if (modelQueryModes.containsKey(modelSelect)) {
-                log.info("selectModel with only one Model [{}]", ModelSelect);
+                log.info("selectModel with only one Model [{}]", modelSelect);
-                return ModelSelect;
+                return modelSelect;
             }
         } else {
-            Map.Entry<Long, ModelMatchResult> maxModel = ModelTypeMap.entrySet().stream()
+            Map.Entry<Long, ModelMatchResult> maxModel = modelTypeMap.entrySet().stream()
-                    .filter(entry -> ModelQueryModes.containsKey(entry.getKey()))
+                    .filter(entry -> modelQueryModes.containsKey(entry.getKey()))
                     .sorted((o1, o2) -> {
                         int difference = o2.getValue().getCount() - o1.getValue().getCount();
                         if (difference == 0) {
@@ -54,23 +56,24 @@ public class HeuristicModelResolver implements ModelResolver {
      *
      * @return false will use context Model, true will use other Model , maybe include context Model
      */
-    protected static boolean isAllowSwitch(Map<Long, SemanticQuery> ModelQueryModes, SchemaMapInfo schemaMap,
+    protected static boolean isAllowSwitch(Map<Long, SemanticQuery> modelQueryModes, SchemaMapInfo schemaMap,
-            ChatContext chatCtx, QueryReq searchCtx, Long modelId, List<Long> restrictiveModels) {
+                                           ChatContext chatCtx, QueryReq searchCtx,
+                                           Long modelId, Set<Long> restrictiveModels) {
         if (!Objects.nonNull(modelId) || modelId <= 0) {
             return true;
         }
         // except content Model, calculate the number of types for each Model, if numbers<=1 will not switch
-        Map<Long, ModelMatchResult> ModelTypeMap = getModelTypeMap(schemaMap);
+        Map<Long, ModelMatchResult> modelTypeMap = getModelTypeMap(schemaMap);
-        log.info("isAllowSwitch ModelTypeMap [{}]", ModelTypeMap);
+        log.info("isAllowSwitch ModelTypeMap [{}]", modelTypeMap);
-        long otherModelTypeNumBigOneCount = ModelTypeMap.entrySet().stream()
+        long otherModelTypeNumBigOneCount = modelTypeMap.entrySet().stream()
-                .filter(entry -> ModelQueryModes.containsKey(entry.getKey()) && !entry.getKey().equals(modelId))
+                .filter(entry -> modelQueryModes.containsKey(entry.getKey()) && !entry.getKey().equals(modelId))
                 .filter(entry -> entry.getValue().getCount() > 1).count();
         if (otherModelTypeNumBigOneCount >= 1) {
             return true;
         }
         // if query text only contain time , will not switch
-        if (!CollectionUtils.isEmpty(ModelQueryModes.values())) {
+        if (!CollectionUtils.isEmpty(modelQueryModes.values())) {
-            for (SemanticQuery semanticQuery : ModelQueryModes.values()) {
+            for (SemanticQuery semanticQuery : modelQueryModes.values()) {
                 if (semanticQuery == null) {
                     continue;
                 }
@@ -104,14 +107,14 @@ public class HeuristicModelResolver implements ModelResolver {
     }
 
     public static Map<Long, ModelMatchResult> getModelTypeMap(SchemaMapInfo schemaMap) {
-        Map<Long, ModelMatchResult> ModelCount = new HashMap<>();
+        Map<Long, ModelMatchResult> modelCount = new HashMap<>();
         for (Map.Entry<Long, List<SchemaElementMatch>> entry : schemaMap.getModelElementMatches().entrySet()) {
             List<SchemaElementMatch> schemaElementMatches = schemaMap.getMatchedElements(entry.getKey());
             if (schemaElementMatches != null && schemaElementMatches.size() > 0) {
-                if (!ModelCount.containsKey(entry.getKey())) {
+                if (!modelCount.containsKey(entry.getKey())) {
-                    ModelCount.put(entry.getKey(), new ModelMatchResult());
+                    modelCount.put(entry.getKey(), new ModelMatchResult());
                 }
-                ModelMatchResult ModelMatchResult = ModelCount.get(entry.getKey());
+                ModelMatchResult modelMatchResult = modelCount.get(entry.getKey());
                 Set<SchemaElementType> schemaElementTypes = new HashSet<>();
                 schemaElementMatches.stream()
                         .forEach(schemaElementMatch -> schemaElementTypes.add(
@@ -121,17 +124,17 @@ public class HeuristicModelResolver implements ModelResolver {
                                 ((int) ((o2.getSimilarity() - o1.getSimilarity()) * 100))
                         ).findFirst().orElse(null);
                 if (schemaElementMatchMax != null) {
-                    ModelMatchResult.setMaxSimilarity(schemaElementMatchMax.getSimilarity());
+                    modelMatchResult.setMaxSimilarity(schemaElementMatchMax.getSimilarity());
                 }
-                ModelMatchResult.setCount(schemaElementTypes.size());
+                modelMatchResult.setCount(schemaElementTypes.size());
 
             }
         }
-        return ModelCount;
+        return modelCount;
     }
 
 
-    public Long resolve(QueryContext queryContext, ChatContext chatCtx, List<Long> restrictiveModels) {
+    public Long resolve(QueryContext queryContext, ChatContext chatCtx, Set<Long> restrictiveModels) {
         Long modelId = queryContext.getRequest().getModelId();
         if (Objects.nonNull(modelId) && modelId > 0) {
             if (CollectionUtils.isNotEmpty(restrictiveModels) && restrictiveModels.contains(modelId)) {
@@ -147,41 +150,41 @@ public class HeuristicModelResolver implements ModelResolver {
                     .filter(restrictiveModels::contains)
                     .collect(Collectors.toSet());
         }
-        Map<Long, SemanticQuery> ModelQueryModes = new HashMap<>();
+        Map<Long, SemanticQuery> modelQueryModes = new HashMap<>();
         for (Long matchedModel : matchedModels) {
-            ModelQueryModes.put(matchedModel, null);
+            modelQueryModes.put(matchedModel, null);
         }
-        if (ModelQueryModes.size() == 1) {
+        if (modelQueryModes.size() == 1) {
-            return ModelQueryModes.keySet().stream().findFirst().get();
+            return modelQueryModes.keySet().stream().findFirst().get();
         }
-        return resolve(ModelQueryModes, queryContext, chatCtx,
+        return resolve(modelQueryModes, queryContext, chatCtx,
                 queryContext.getMapInfo(), restrictiveModels);
     }
 
-    public Long resolve(Map<Long, SemanticQuery> ModelQueryModes, QueryContext queryContext,
+    public Long resolve(Map<Long, SemanticQuery> modelQueryModes, QueryContext queryContext,
-            ChatContext chatCtx, SchemaMapInfo schemaMap, List<Long> restrictiveModels) {
+                        ChatContext chatCtx, SchemaMapInfo schemaMap, Set<Long> restrictiveModels) {
-        Long selectModel = selectModel(ModelQueryModes, queryContext.getRequest(), chatCtx, schemaMap,
+        Long selectModel = selectModel(modelQueryModes, queryContext.getRequest(),
-                restrictiveModels);
+                chatCtx, schemaMap, restrictiveModels);
         if (selectModel > 0) {
             log.info("selectModel {} ", selectModel);
             return selectModel;
         }
         // get the max SchemaElementType number
-        return selectModelBySchemaElementCount(ModelQueryModes, schemaMap);
+        return selectModelBySchemaElementCount(modelQueryModes, schemaMap);
     }
 
-    public Long selectModel(Map<Long, SemanticQuery> ModelQueryModes, QueryReq queryContext,
+    public Long selectModel(Map<Long, SemanticQuery> modelQueryModes, QueryReq queryContext,
-            ChatContext chatCtx,
+                            ChatContext chatCtx,
-            SchemaMapInfo schemaMap, List<Long> restrictiveModels) {
+                            SchemaMapInfo schemaMap, Set<Long> restrictiveModels) {
         // if QueryContext has modelId and in ModelQueryModes
-        if (ModelQueryModes.containsKey(queryContext.getModelId())) {
+        if (modelQueryModes.containsKey(queryContext.getModelId())) {
             log.info("selectModel from QueryContext [{}]", queryContext.getModelId());
             return queryContext.getModelId();
         }
         // if ChatContext has modelId and in ModelQueryModes
         if (chatCtx.getParseInfo().getModelId() > 0) {
             Long modelId = chatCtx.getParseInfo().getModelId();
-            if (!isAllowSwitch(ModelQueryModes, schemaMap, chatCtx, queryContext, modelId, restrictiveModels)) {
+            if (!isAllowSwitch(modelQueryModes, schemaMap, chatCtx, queryContext, modelId, restrictiveModels)) {
                 log.info("selectModel from ChatContext [{}]", modelId);
                 return modelId;
             }

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/ModelMatchResult.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
 import lombok.Data;
 

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/ModelResolver.java

@@ -1,12 +1,12 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
 
 import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import java.util.List;
+import java.util.Set;
 
 public interface ModelResolver {
 
-    Long resolve(QueryContext queryContext, ChatContext chatCtx, List<Long> restrictiveModels);
+    Long resolve(QueryContext queryContext, ChatContext chatCtx, Set<Long> restrictiveModels);
 
-}
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/plugin/function/Parameters.java

@@ -1,4 +1,4 @@
-package com.tencent.supersonic.chat.parser.function;
+package com.tencent.supersonic.chat.parser.plugin.function;
 
 import java.util.List;
 import java.util.Map;

chat/core/src/main/java/com/tencent/supersonic/chat/parser/rule/AgentCheckParser.java

@@ -0,0 +1,70 @@
+package com.tencent.supersonic.chat.parser.rule;
+
+import com.alibaba.fastjson.JSONObject;
+import com.google.common.collect.Lists;
+import com.tencent.supersonic.chat.agent.Agent;
+import com.tencent.supersonic.chat.agent.tool.AgentToolType;
+import com.tencent.supersonic.chat.agent.tool.RuleQueryTool;
+import com.tencent.supersonic.chat.api.component.SemanticParser;
+import com.tencent.supersonic.chat.api.component.SemanticQuery;
+import com.tencent.supersonic.chat.api.pojo.ChatContext;
+import com.tencent.supersonic.chat.api.pojo.QueryContext;
+import com.tencent.supersonic.chat.service.AgentService;
+import com.tencent.supersonic.common.util.ContextUtils;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.commons.collections.CollectionUtils;
+import java.util.List;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class AgentCheckParser implements SemanticParser {
+
+    @Override
+    public void parse(QueryContext queryContext, ChatContext chatContext) {
+        List<SemanticQuery> queries = queryContext.getCandidateQueries();
+        agentCanSupport(queryContext.getRequest().getAgentId(), queries);
+    }
+
+    private void agentCanSupport(Integer agentId, List<SemanticQuery> queries) {
+        AgentService agentService = ContextUtils.getBean(AgentService.class);
+        Agent agent = agentService.getAgent(agentId);
+        if (agent == null) {
+            return;
+        }
+        List<RuleQueryTool> queryTools = getRuleTools(agentId);
+        if (CollectionUtils.isEmpty(queryTools)) {
+            queries.clear();
+            return;
+        }
+        log.info("queries resolved:{} {}", agent.getName(),
+                queries.stream().map(SemanticQuery::getQueryMode).collect(Collectors.toList()));
+        queries.removeIf(query -> {
+            for (RuleQueryTool tool : queryTools) {
+                if (!tool.getQueryModes().contains(query.getQueryMode())) {
+                    return true;
+                }
+                if (tool.isContainsAllModel() || tool.getModelIds().contains(query.getParseInfo().getModelId())) {
+                    return false;
+                }
+            }
+            return true;
+        });
+        log.info("rule queries witch can be supported by agent :{} {}", agent.getName(),
+                queries.stream().map(SemanticQuery::getQueryMode).collect(Collectors.toList()));
+    }
+
+    private static List<RuleQueryTool> getRuleTools(Integer agentId) {
+        AgentService agentService = ContextUtils.getBean(AgentService.class);
+        Agent agent = agentService.getAgent(agentId);
+        if (agent == null) {
+            return Lists.newArrayList();
+        }
+        List<String> tools = agent.getTools(AgentToolType.RULE);
+        if (CollectionUtils.isEmpty(tools)) {
+            return Lists.newArrayList();
+        }
+        return tools.stream().map(tool -> JSONObject.parseObject(tool, RuleQueryTool.class))
+                .collect(Collectors.toList());
+    }
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/parser/rule/AggregateTypeParser.java

@@ -14,6 +14,7 @@ import com.tencent.supersonic.chat.api.component.SemanticQuery;
 import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
 import com.tencent.supersonic.common.pojo.enums.AggregateTypeEnum;
+
 import java.util.AbstractMap;
 import java.util.HashMap;
 import java.util.Map;
@@ -21,6 +22,7 @@ import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 import java.util.stream.Collectors;
 import java.util.stream.Stream;
+
 import lombok.AllArgsConstructor;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.commons.lang3.StringUtils;

chat/core/src/main/java/com/tencent/supersonic/chat/parser/rule/ContextInheritParser.java

@@ -1,12 +1,5 @@
 package com.tencent.supersonic.chat.parser.rule;
 
-import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.DIMENSION;
-import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.ENTITY;
-import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.ID;
-import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.METRIC;
-import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.MODEL;
-import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.VALUE;
-
 import com.tencent.supersonic.chat.api.component.SemanticParser;
 import com.tencent.supersonic.chat.api.component.SemanticQuery;
 import com.tencent.supersonic.chat.api.pojo.ChatContext;
@@ -15,8 +8,8 @@ import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
 import com.tencent.supersonic.chat.api.pojo.SchemaElementType;
 import com.tencent.supersonic.chat.query.QueryManager;
 import com.tencent.supersonic.chat.query.rule.RuleSemanticQuery;
-import com.tencent.supersonic.chat.query.rule.metric.MetricEntityQuery;
 import com.tencent.supersonic.chat.query.rule.metric.MetricModelQuery;
+import com.tencent.supersonic.chat.query.rule.metric.MetricEntityQuery;
 import com.tencent.supersonic.chat.query.rule.metric.MetricSemanticQuery;
 import java.util.AbstractMap;
 import java.util.ArrayList;
@@ -28,6 +21,13 @@ import java.util.stream.Collectors;
 import java.util.stream.Stream;
 import lombok.extern.slf4j.Slf4j;
 
+import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.METRIC;
+import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.DIMENSION;
+import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.VALUE;
+import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.ENTITY;
+import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.MODEL;
+import static com.tencent.supersonic.chat.api.pojo.SchemaElementType.ID;
+
 @Slf4j
 public class ContextInheritParser implements SemanticParser {
 

chat/core/src/main/java/com/tencent/supersonic/chat/parser/rule/QueryModeParser.java

@@ -1,10 +1,10 @@
 package com.tencent.supersonic.chat.parser.rule;
 
 import com.tencent.supersonic.chat.api.component.SemanticParser;
-import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
+import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.SchemaMapInfo;
+import com.tencent.supersonic.chat.api.pojo.SchemaElementMatch;
 import com.tencent.supersonic.chat.query.rule.RuleSemanticQuery;
 import java.util.List;
 import lombok.extern.slf4j.Slf4j;
@@ -15,12 +15,10 @@ public class QueryModeParser implements SemanticParser {
     @Override
     public void parse(QueryContext queryContext, ChatContext chatContext) {
         SchemaMapInfo mapInfo = queryContext.getMapInfo();
-
         // iterate all schemaElementMatches to resolve semantic query
         for (Long modelId : mapInfo.getMatchedModels()) {
             List<SchemaElementMatch> elementMatches = mapInfo.getMatchedElements(modelId);
             List<RuleSemanticQuery> queries = RuleSemanticQuery.resolve(elementMatches, queryContext);
-
             for (RuleSemanticQuery query : queries) {
                 query.fillParseInfo(modelId, queryContext, chatContext);
                 queryContext.getCandidateQueries().add(query);

chat/core/src/main/java/com/tencent/supersonic/chat/parser/rule/TimeRangeParser.java

@@ -4,21 +4,23 @@ import com.tencent.supersonic.chat.api.component.SemanticParser;
 import com.tencent.supersonic.chat.api.component.SemanticQuery;
 import com.tencent.supersonic.chat.api.pojo.ChatContext;
 import com.tencent.supersonic.chat.api.pojo.QueryContext;
-import com.tencent.supersonic.chat.query.QueryManager;
 import com.tencent.supersonic.chat.query.rule.RuleSemanticQuery;
+import com.tencent.supersonic.chat.query.QueryManager;
 import com.tencent.supersonic.common.pojo.Constants;
 import com.tencent.supersonic.common.pojo.DateConf;
-import com.xkzhangsan.time.nlp.TimeNLP;
+
-import com.xkzhangsan.time.nlp.TimeNLPUtil;
 import java.text.DateFormat;
 import java.text.ParseException;
 import java.text.SimpleDateFormat;
 import java.time.LocalDate;
+import java.util.Stack;
 import java.util.Date;
 import java.util.List;
-import java.util.Stack;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
+
+import com.xkzhangsan.time.nlp.TimeNLP;
+import com.xkzhangsan.time.nlp.TimeNLPUtil;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.logging.log4j.util.Strings;
 

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/AgentDO.java

@@ -0,0 +1,206 @@
+package com.tencent.supersonic.chat.persistence.dataobject;
+
+import java.util.Date;
+
+public class AgentDO {
+    /**
+     */
+    private Integer id;
+
+    /**
+     */
+    private String name;
+
+    /**
+     */
+    private String description;
+
+    /**
+     * 0 offline, 1 online
+     */
+    private Integer status;
+
+    /**
+     */
+    private String examples;
+
+    /**
+     */
+    private String config;
+
+    /**
+     */
+    private String createdBy;
+
+    /**
+     */
+    private Date createdAt;
+
+    /**
+     */
+    private String updatedBy;
+
+    /**
+     */
+    private Date updatedAt;
+
+    /**
+     */
+    private Integer enableSearch;
+
+    /**
+     * @return id
+     */
+    public Integer getId() {
+        return id;
+    }
+
+    /**
+     * @param id
+     */
+    public void setId(Integer id) {
+        this.id = id;
+    }
+
+    /**
+     * @return name
+     */
+    public String getName() {
+        return name;
+    }
+
+    /**
+     * @param name
+     */
+    public void setName(String name) {
+        this.name = name == null ? null : name.trim();
+    }
+
+    /**
+     * @return description
+     */
+    public String getDescription() {
+        return description;
+    }
+
+    /**
+     * @param description
+     */
+    public void setDescription(String description) {
+        this.description = description == null ? null : description.trim();
+    }
+
+    /**
+     * 0 offline, 1 online
+     * @return status 0 offline, 1 online
+     */
+    public Integer getStatus() {
+        return status;
+    }
+
+    /**
+     * 0 offline, 1 online
+     * @param status 0 offline, 1 online
+     */
+    public void setStatus(Integer status) {
+        this.status = status;
+    }
+
+    /**
+     * @return examples
+     */
+    public String getExamples() {
+        return examples;
+    }
+
+    /**
+     * @param examples
+     */
+    public void setExamples(String examples) {
+        this.examples = examples == null ? null : examples.trim();
+    }
+
+    /**
+     * @return config
+     */
+    public String getConfig() {
+        return config;
+    }
+
+    /**
+     * @param config
+     */
+    public void setConfig(String config) {
+        this.config = config == null ? null : config.trim();
+    }
+
+    /**
+     * @return created_by
+     */
+    public String getCreatedBy() {
+        return createdBy;
+    }
+
+    /**
+     * @param createdBy
+     */
+    public void setCreatedBy(String createdBy) {
+        this.createdBy = createdBy == null ? null : createdBy.trim();
+    }
+
+    /**
+     * @return created_at
+     */
+    public Date getCreatedAt() {
+        return createdAt;
+    }
+
+    /**
+     * @param createdAt
+     */
+    public void setCreatedAt(Date createdAt) {
+        this.createdAt = createdAt;
+    }
+
+    /**
+     * @return updated_by
+     */
+    public String getUpdatedBy() {
+        return updatedBy;
+    }
+
+    /**
+     * @param updatedBy
+     */
+    public void setUpdatedBy(String updatedBy) {
+        this.updatedBy = updatedBy == null ? null : updatedBy.trim();
+    }
+
+    /**
+     * @return updated_at
+     */
+    public Date getUpdatedAt() {
+        return updatedAt;
+    }
+
+    /**
+     * @param updatedAt
+     */
+    public void setUpdatedAt(Date updatedAt) {
+        this.updatedAt = updatedAt;
+    }
+
+    /**
+     * @return enable_search
+     */
+    public Integer getEnableSearch() {
+        return enableSearch;
+    }
+
+    /**
+     * @param enableSearch
+     */
+    public void setEnableSearch(Integer enableSearch) {
+        this.enableSearch = enableSearch;
+    }
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/ChatParseDO.java

@@ -0,0 +1,142 @@
+package com.tencent.supersonic.chat.persistence.dataobject;
+
+import java.util.Date;
+
+
+public class ChatParseDO {
+
+    /**
+     * questionId
+     */
+    private Long questionId;
+
+    /**
+     * chatId
+     */
+    private Long chatId;
+
+    /**
+     * parseId
+     */
+    private Integer parseId;
+
+    /**
+     * createTime
+     */
+    private Date createTime;
+
+    /**
+     * queryText
+     */
+    private String queryText;
+
+    /**
+     * userName
+     */
+    private String userName;
+
+
+    /**
+     * parseInfo
+     */
+    private String parseInfo;
+
+    /**
+     * isCandidate
+     */
+    private Integer isCandidate;
+
+    /**
+     * return question_id
+     */
+    public Long getQuestionId() {
+        return questionId;
+    }
+
+    /**
+     * questionId
+     */
+    public void setQuestionId(Long questionId) {
+        this.questionId = questionId;
+    }
+
+    /**
+     * return create_time
+     */
+    public Date getCreateTime() {
+        return createTime;
+    }
+
+    /**
+     * createTime
+     */
+    public void setCreateTime(Date createTime) {
+        this.createTime = createTime;
+    }
+
+    /**
+     * return user_name
+     */
+    public String getUserName() {
+        return userName;
+    }
+
+    /**
+     * userName
+     */
+    public void setUserName(String userName) {
+        this.userName = userName == null ? null : userName.trim();
+    }
+
+    /**
+     * return chat_id
+     */
+    public Long getChatId() {
+        return chatId;
+    }
+
+    /**
+     * chatId
+     */
+    public void setChatId(Long chatId) {
+        this.chatId = chatId;
+    }
+
+    /**
+     * return query_text
+     */
+    public String getQueryText() {
+        return queryText;
+    }
+
+    /**
+     * queryText
+     */
+    public void setQueryText(String queryText) {
+        this.queryText = queryText == null ? null : queryText.trim();
+    }
+
+    public Integer getIsCandidate() {
+        return isCandidate;
+    }
+
+    public Integer getParseId() {
+        return parseId;
+    }
+
+    public String getParseInfo() {
+        return parseInfo;
+    }
+
+    public void setParseId(Integer parseId) {
+        this.parseId = parseId;
+    }
+
+    public void setIsCandidate(Integer isCandidate) {
+        this.isCandidate = isCandidate;
+    }
+
+    public void setParseInfo(String parseInfo) {
+        this.parseInfo = parseInfo;
+    }
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/ChatQueryDO.java

@@ -2,177 +2,196 @@ package com.tencent.supersonic.chat.persistence.dataobject;
 
 import java.util.Date;
 
-
 public class ChatQueryDO {
-
     /**
-     * questionId
      */
     private Long questionId;
 
     /**
-     * createTime
+     */
+    private Integer agentId;
+
+    /**
      */
     private Date createTime;
 
     /**
-     * userName
      */
     private String userName;
 
     /**
-     * queryState
      */
     private Integer queryState;
 
     /**
-     * chatId
      */
     private Long chatId;
 
     /**
-     * score
      */
     private Integer score;
 
     /**
-     * feedback
      */
     private String feedback;
 
     /**
-     * queryText
      */
     private String queryText;
 
     /**
-     * queryResponse
      */
     private String queryResult;
 
     /**
-     * return question_id
+     * @return question_id
      */
     public Long getQuestionId() {
         return questionId;
     }
 
     /**
-     * questionId
+     * @param questionId
      */
     public void setQuestionId(Long questionId) {
         this.questionId = questionId;
     }
 
     /**
-     * return create_time
+     * @return agent_id
+     */
+    public Integer getAgentId() {
+        return agentId;
+    }
+
+    /**
+     * @param agentId
+     */
+    public void setAgentId(Integer agentId) {
+        this.agentId = agentId;
+    }
+
+    /**
+     * @return create_time
      */
     public Date getCreateTime() {
         return createTime;
     }
 
     /**
-     * createTime
+     * @param createTime
      */
     public void setCreateTime(Date createTime) {
         this.createTime = createTime;
     }
 
     /**
-     * return user_name
+     * @return user_name
      */
     public String getUserName() {
         return userName;
     }
 
     /**
-     * userName
+     * @param userName
      */
     public void setUserName(String userName) {
         this.userName = userName == null ? null : userName.trim();
     }
 
     /**
-     * return query_state
+     *
+     * @return query_state
      */
     public Integer getQueryState() {
         return queryState;
     }
 
     /**
-     * queryState
+     *
+     * @param queryState
      */
     public void setQueryState(Integer queryState) {
         this.queryState = queryState;
     }
 
     /**
-     * return chat_id
+     *
+     * @return chat_id
      */
     public Long getChatId() {
         return chatId;
     }
 
     /**
-     * chatId
+     *
+     * @param chatId
      */
     public void setChatId(Long chatId) {
         this.chatId = chatId;
     }
 
     /**
-     * return score
+     *
+     * @return score
      */
     public Integer getScore() {
         return score;
     }
 
     /**
-     * score
+     *
+     * @param score
      */
     public void setScore(Integer score) {
         this.score = score;
     }
 
     /**
-     * return feedback
+     *
+     * @return feedback
      */
     public String getFeedback() {
         return feedback;
     }
 
     /**
-     * feedback
+     *
+     * @param feedback
      */
     public void setFeedback(String feedback) {
         this.feedback = feedback == null ? null : feedback.trim();
     }
 
     /**
-     * return query_text
+     *
+     * @return query_text
      */
     public String getQueryText() {
         return queryText;
     }
 
     /**
-     * queryText
+     *
+     * @param queryText
      */
     public void setQueryText(String queryText) {
         this.queryText = queryText == null ? null : queryText.trim();
     }
 
     /**
-     * return query_response
+     *
+     * @return query_result
      */
     public String getQueryResult() {
         return queryResult;
     }
 
     /**
-     * queryResponse
+     *
+     * @param queryResult
      */
     public void setQueryResult(String queryResult) {
         this.queryResult = queryResult == null ? null : queryResult.trim();
     }
-}
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/ChatQueryDOExample.java

@@ -5,47 +5,92 @@ import java.util.Date;
 import java.util.List;
 
 public class ChatQueryDOExample {
-
+    /**
+     * s2_chat_query
+     */
     protected String orderByClause;
+
+    /**
+     * s2_chat_query
+     */
     protected boolean distinct;
+
+    /**
+     * s2_chat_query
+     */
     protected List<Criteria> oredCriteria;
+
+    /**
+     * s2_chat_query
+     */
     protected Integer limitStart;
+
+    /**
+     * s2_chat_query
+     */
     protected Integer limitEnd;
 
+    /**
+     * @mbg.generated
+     */
     public ChatQueryDOExample() {
         oredCriteria = new ArrayList<Criteria>();
     }
 
-    public String getOrderByClause() {
+    /**
-        return orderByClause;
+     * @mbg.generated
-    }
+     */
-
     public void setOrderByClause(String orderByClause) {
         this.orderByClause = orderByClause;
     }
 
-    public boolean isDistinct() {
+    /**
-        return distinct;
+     * @mbg.generated
+     */
+    public String getOrderByClause() {
+        return orderByClause;
     }
 
+    /**
+     * @mbg.generated
+     */
     public void setDistinct(boolean distinct) {
         this.distinct = distinct;
     }
 
+    /**
+     * @mbg.generated
+     */
+    public boolean isDistinct() {
+        return distinct;
+    }
+
+    /**
+     * @mbg.generated
+     */
     public List<Criteria> getOredCriteria() {
         return oredCriteria;
     }
 
+    /**
+     * @mbg.generated
+     */
     public void or(Criteria criteria) {
         oredCriteria.add(criteria);
     }
 
+    /**
+     * @mbg.generated
+     */
     public Criteria or() {
         Criteria criteria = createCriteriaInternal();
         oredCriteria.add(criteria);
         return criteria;
     }
 
+    /**
+     * @mbg.generated
+     */
     public Criteria createCriteria() {
         Criteria criteria = createCriteriaInternal();
         if (oredCriteria.size() == 0) {
@@ -54,35 +99,55 @@ public class ChatQueryDOExample {
         return criteria;
     }
 
+    /**
+     * @mbg.generated
+     */
     protected Criteria createCriteriaInternal() {
         Criteria criteria = new Criteria();
         return criteria;
     }
 
+    /**
+     * @mbg.generated
+     */
     public void clear() {
         oredCriteria.clear();
         orderByClause = null;
         distinct = false;
     }
 
-    public Integer getLimitStart() {
+    /**
-        return limitStart;
+     * @mbg.generated
-    }
+     */
-
     public void setLimitStart(Integer limitStart) {
         this.limitStart = limitStart;
     }
 
-    public Integer getLimitEnd() {
+    /**
-        return limitEnd;
+     * @mbg.generated
+     */
+    public Integer getLimitStart() {
+        return limitStart;
     }
 
+    /**
+     * @mbg.generated
+     */
     public void setLimitEnd(Integer limitEnd) {
         this.limitEnd = limitEnd;
     }
 
-    protected abstract static class GeneratedCriteria {
+    /**
+     * @mbg.generated
+     */
+    public Integer getLimitEnd() {
+        return limitEnd;
+    }
 
+    /**
+     * s2_chat_query null
+     */
+    protected abstract static class GeneratedCriteria {
         protected List<Criterion> criteria;
 
         protected GeneratedCriteria() {
@@ -183,6 +248,66 @@ public class ChatQueryDOExample {
             return (Criteria) this;
         }
 
+        public Criteria andAgentIdIsNull() {
+            addCriterion("agent_id is null");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdIsNotNull() {
+            addCriterion("agent_id is not null");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdEqualTo(Integer value) {
+            addCriterion("agent_id =", value, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdNotEqualTo(Integer value) {
+            addCriterion("agent_id <>", value, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdGreaterThan(Integer value) {
+            addCriterion("agent_id >", value, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdGreaterThanOrEqualTo(Integer value) {
+            addCriterion("agent_id >=", value, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdLessThan(Integer value) {
+            addCriterion("agent_id <", value, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdLessThanOrEqualTo(Integer value) {
+            addCriterion("agent_id <=", value, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdIn(List<Integer> values) {
+            addCriterion("agent_id in", values, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdNotIn(List<Integer> values) {
+            addCriterion("agent_id not in", values, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdBetween(Integer value1, Integer value2) {
+            addCriterion("agent_id between", value1, value2, "agentId");
+            return (Criteria) this;
+        }
+
+        public Criteria andAgentIdNotBetween(Integer value1, Integer value2) {
+            addCriterion("agent_id not between", value1, value2, "agentId");
+            return (Criteria) this;
+        }
+
         public Criteria andCreateTimeIsNull() {
             addCriterion("create_time is null");
             return (Criteria) this;
@@ -564,6 +689,9 @@ public class ChatQueryDOExample {
         }
     }
 
+    /**
+     * s2_chat_query
+     */
     public static class Criteria extends GeneratedCriteria {
 
         protected Criteria() {
@@ -571,8 +699,10 @@ public class ChatQueryDOExample {
         }
     }
 
+    /**
+     * s2_chat_query null
+     */
     public static class Criterion {
-
         private String condition;
 
         private Object value;
@@ -657,4 +787,4 @@ public class ChatQueryDOExample {
             return typeHandler;
         }
     }
-}
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/CostType.java

@@ -0,0 +1,23 @@
+package com.tencent.supersonic.chat.persistence.dataobject;
+
+public enum CostType {
+    MAPPER(1, "mapper"),
+    PARSER(2, "parser"),
+    QUERY(3, "query");
+
+    private Integer type;
+    private String name;
+
+    CostType(Integer type, String name) {
+        this.type = type;
+        this.name = name;
+    }
+
+    public Integer getType() {
+        return type;
+    }
+
+    public String getName() {
+        return name;
+    }
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/PluginDO.java

@@ -254,4 +254,4 @@ public class PluginDO {
     public void setComment(String comment) {
         this.comment = comment == null ? null : comment.trim();
     }
-}
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/PluginDOExample.java

@@ -892,38 +892,6 @@ public class PluginDOExample {
 
         private String typeHandler;
 
-        public String getCondition() {
-            return condition;
-        }
-
-        public Object getValue() {
-            return value;
-        }
-
-        public Object getSecondValue() {
-            return secondValue;
-        }
-
-        public boolean isNoValue() {
-            return noValue;
-        }
-
-        public boolean isSingleValue() {
-            return singleValue;
-        }
-
-        public boolean isBetweenValue() {
-            return betweenValue;
-        }
-
-        public boolean isListValue() {
-            return listValue;
-        }
-
-        public String getTypeHandler() {
-            return typeHandler;
-        }
-
         protected Criterion(String condition) {
             super();
             this.condition = condition;
@@ -959,5 +927,37 @@ public class PluginDOExample {
         protected Criterion(String condition, Object value, Object secondValue) {
             this(condition, value, secondValue, null);
         }
+
+        public String getCondition() {
+            return condition;
+        }
+
+        public Object getValue() {
+            return value;
+        }
+
+        public Object getSecondValue() {
+            return secondValue;
+        }
+
+        public boolean isNoValue() {
+            return noValue;
+        }
+
+        public boolean isSingleValue() {
+            return singleValue;
+        }
+
+        public boolean isBetweenValue() {
+            return betweenValue;
+        }
+
+        public boolean isListValue() {
+            return listValue;
+        }
+
+        public String getTypeHandler() {
+            return typeHandler;
+        }
     }
-}
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/dataobject/StatisticsDO.java

@@ -0,0 +1,54 @@
+package com.tencent.supersonic.chat.persistence.dataobject;
+
+import lombok.AllArgsConstructor;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import lombok.Builder;
+import lombok.Getter;
+
+import java.util.Date;
+
+@Data
+@Builder
+@NoArgsConstructor
+@Getter
+@AllArgsConstructor
+public class StatisticsDO {
+    /**
+     * questionId
+     */
+    private Long questionId;
+
+    /**
+     * chatId
+     */
+    private Long chatId;
+
+    /**
+     * createTime
+     */
+    private Date createTime;
+
+    /**
+     * queryText
+     */
+    private String queryText;
+
+    /**
+     * userName
+     */
+    private String userName;
+
+
+    /**
+     * interface
+     */
+    private String interfaceName;
+
+    /**
+     * cost
+     */
+    private Integer cost;
+
+    private Integer type;
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/mapper/AgentDOMapper.java

@@ -0,0 +1,71 @@
+package com.tencent.supersonic.chat.persistence.mapper;
+
+import com.tencent.supersonic.chat.persistence.dataobject.AgentDO;
+import com.tencent.supersonic.chat.persistence.dataobject.AgentDOExample;
+import org.apache.ibatis.annotations.Mapper;
+import org.apache.ibatis.annotations.Param;
+
+import java.util.List;
+
+@Mapper
+public interface AgentDOMapper {
+    /**
+     *
+     * @mbg.generated
+     */
+    long countByExample(AgentDOExample example);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    int deleteByPrimaryKey(Integer id);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    int insert(AgentDO record);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    int insertSelective(AgentDO record);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    List<AgentDO> selectByExample(AgentDOExample example);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    AgentDO selectByPrimaryKey(Integer id);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    int updateByExampleSelective(@Param("record") AgentDO record, @Param("example") AgentDOExample example);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    int updateByExample(@Param("record") AgentDO record, @Param("example") AgentDOExample example);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    int updateByPrimaryKeySelective(AgentDO record);
+
+    /**
+     *
+     * @mbg.generated
+     */
+    int updateByPrimaryKey(AgentDO record);
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/mapper/ChatParseMapper.java

@@ -0,0 +1,17 @@
+package com.tencent.supersonic.chat.persistence.mapper;
+
+import com.tencent.supersonic.chat.persistence.dataobject.ChatParseDO;
+import org.apache.ibatis.annotations.Mapper;
+import org.apache.ibatis.annotations.Param;
+
+import java.util.List;
+
+
+@Mapper
+public interface ChatParseMapper {
+
+    boolean batchSaveParseInfo(@Param("list") List<ChatParseDO> list);
+
+    ChatParseDO getParseInfo(Long questionId, String userName, int parseId);
+
+}

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/mapper/ChatQueryDOMapper.java

@@ -14,4 +14,5 @@ public interface ChatQueryDOMapper {
 
     int updateByPrimaryKeyWithBLOBs(ChatQueryDO record);
 
+    Boolean deleteByPrimaryKey(Long questionId);
 }

chat/core/src/main/java/com/tencent/supersonic/chat/persistence/mapper/StatisticsMapper.java

@@ -0,0 +1,12 @@
+package com.tencent.supersonic.chat.persistence.mapper;
+
+import com.tencent.supersonic.chat.persistence.dataobject.StatisticsDO;
+import org.apache.ibatis.annotations.Mapper;
+import org.apache.ibatis.annotations.Param;
+
+import java.util.List;
+
+@Mapper
+public interface StatisticsMapper {
+    boolean batchSaveStatistics(@Param("list") List<StatisticsDO> list);
+}