[improvement][headless-fe] Added the ability to configure embedding models. (#1362)

* [improvement][semantic-fe] Add model alias setting & Add view permission restrictions to the model permission management tab. [improvement][semantic-fe] Add permission control to the action buttons for the main domain; apply high sensitivity filtering to the authorization of metrics/dimensions. [improvement][semantic-fe] Optimize the editing mode in the dimension/metric/datasource components to use the modelId stored in the database for data, instead of relying on the data from the state manager. * [improvement][semantic-fe] Add time granularity setting in the data source configuration. * [improvement][semantic-fe] Dictionary import for dimension values supported in Q&A visibility * [improvement][semantic-fe] Modification of data source creation prompt wording" * [improvement][semantic-fe] metric market experience optimization * [improvement][semantic-fe] enhance the analysis of metric trends * [improvement][semantic-fe] optimize the presentation of metric trend permissions * [improvement][semantic-fe] add metric trend download functionality * [improvement][semantic-fe] fix the dimension initialization issue in metric correlation * [improvement][semantic-fe] Fix the issue of database changes not taking effect when creating based on an SQL data source. * [improvement][semantic-fe] Optimizing pagination logic and some CSS styles * [improvement][semantic-fe] Fixing the API for the indicator list by changing "current" to "pageNum" * [improvement][semantic-fe] Fixing the default value setting for the indicator list * [improvement][semantic-fe] Adding batch operations for indicators/dimensions/models * [improvement][semantic-fe] Replacing the single status update API for indicators/dimensions with a batch update API * [improvement][semantic-fe] Redesigning the indicator homepage to incorporate trend charts and table functionality for indicators * [improvement][semantic-fe] Optimizing the logic for setting dimension values and editing data sources, and adding system settings functionality * [improvement][semantic-fe] Upgrading antd version to 5.x, extracting the batch operation button component, optimizing the interaction for system settings, and expanding the configuration generation types for list-to-select component. * [improvement][semantic-fe] Adding the ability to filter dimensions based on whether they are tags or not. * [improvement][semantic-fe] Adding the ability to edit relationships between models in the canvas. * [improvement][semantic-fe] Updating the datePicker component to use dayjs instead. * [improvement][semantic-fe] Fixing the issue with passing the model ID for dimensions in the indicator market. * [improvement][semantic-fe] Fixing the abnormal state of the popup when creating a model. * [improvement][semantic-fe] Adding permission logic for bulk operations in the indicator market. * [improvement][semantic-fe] Adding the ability to download and transpose data. * [improvement][semantic-fe] Fixing the initialization issue with the date selection component in the indicator details page when switching time granularity. * [improvement][semantic-fe] Fixing the logic error in the dimension value setting. * [improvement][semantic-fe] Fixing the synchronization issue with the question and answer settings information. * [improvement][semantic-fe] Optimizing the canvas functionality for better performance and user experience. * [improvement][semantic-fe] Optimizing the update process for drawing model relationship edges in the canvas. * [improvement][semantic-fe] Changing the line type for canvas connections. * [improvement][semantic-fe] Replacing the initialization variable from "semantic" to "headless". * [improvement][semantic-fe] Fixing the missing migration issue for default drill-down dimension configuration in model editing. Additionally, optimizing the data retrieval method for initializing fields in the model. * [improvement][semantic-fe] Updating the logic for the fieldName. * [improvement][semantic-fe] Adjusting the position of the metrics tab. * [improvement][semantic-fe] Changing the 字段名称 to 英文名称. * [improvement][semantic-fe] Fix metric measurement deletion. * [improvement][semantic-fe] UI optimization for metric details page. * [improvement][semantic-fe] UI optimization for metric details page. * [improvement][semantic-fe] UI adjustment for metric details page. * [improvement][semantic-fe] The granularity field in the time type of model editing now supports setting it as empty. * [improvement][semantic-fe] Added field type and metric type to the metric creation options. * [improvement][semantic-fe] The organization structure selection feature has been added to the permission management. * [improvement][semantic-fe] Improved user experience for the metric list. * [improvement][semantic-fe] fix update the metric list. * [improvement][headless-fe] Added view management functionality. * [improvement][headless-fe] The view management functionality has been added. This feature allows users to create, edit, and manage different views within the system. * [improvement][headless-fe] Added model editing side effect detection. * [improvement][headless-fe] Fixed the logic error in view editing. * [improvement][headless-fe] Fixed the issue with initializing dimension associations in metric settings. * [improvement][headless-fe] Added the ability to hide the Q&A settings entry point. * [improvement][headless-fe] Fixed the issue with selecting search results in metric field creation. * [improvement][headless-fe] Added search functionality to the field list in model editing. * [improvement][headless-fe] fix the field list in model editing * [improvement][headless-fe] Restructured the data for the dimension value settings interface. * [improvement][headless-fe] Added dynamic variable functionality to model creation based on SQL scripts. * [improvement][headless-fe] Added support for passing dynamic variables as parameters in the executeSql function. * [improvement][headless-fe] Resolved the issue where users were unable to select all options for dimensions, metrics, and fields in the metric generation process. * [improvement][headless-fe] Replaced the term "view" with "dataset" * [improvement][headless-fe] Added the ability to export metrics and dimensions to a specific target. * [improvement][headless-fe] Enhanced dataset creation to support the tag mode. * [improvement][headless-fe] Added tag value setting. * [improvement][headless-fe] Optimized the tag setting system. * [improvement][headless-fe] Optimized the tag setting system. * [improvement][headless-fe] Updated the data initialization for model editing to use API requests instead. * [improvement][headless-fe] Added search functionality to model management. * [improvement][headless-fe] Removed field null validation during model editing. * [improvement][headless-fe] Updated the batch operation button component. * [improvement][headless-fe] Optimized the logic for initializing indicators in dimension value settings. * [improvement][headless-fe] Adjusted the length of the input field for model editing names. * [improvement][headless-fe] Lock the version of the @ant-design/pro-table component and replace it with @ant-design/pro-components. * [improvement][headless-fe] Optimized the style of the metrics market and tags market. * [improvement][headless-fe] The quick creation of model fields now defaults to using the "comment" field for filling. * [improvement][headless-fe] The quick creation of model fields now defaults to using the "comment" field for filling * [improvement][headless-fe] The quick creation of model fields now defaults to using the "comment" field for filling. * [improvement][headless-fe] Fixed the issue where the conditions for metric measurement creation were not being saved correctly. * [improvement][headless-fe] Default value setting for hiding dimensions. * [improvement][headless-fe] Updated the file imports in the project. * [improvement][headless-fe] Adjusted the logic for displaying the tab in the theme domain. * [improvement][headless-fe] Added term management functionality. * [improvement][headless-fe] When creating a model, the current metric operator now allows for clearance. * [improvement][headless-fe] Term management interface transformation * [improvement][headless-fe] Migrating scaffold version to @umi/max * [improvement][headless-fe] remove modle column * [improvement][headless-fe] 1.Added configuration for the large language model in the agent; 2.upgraded React version from 17 to 18; 3.modified some UI effects. * [improvement][headless-fe] Added a simplified mode to the question-answering system. * [improvement][headless-fe] remove pnpm-lock * [improvement][headless-fe] add pnpm-lock * [improvement][headless-fe] Fixed the issue with passing the modelId during initialization. * [improvement][headless-fe] Fixed the issue with abnormal comments during model creation. * [improvement][headless-fe] fix build bugs * [improvement][headless-fe] change build config * [improvement][headless-fe] route config change * [improvement][headless-fe] Optimized data updating when switching domains. * [improvement][headless-fe] css change * [improvement][semantic-fe] logo css change * [improvement][semantic-fe] system config add defaultValue support * [improvement][semantic-fe] tag mode wording change * [improvement][semantic-fe] fix metric edit page init error * [improvement][supersonic-fe] Updated the way chat projects are imported in supersonic-fe. * [improvement][chat-engine] Added a background silent mode for watching chat projects. * [improvement][supersonic-fe] fix proxy url * [improvement][headless-fe] agent setting update * [improvement][headless-fe] Agent configuration: Added connectivity testing for large models. * [improvement][headless-fe] Chat: Enabled integration with agent configuration. * [improvement][headless-fe] SQL formatter: Updated the import method. * [improvement][headless-fe] login fixed * [improvement][headless-fe] Agent: Optimized the logic for saving. * [improvement][headless-fe] Model management: Integrated into the main theme domain. * [improvement][headless-fe] Sensitivity: Added bulk modification functionality. * [improvement][headless-fe] wording change * [improvement][headless-fe] Prompt configuration: Added the ability to configure prompts. * [improvement][headless-fe] Added the ability to configure embedding models.
2025-12-10 11:07:06 +00:00 · 2024-07-06 23:29:33 +08:00
parent 6db6aaf98d
commit 609146d12c
7 changed files with 243 additions and 25 deletions
--- a/webapp/packages/chat-sdk/src/Chat/type.ts
+++ b/webapp/packages/chat-sdk/src/Chat/type.ts
@@ -90,7 +90,8 @@ export type AgentType = {
  updatedAt: string;
  enableSearch: number;
  agentConfig: string;
-  llmConfig: {
+  modelConfig: {
+    chatModel: {
      provider: string;
      baseUrl: string;
      apiKey: string;
@@ -98,6 +99,19 @@ export type AgentType = {
      temperature: number;
      timeOut: number;
    };
+    embeddingModel: {
+      apiKey: string;
+      baseUrl: string;
+      // logRequests: true,
+      // logResponses: true,
+      // maxRetries: number,
+      // maxToken: number,
+      modelName: string;
+      modelPath: string;
+      provider: string;
+      vocabularyPath: string;
+    };
+  };
  multiTurnConfig: {
    enableMultiTurn: boolean;
  };
--- a/webapp/packages/supersonic-fe/src/components/MainTitleMark/index.tsx
+++ b/webapp/packages/supersonic-fe/src/components/MainTitleMark/index.tsx
@@ -0,0 +1,15 @@
+import styles from './style.less';
+
+type Props = {};
+
+const MainTitleMark: React.FC<Props> = ({}) => {
+  return (
+    <div className={styles.mainTitleMark}>
+      <i className={styles.mark} />
+      <i className={styles.mark} />
+      <i className={styles.mark} />
+    </div>
+  );
+};
+
+export default MainTitleMark;
--- a/webapp/packages/supersonic-fe/src/components/MainTitleMark/style.less
+++ b/webapp/packages/supersonic-fe/src/components/MainTitleMark/style.less
@@ -0,0 +1,25 @@
+.mainTitleMark {
+  margin-right: 24px;
+  margin-left: 8px;
+
+  .mark {
+    display: inline-block;
+    width: 4px;
+    height: 11px;
+    background: var(--tme-primary-color);
+    border-radius: 4px;
+    transform: rotate(15deg);
+
+    &:not(:last-of-type) {
+      margin-right: 4px;
+    }
+
+    &:nth-of-type(2) {
+      opacity: 0.6;
+    }
+
+    &:nth-of-type(3) {
+      opacity: 0.2;
+    }
+  }
+}
--- a/webapp/packages/supersonic-fe/src/pages/Agent/AgentForm.tsx
+++ b/webapp/packages/supersonic-fe/src/pages/Agent/AgentForm.tsx
@@ -12,6 +12,7 @@ import {
  message,
  Space,
 } from 'antd';
+import MainTitleMark from '@/components/MainTitleMark';
 import { AgentType } from './type';
 import { useEffect, useState } from 'react';
 import styles from './style.less';
@@ -43,11 +44,20 @@ const AgentForm: React.FC<Props> = ({ editAgent, onSaveAgent, onCreateToolBtnCli
  const [llmTestLoading, setLlmTestLoading] = useState<boolean>(false);
  const [formData, setFormData] = useState<any>({
    enableSearch: true,
-    llmConfig: {
+    modelConfig: {
+      chatModel: {
        timeOut: 60,
        provider: 'OPEN_AI',
        temperature: 0,
      },
+      embeddingModel: {
+        provider: 'OPEN_AI',
+      },
+    },
+    embeddingStore: {
+      provider: 'MILVUS',
+      timeOut: 60,
+    },
    agentConfig: {
      ...defaultAgentConfig,
    },
@@ -57,8 +67,8 @@ const AgentForm: React.FC<Props> = ({ editAgent, onSaveAgent, onCreateToolBtnCli
  useEffect(() => {
    if (editAgent) {
      const sourceData = { ...editAgent };
-      if (!sourceData.llmConfig) {
-        delete sourceData.llmConfig;
+      if (!sourceData.modelConfig) {
+        delete sourceData.modelConfig;
      }

      const config = jsonParse(editAgent.agentConfig, {});
@@ -217,10 +227,15 @@ const AgentForm: React.FC<Props> = ({ editAgent, onSaveAgent, onCreateToolBtnCli
    },
    {
      label: '大模型配置',
-      key: 'llmConfig',
+      key: 'modelConfig',
      children: (
        <div className={styles.agentFormContainer}>
-          <FormItem name={['llmConfig', 'provider']} label="接口协议">
+          <div className={styles.agentFormTitle}>
+            <Space>
+              语言模型 <MainTitleMark />
+            </Space>
+          </div>
+          <FormItem name={['modelConfig', 'chatModel', 'provider']} label="接口协议">
            <Select placeholder="">
              {['OPEN_AI', 'OLLAMA'].map((item) => (
                <Select.Option key={item} value={item}>
@@ -229,16 +244,16 @@ const AgentForm: React.FC<Props> = ({ editAgent, onSaveAgent, onCreateToolBtnCli
              ))}
            </Select>
          </FormItem>
-          <FormItem name={['llmConfig', 'modelName']} label="Model Name">
-            <Input placeholder="请输入大模型名称" />
+          <FormItem name={['modelConfig', 'chatModel', 'modelName']} label="Model Name">
+            <Input placeholder="请输入语言模型名称" />
          </FormItem>
-          <FormItem name={['llmConfig', 'baseUrl']} label="Base URL">
+          <FormItem name={['modelConfig', 'chatModel', 'baseUrl']} label="Base URL">
            <Input placeholder="请输入Base URL" />
          </FormItem>
          <FormItem
-            name={['llmConfig', 'apiKey']}
+            name={['modelConfig', 'chatModel', 'apiKey']}
            label="API Key"
-            hidden={formData?.llmConfig?.provider === 'OLLAMA'}
+            hidden={formData?.modelConfig?.chatModel?.provider === 'OLLAMA'}
            getValueFromEvent={(event) => {
              const value = event.target.value;
              return encryptPassword(value);
@@ -252,7 +267,7 @@ const AgentForm: React.FC<Props> = ({ editAgent, onSaveAgent, onCreateToolBtnCli
            <Input.Password placeholder="请输入API Key" visibilityToggle />
          </FormItem>

-          <FormItem name={['llmConfig', 'temperature']} label="Temperature">
+          <FormItem name={['modelConfig', 'chatModel', 'temperature']} label="Temperature">
            <Slider
              min={0}
              max={1}
@@ -263,9 +278,136 @@ const AgentForm: React.FC<Props> = ({ editAgent, onSaveAgent, onCreateToolBtnCli
              }}
            />
          </FormItem>
-          <FormItem name={['llmConfig', 'timeOut']} label="超时时间(秒)">
+          <FormItem name={['modelConfig', 'chatModel', 'timeOut']} label="超时时间(秒)">
            <InputNumber />
          </FormItem>
+
+          <div className={styles.agentFormTitle}>
+            <Space>
+              向量模型 <MainTitleMark />
+            </Space>
+          </div>
+
+          <FormItem name={['modelConfig', 'embeddingModel', 'provider']} label="接口协议">
+            <Select placeholder="">
+              {[
+                'OPEN_AI',
+                'OLLAMA',
+                'LOCAL_AI',
+                'IN_MEMORY',
+                'ZHIPU',
+                'AZURE',
+                'QIANFAN',
+                'DASHSCOPE',
+              ].map((item) => (
+                <Select.Option key={item} value={item}>
+                  {item}
+                </Select.Option>
+              ))}
+            </Select>
+          </FormItem>
+          <FormItem name={['modelConfig', 'embeddingModel', 'modelName']} label="Model Name">
+            <Input placeholder="请输入向量模型名称" />
+          </FormItem>
+          {formData?.modelConfig?.embeddingModel?.provider === 'IN_MEMORY' ? (
+            <>
+              <FormItem name={['modelConfig', 'embeddingModel', 'modelPath']} label="模型路径">
+                <Input placeholder="请输入模型路径" />
+              </FormItem>
+              <FormItem
+                name={['modelConfig', 'embeddingModel', 'vocabularyPath']}
+                label="词汇表路径"
+              >
+                <Input placeholder="请输入模型路径" />
+              </FormItem>
+            </>
+          ) : (
+            <>
+              <FormItem name={['modelConfig', 'embeddingModel', 'baseUrl']} label="Base URL">
+                <Input placeholder="请输入Base URL" />
+              </FormItem>
+              <FormItem
+                name={['modelConfig', 'embeddingModel', 'apiKey']}
+                label="API Key"
+                getValueFromEvent={(event) => {
+                  const value = event.target.value;
+                  return encryptPassword(value);
+                }}
+                getValueProps={(value) => {
+                  return {
+                    value: value ? decryptPassword(value) : '',
+                  };
+                }}
+              >
+                <Input.Password placeholder="请输入API Key" visibilityToggle />
+              </FormItem>
+            </>
+          )}
+        </div>
+      ),
+    },
+    {
+      label: '向量库配置',
+      key: 'embeddingStore',
+      children: (
+        <div className={styles.agentFormContainer}>
+          <FormItem name={['embeddingStore', 'provider']} label="接口协议">
+            <Select placeholder="">
+              {['MILVUS', 'CHROMA', 'IN_MEMORY'].map((item) => (
+                <Select.Option key={item} value={item}>
+                  {item}
+                </Select.Option>
+              ))}
+            </Select>
+          </FormItem>
+          {formData?.embeddingStore?.provider === 'IN_MEMORY' ? (
+            <>
+              <FormItem name={['embeddingStore', 'persistPath']} label="持久化路径">
+                <Input placeholder="请输入持久化路径" />
+              </FormItem>
+            </>
+          ) : (
+            <>
+              <FormItem name={['embeddingStore', 'baseUrl']} label="Base URL">
+                <Input placeholder="请输入Base URL" />
+              </FormItem>
+              <FormItem
+                name={['embeddingStore', 'apiKey']}
+                label="API Key"
+                getValueFromEvent={(event) => {
+                  const value = event.target.value;
+                  return encryptPassword(value);
+                }}
+                getValueProps={(value) => {
+                  return {
+                    value: value ? decryptPassword(value) : '',
+                  };
+                }}
+              >
+                <Input.Password placeholder="请输入API Key" visibilityToggle />
+              </FormItem>
+              <FormItem name={['embeddingStore', 'timeOut']} label="超时时间(秒)">
+                <InputNumber />
+              </FormItem>
+            </>
+          )}
+        </div>
+      ),
+    },
+    {
+      label: '提示词配置',
+      key: 'promptConfig',
+      children: (
+        <div className={styles.agentFormContainer}>
+          <FormItem name={['promptConfig', 'promptTemplate']} label="提示词模板">
+            <Input.TextArea
+              style={{ minHeight: 600 }}
+              placeholder=" &nbsp;自定义提示词模板可嵌入以下变量，将由系统自动进行替换：&#13;&#10;
+                    -&nbsp;{{exemplar}} &nbsp;:替换成few-shot示例，示例个数由系统配置&#13;&#10;
+                    -&nbsp;{{question}} &nbsp;:替换成用户问题，拼接了一定的补充信息&#13;&#10;
+                    -&nbsp;{{schema}} &nbsp;:替换成数据语义信息，根据用户问题映射而来"
+            />
+          </FormItem>
        </div>
      ),
    },
@@ -332,12 +474,12 @@ const AgentForm: React.FC<Props> = ({ editAgent, onSaveAgent, onCreateToolBtnCli
                <PlusOutlined /> 新增工具
              </Button>
            )}
-            {activeKey === 'llmConfig' && (
+            {activeKey === 'modelConfig' && (
              <Button
                type="primary"
                loading={llmTestLoading}
                onClick={() => {
-                  testLLMConnect(formData.llmConfig);
+                  testLLMConnect(formData.modelConfig.chatModel);
                }}
              >
                大模型连接测试
--- a/webapp/packages/supersonic-fe/src/pages/Agent/style.less
+++ b/webapp/packages/supersonic-fe/src/pages/Agent/style.less
@@ -19,6 +19,12 @@
  padding: 20px 0;
 }

+.agentFormTitle{
+  font-size: 16px;
+  margin-bottom: 20px;
+  font-weight: 500;
+}
+
 .content {
  // margin-top: 20px;
  .searchBar {
--- a/webapp/packages/supersonic-fe/src/pages/Agent/type.ts
+++ b/webapp/packages/supersonic-fe/src/pages/Agent/type.ts
@@ -65,6 +65,19 @@ export type LlmConfigType = {
  timeOut: number;
 };

+export type EmbeddingModelConfigType = {
+  apiKey: string;
+  baseUrl: string;
+  // logRequests: true,
+  // logResponses: true,
+  // maxRetries: number,
+  // maxToken: number,
+  modelName: string;
+  modelPath: string;
+  provider: string;
+  vocabularyPath: string;
+};
+
 export type MultiTurnConfig = {
  enableMultiTurn: boolean;
 };
@@ -84,7 +97,10 @@ export type AgentType = {
  status?: 0 | 1;
  enableSearch?: 0 | 1;
  agentConfig?: AgentConfigType;
-  llmConfig?: LlmConfigType;
+  modelConfig: {
+    chatModel: LlmConfigType;
+    embeddingModel: EmbeddingModelConfigType;
+  };
  multiTurnConfig?: MultiTurnConfig;
  visualConfig?: VisualConfig;
 };
--- a/webapp/packages/supersonic-fe/src/utils/utils.ts
+++ b/webapp/packages/supersonic-fe/src/utils/utils.ts
@@ -476,7 +476,7 @@ const encryptKey = CryptoJS.enc.Hex.parse(
  '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08',
 );

-export const encryptPassword = (password: string, key?: any) => {
+export const encryptPassword = (password: string, key?: string) => {
  if (!password) {
    return password;
  }