[improvement][headless-fe] Added a configuration interface for large language models (#1752)

* [improvement][semantic-fe] Optimizing the logic for setting dimension values and editing data sources, and adding system settings functionality

* [improvement][semantic-fe] Upgrading antd version to 5.x, extracting the batch operation button component, optimizing the interaction for system settings, and expanding the configuration generation types for list-to-select component.

* [improvement][semantic-fe] Adding the ability to filter dimensions based on whether they are tags or not.

* [improvement][semantic-fe] Adding the ability to edit relationships between models in the canvas.

* [improvement][semantic-fe] Updating the datePicker component to use dayjs instead.

* [improvement][semantic-fe] Fixing the issue with passing the model ID for dimensions in the indicator market.

* [improvement][semantic-fe] Fixing the abnormal state of the popup when creating a model.

* [improvement][semantic-fe] Adding permission logic for bulk operations in the indicator market.

* [improvement][semantic-fe] Adding the ability to download and transpose data.

* [improvement][semantic-fe] Fixing the initialization issue with the date selection component in the indicator details page when switching time granularity.

* [improvement][semantic-fe] Fixing the logic error in the dimension value setting.

* [improvement][semantic-fe] Fixing the synchronization issue with the question and answer settings information.

* [improvement][semantic-fe] Optimizing the canvas functionality for better performance and user experience.

* [improvement][semantic-fe] Optimizing the update process for drawing model relationship edges in the canvas.

* [improvement][semantic-fe] Changing the line type for canvas connections.

* [improvement][semantic-fe] Replacing the initialization variable from "semantic" to "headless".

* [improvement][semantic-fe] Fixing the missing migration issue for default drill-down dimension configuration in model editing. Additionally, optimizing the data retrieval method for initializing fields in the model.

* [improvement][semantic-fe] Updating the logic for the fieldName.

* [improvement][semantic-fe] Adjusting the position of the metrics tab.

* [improvement][semantic-fe] Changing the 字段名称 to 英文名称.

* [improvement][semantic-fe] Fix metric measurement deletion.

* [improvement][semantic-fe] UI optimization for metric details page.

* [improvement][semantic-fe] UI optimization for metric details page.

* [improvement][semantic-fe] UI adjustment for metric details page.

* [improvement][semantic-fe] The granularity field in the time type of model editing now supports setting it as empty.

* [improvement][semantic-fe] Added field type and metric type to the metric creation options.

* [improvement][semantic-fe] The organization structure selection feature has been added to the permission management.

* [improvement][semantic-fe] Improved user experience for the metric list.

* [improvement][semantic-fe] fix update the metric list.

* [improvement][headless-fe] Added view management functionality.

* [improvement][headless-fe] The view management functionality has been added. This feature allows users to create, edit, and manage different views within the system.

* [improvement][headless-fe] Added model editing side effect detection.

* [improvement][headless-fe] Fixed the logic error in view editing.

* [improvement][headless-fe] Fixed the issue with initializing dimension associations in metric settings.

* [improvement][headless-fe] Added the ability to hide the Q&A settings entry point.

* [improvement][headless-fe] Fixed the issue with selecting search results in metric field creation.

* [improvement][headless-fe] Added search functionality to the field list in model editing.

* [improvement][headless-fe] fix the field list in model editing

* [improvement][headless-fe] Restructured the data for the dimension value settings interface.

* [improvement][headless-fe] Added dynamic variable functionality to model creation based on SQL scripts.

* [improvement][headless-fe] Added support for passing dynamic variables as parameters in the executeSql function.

* [improvement][headless-fe] Resolved the issue where users were unable to select all options for dimensions, metrics, and fields in the metric generation process.

* [improvement][headless-fe] Replaced the term "view" with "dataset"

* [improvement][headless-fe] Added the ability to export metrics and dimensions to a specific target.

* [improvement][headless-fe] Enhanced dataset creation to support the tag mode.

* [improvement][headless-fe] Added tag value setting.

* [improvement][headless-fe] Optimized the tag setting system.

* [improvement][headless-fe] Optimized the tag setting system.

* [improvement][headless-fe] Updated the data initialization for model editing to use API requests instead.

* [improvement][headless-fe] Added search functionality to model management.

* [improvement][headless-fe] Removed field null validation during model editing.

* [improvement][headless-fe] Updated the batch operation button component.

* [improvement][headless-fe] Optimized the logic for initializing indicators in dimension value settings.

* [improvement][headless-fe] Adjusted the length of the input field for model editing names.

* [improvement][headless-fe]  Lock the version of the @ant-design/pro-table component and replace it with @ant-design/pro-components.

* [improvement][headless-fe] Optimized the style of the metrics market and tags market.

* [improvement][headless-fe] The quick creation of model fields now defaults to using the "comment" field for filling.

* [improvement][headless-fe] The quick creation of model fields now defaults to using the "comment" field for filling

* [improvement][headless-fe] The quick creation of model fields now defaults to using the "comment" field for filling.

* [improvement][headless-fe] Fixed the issue where the conditions for metric measurement creation were not being saved correctly.

* [improvement][headless-fe] Default value setting for hiding dimensions.

* [improvement][headless-fe] Updated the file imports in the project.

* [improvement][headless-fe] Adjusted the logic for displaying the tab in the theme domain.

* [improvement][headless-fe] Added term management functionality.

* [improvement][headless-fe] When creating a model, the current metric operator now allows for clearance.

* [improvement][headless-fe] Term management interface transformation

* [improvement][headless-fe] Migrating scaffold version to @umi/max

* [improvement][headless-fe] remove modle column

* [improvement][headless-fe] 1.Added configuration for the large language model in the agent; 2.upgraded React version from 17 to 18; 3.modified some UI effects.

* [improvement][headless-fe] Added a simplified mode to the question-answering system.

* [improvement][headless-fe] remove pnpm-lock

* [improvement][headless-fe] add pnpm-lock

* [improvement][headless-fe] Fixed the issue with passing the modelId during initialization.

* [improvement][headless-fe] Fixed the issue with abnormal comments during model creation.

* [improvement][headless-fe] fix  build bugs

* [improvement][headless-fe]  change build config

* [improvement][headless-fe] route config change

* [improvement][headless-fe] Optimized data updating when switching domains.

* [improvement][headless-fe] css change

* [improvement][semantic-fe] logo css change

* [improvement][semantic-fe] system config add defaultValue support

* [improvement][semantic-fe] tag mode wording change

* [improvement][semantic-fe] fix metric edit page init error

* [improvement][supersonic-fe] Updated the way chat projects are imported in supersonic-fe.

* [improvement][chat-engine] Added a background silent mode for watching chat projects.

* [improvement][supersonic-fe] fix proxy url

* [improvement][headless-fe] agent setting update

* [improvement][headless-fe] Agent configuration: Added connectivity testing for large models.

* [improvement][headless-fe] Chat: Enabled integration with agent configuration.

* [improvement][headless-fe] SQL formatter: Updated the import method.

* [improvement][headless-fe] login fixed

* [improvement][headless-fe] Agent: Optimized the logic for saving.

* [improvement][headless-fe] Model management: Integrated into the main theme domain.

* [improvement][headless-fe] Sensitivity: Added bulk modification functionality.

* [improvement][headless-fe] wording change

* [improvement][headless-fe] Prompt configuration: Added the ability to configure prompts.

* [improvement][headless-fe] Added the ability to configure embedding models.

* [improvement][headless-fe] hidden configure embedding models.

* [improvement][headless-fe] Connection test prompt update for large language model settings.

* [improvement][headless-fe]  add memory review config

* [improvement][headless-fe] Rollback of data structure for large language model configuration.

* [improvement][headless-fe] Added dependency relationships between various items in system configuration.

* [improvement][headless-fe] Added password parsing to the generation rules of system settings.

* [improvement][headless-fe] Added slider parsing to the generation rules of system settings.

* [improvement][headless-fe] Optimized the logic for initializing values in system settings.

* [improvement][headless-fe] Optimized the time format

* [improvement][headless-fe] Unified the SQL generation method for data sets to ensure consistency and improve efficiency.

* [improvement][headless-fe] Added support for data sets with non-partitioned time fields

* [improvement][headless-fe] Added support for editing time formats in dimension settings

* [improvement][headless-fe] Rolled back the time parameter in the metric details page to its previous state

* [improvement][headless-fe] Fixed the issue with hidden field validation when editing dimensions

* [improvement][headless-fe] Added a connectivity test for large models in the system settings

* [improvement][headless-fe] Changed the METRIC type in ChatContextType queryType to AGGREGATE

* [improvement][headless-fe] Added query and batch delete operations to the terminology management feature

* [improvement][headless-fe] Enhanced the memory management feature by adding sorting functionality and other optimization features.

* [improvement][headless-fe] Resolved the compatibility issue with the backend service where the updatedAt and other time fields were not being properly converted to the updated_at format for sorting purposes.

* [improvement][headless-fe] Added a configuration interface for large language models

---------

Co-authored-by: tristanliu <tristanliu@tencent.com>
This commit is contained in:
tristanliu
2024-10-09 11:50:23 +08:00
committed by GitHub
parent 0654a54c8d
commit df13e4363d
9 changed files with 448 additions and 4 deletions

View File

@@ -1,7 +1,8 @@
export default {
dev: {
'/api/': {
target: 'http://localhost:9080',
target: 'http://10.91.217.39:9080',
// target: 'http://s2.tmeoa.com',
changeOrigin: true,
},
},

View File

@@ -152,10 +152,15 @@ const ROUTES = [
{
path: '/database',
name: 'database',
// hideInMenu: true,
component: './SemanticModel/components/Database/DatabaseTable',
envEnableList: [ENV_KEY.SEMANTIC],
},
{
path: '/llm',
name: 'llm',
component: './SemanticModel/components/LLM/LlmTable',
envEnableList: [ENV_KEY.SEMANTIC],
},
{
path: '/system',
name: 'system',

View File

@@ -14,6 +14,7 @@ export default {
'menu.tag': '标签市场',
'menu.tag.tagDetail': '标签详情页',
'menu.database': '数据库管理',
'menu.llm': '大模型管理',
'menu.chatSetting': '问答设置',
'menu.plugin': '插件管理',
'menu.login': '登录',

View File

@@ -1,5 +1,5 @@
import React, { useEffect, useState } from 'react';
import { Button, Modal, message, Space, Tooltip, Tabs } from 'antd';
import { Button, Modal, message, Space, Tooltip, Tabs, Drawer } from 'antd';
import { InfoCircleOutlined } from '@ant-design/icons';
import { ISemantic } from '../data';
import CommonEditTable from './CommonEditTable';

View File

@@ -0,0 +1,167 @@
import { useEffect, forwardRef, useImperativeHandle, useState } from 'react';
import type { ForwardRefRenderFunction } from 'react';
import { message, Form, Input, Select, Button, Space, Slider, InputNumber } from 'antd';
import { saveLlmConfig } from '../../service';
import { testLLMConn } from '@/services/system';
import { formLayout } from '@/components/FormHelper/utils';
import { encryptPassword, decryptPassword } from '@/utils/utils';
import styles from '../style.less';
type Props = {
llmItem?: any;
onSubmit?: (params?: any) => void;
};
const FormItem = Form.Item;
const TextArea = Input.TextArea;
const DatabaseCreateForm: ForwardRefRenderFunction<any, Props> = ({ llmItem, onSubmit }, ref) => {
const [form] = Form.useForm();
const [formData, setFormData] = useState<any>({
config: {
timeOut: 60,
provider: 'OPEN_AI',
temperature: 0,
},
});
const [testLoading, setTestLoading] = useState<boolean>(false);
useEffect(() => {
form.resetFields();
if (llmItem) {
form.setFieldsValue({ ...llmItem });
}
}, [llmItem]);
const getFormValidateFields = async () => {
return await form.validateFields();
};
useImperativeHandle(ref, () => ({
getFormValidateFields,
saveDatabaseConfig,
testDatabaseConnection,
}));
const saveDatabaseConfig = async () => {
const values = await form.validateFields();
const { code, msg } = await saveLlmConfig({
...(llmItem || {}),
...values,
});
if (code === 200) {
message.success('保存成功');
onSubmit?.();
return;
}
message.error(msg);
};
const testDatabaseConnection = async () => {
const values = await form.validateFields();
setTestLoading(true);
const { code, data } = await testLLMConn({
...values.config,
});
setTestLoading(false);
if (code === 200 && data) {
message.success('连接测试通过');
return;
}
message.error('连接测试失败');
};
return (
<>
<Form
{...formLayout}
form={form}
layout="vertical"
className={styles.form}
initialValues={formData}
>
<FormItem
name="name"
label="连接名称"
rules={[{ required: true, message: '请输入连接名称' }]}
>
<Input placeholder="请输入连接名称" />
</FormItem>
<FormItem name={['config', 'provider']} label="接口协议">
<Select placeholder="">
{['OPEN_AI', 'OLLAMA'].map((item) => (
<Select.Option key={item} value={item}>
{item}
</Select.Option>
))}
</Select>
</FormItem>
<FormItem name={['config', 'modelName']} label="Model Name">
<Input placeholder="请输入语言模型名称" />
</FormItem>
<FormItem name={['config', 'baseUrl']} label="Base URL">
<Input placeholder="请输入Base URL" />
</FormItem>
<FormItem
name={['config', 'apiKey']}
label="API Key"
hidden={formData?.config?.provider === 'OLLAMA'}
getValueFromEvent={(event) => {
const value = event.target.value;
return encryptPassword(value);
}}
getValueProps={(value) => {
return {
value: value ? decryptPassword(value) : '',
};
}}
>
<Input.Password placeholder="请输入API Key" visibilityToggle />
</FormItem>
<FormItem name={['config', 'temperature']} label="Temperature">
<Slider
min={0}
max={1}
step={0.1}
marks={{
0: '精准',
1: '随机',
}}
/>
</FormItem>
<FormItem name={['config', 'timeOut']} label="超时时间(秒)">
<InputNumber />
</FormItem>
<FormItem name="description" label="描述">
<TextArea placeholder="请输入大模型连接描述" style={{ height: 100 }} />
</FormItem>
<FormItem>
<Space>
<Button
type="primary"
loading={testLoading}
onClick={() => {
testDatabaseConnection();
}}
>
</Button>
<Button
type="primary"
onClick={() => {
saveDatabaseConfig();
}}
>
</Button>
</Space>
</FormItem>
</Form>
</>
);
};
export default forwardRef(DatabaseCreateForm);

View File

@@ -0,0 +1,73 @@
import React, { useState, useRef } from 'react';
import { Button, Modal, Space } from 'antd';
import LlmCreateForm from './LlmCreateForm';
import { ISemantic } from '../../data';
export type CreateFormProps = {
onCancel: () => void;
llmItem?: ISemantic.IDatabaseItem;
open: boolean;
onSubmit: (values?: any) => void;
};
const DatabaseSettingModal: React.FC<CreateFormProps> = ({ onCancel, llmItem, open, onSubmit }) => {
const [testLoading, setTestLoading] = useState<boolean>(false);
const createFormRef = useRef<any>({});
const handleTestConnection = async () => {
setTestLoading(true);
await createFormRef.current.testDatabaseConnection();
setTestLoading(false);
};
const renderFooter = () => {
return (
<>
<Space>
<Button
type="primary"
loading={testLoading}
onClick={() => {
handleTestConnection();
}}
>
</Button>
<Button
type="primary"
onClick={() => {
createFormRef.current.saveDatabaseConfig();
}}
>
</Button>
</Space>
</>
);
};
return (
<Modal
width={800}
destroyOnClose
title="大模型设置"
style={{ top: 48 }}
maskClosable={false}
open={open}
footer={renderFooter()}
onCancel={onCancel}
>
<LlmCreateForm
ref={createFormRef}
llmItem={llmItem}
onSubmit={() => {
onSubmit?.();
}}
/>
</Modal>
);
};
export default DatabaseSettingModal;

View File

@@ -0,0 +1,165 @@
import type { ActionType, ProColumns } from '@ant-design/pro-components';
import { ProTable } from '@ant-design/pro-components';
import { message, Button, Space, Popconfirm } from 'antd';
import React, { useRef, useState, useEffect } from 'react';
import LlmSettingModal from './LlmSettingModal';
import { ISemantic } from '../../data';
import { getLlmList, deleteLlmConfig } from '../../service';
import dayjs from 'dayjs';
type Props = {};
const LlmTable: React.FC<Props> = ({}) => {
const [createModalVisible, setCreateModalVisible] = useState<boolean>(false);
const [llmItem, setLlmItem] = useState<ISemantic.ILlmItem>();
const [dataBaseList, setDataBaseList] = useState<any[]>([]);
const actionRef = useRef<ActionType>();
const queryLlmList = async () => {
const { code, data, msg } = await getLlmList();
if (code === 200) {
setDataBaseList(data);
} else {
message.error(msg);
}
};
useEffect(() => {
queryLlmList();
}, []);
const columns: ProColumns[] = [
{
dataIndex: 'id',
title: 'ID',
width: 80,
},
{
dataIndex: 'name',
title: '连接名称',
},
{
dataIndex: ['config', 'modelName'],
title: '模型名称',
search: false,
},
{
dataIndex: ['config', 'provider'],
title: '接口协议',
search: false,
},
{
dataIndex: 'createdBy',
title: '创建人',
search: false,
},
{
dataIndex: 'description',
title: '描述',
search: false,
},
{
dataIndex: 'updatedAt',
title: '更新时间',
search: false,
render: (value: any) => {
return value && value !== '-' ? dayjs(value).format('YYYY-MM-DD HH:mm:ss') : '-';
},
},
{
title: '操作',
dataIndex: 'x',
valueType: 'option',
width: 100,
render: (_, record) => {
// if (!record.hasEditPermission) {
// return <></>;
// }
return (
<Space>
<a
key="dimensionEditBtn"
onClick={() => {
setLlmItem(record);
setCreateModalVisible(true);
}}
>
</a>
<Popconfirm
title="确认删除?"
okText="是"
cancelText="否"
onConfirm={async () => {
const { code, msg } = await deleteLlmConfig(record.id);
if (code === 200) {
setLlmItem(undefined);
queryLlmList();
} else {
message.error(msg);
}
}}
>
<a
key="dimensionDeleteEditBtn"
onClick={() => {
setLlmItem(record);
}}
>
</a>
</Popconfirm>
</Space>
);
},
},
];
return (
<div style={{ margin: 20 }}>
<ProTable
actionRef={actionRef}
rowKey="id"
columns={columns}
dataSource={dataBaseList}
search={false}
tableAlertRender={() => {
return false;
}}
size="small"
options={{ reload: false, density: false, fullScreen: false }}
toolBarRender={() => [
<Button
key="create"
type="primary"
onClick={() => {
setLlmItem(undefined);
setCreateModalVisible(true);
}}
>
</Button>,
]}
/>
{createModalVisible && (
<LlmSettingModal
open={createModalVisible}
llmItem={llmItem}
onCancel={() => {
setCreateModalVisible(false);
}}
onSubmit={() => {
setCreateModalVisible(false);
queryLlmList();
}}
/>
)}
</div>
);
};
export default LlmTable;

View File

@@ -780,3 +780,35 @@ export function deleteTerm(data: any): Promise<any> {
data: { ...data },
});
}
export function getLlmList(): Promise<any> {
return request(`${process.env.CHAT_API_BASE_URL}model/getModelList`, {
method: 'GET',
});
}
export function createLlmConfig(data: any): Promise<any> {
return request(`${process.env.CHAT_API_BASE_URL}chat/model`, {
method: 'POST',
data: { ...data },
});
}
export function saveLlmConfig(data: any): Promise<any> {
if (data.id) {
return request(`${process.env.CHAT_API_BASE_URL}model`, {
method: 'PUT',
data,
});
}
return request(`${process.env.CHAT_API_BASE_URL}model`, {
method: 'POST',
data,
});
}
export function deleteLlmConfig(id: number): Promise<any> {
return request(`${process.env.CHAT_API_BASE_URL}model/${id}`, {
method: 'DELETE',
});
}

View File

@@ -1,7 +1,7 @@
import request from 'umi-request';
export function testLLMConn(data: any) {
return request('/api/chat/agent/testLLMConn', {
return request(`${process.env.CHAT_API_BASE_URL}model/testConnection`, {
method: 'POST',
data,
});