topdog/supersonic
1
0
Fork 0
mirror of https://github.com/tencentmusic/supersonic.git synced 2025-12-10 19:51:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

[improvement][headless-fe] Added dynamic variable functionality to model creation based on SQL scripts. (#761)

Browse Source 
* [improvement][semantic-fe] Add model alias setting & Add view permission restrictions to the model permission management tab.
[improvement][semantic-fe] Add permission control to the action buttons for the main domain; apply high sensitivity filtering to the authorization of metrics/dimensions.
[improvement][semantic-fe] Optimize the editing mode in the dimension/metric/datasource components to use the modelId stored in the database for data, instead of relying on the data from the state manager.

* [improvement][semantic-fe] Add time granularity setting in the data source configuration.

* [improvement][semantic-fe] Dictionary import for dimension values supported in Q&A visibility

* [improvement][semantic-fe] Modification of data source creation prompt wording"

* [improvement][semantic-fe] metric market experience optimization

* [improvement][semantic-fe] enhance the analysis of metric trends

* [improvement][semantic-fe] optimize the presentation of metric trend permissions

* [improvement][semantic-fe] add metric trend download functionality

* [improvement][semantic-fe] fix the dimension initialization issue in metric correlation

* [improvement][semantic-fe] Fix the issue of database changes not taking effect when creating based on an SQL data source.

* [improvement][semantic-fe] Optimizing pagination logic and some CSS styles

* [improvement][semantic-fe] Fixing the API for the indicator list by changing "current" to "pageNum"

* [improvement][semantic-fe] Fixing the default value setting for the indicator list

* [improvement][semantic-fe] Adding batch operations for indicators/dimensions/models

* [improvement][semantic-fe] Replacing the single status update API for indicators/dimensions with a batch update API

* [improvement][semantic-fe] Redesigning the indicator homepage to incorporate trend charts and table functionality for indicators

* [improvement][semantic-fe] Optimizing the logic for setting dimension values and editing data sources, and adding system settings functionality

* [improvement][semantic-fe] Upgrading antd version to 5.x, extracting the batch operation button component, optimizing the interaction for system settings, and expanding the configuration generation types for list-to-select component.

* [improvement][semantic-fe] Adding the ability to filter dimensions based on whether they are tags or not.

* [improvement][semantic-fe] Adding the ability to edit relationships between models in the canvas.

* [improvement][semantic-fe] Updating the datePicker component to use dayjs instead.

* [improvement][semantic-fe] Fixing the issue with passing the model ID for dimensions in the indicator market.

* [improvement][semantic-fe] Fixing the abnormal state of the popup when creating a model.

* [improvement][semantic-fe] Adding permission logic for bulk operations in the indicator market.

* [improvement][semantic-fe] Adding the ability to download and transpose data.

* [improvement][semantic-fe] Fixing the initialization issue with the date selection component in the indicator details page when switching time granularity.

* [improvement][semantic-fe] Fixing the logic error in the dimension value setting.

* [improvement][semantic-fe] Fixing the synchronization issue with the question and answer settings information.

* [improvement][semantic-fe] Optimizing the canvas functionality for better performance and user experience.

* [improvement][semantic-fe] Optimizing the update process for drawing model relationship edges in the canvas.

* [improvement][semantic-fe] Changing the line type for canvas connections.

* [improvement][semantic-fe] Replacing the initialization variable from "semantic" to "headless".

* [improvement][semantic-fe] Fixing the missing migration issue for default drill-down dimension configuration in model editing. Additionally, optimizing the data retrieval method for initializing fields in the model.

* [improvement][semantic-fe] Updating the logic for the fieldName.

* [improvement][semantic-fe] Adjusting the position of the metrics tab.

* [improvement][semantic-fe] Changing the 字段名称 to 英文名称.

* [improvement][semantic-fe] Fix metric measurement deletion.

* [improvement][semantic-fe] UI optimization for metric details page.

* [improvement][semantic-fe] UI optimization for metric details page.

* [improvement][semantic-fe] UI adjustment for metric details page.

* [improvement][semantic-fe] The granularity field in the time type of model editing now supports setting it as empty.

* [improvement][semantic-fe] Added field type and metric type to the metric creation options.

* [improvement][semantic-fe] The organization structure selection feature has been added to the permission management.

* [improvement][semantic-fe] Improved user experience for the metric list.

* [improvement][semantic-fe] fix update the metric list.

* [improvement][headless-fe] Added view management functionality.

* [improvement][headless-fe] The view management functionality has been added. This feature allows users to create, edit, and manage different views within the system.

* [improvement][headless-fe] Added model editing side effect detection.

* [improvement][headless-fe] Fixed the logic error in view editing.

* [improvement][headless-fe] Fixed the issue with initializing dimension associations in metric settings.

* [improvement][headless-fe] Added the ability to hide the Q&A settings entry point.

* [improvement][headless-fe] Fixed the issue with selecting search results in metric field creation.

* [improvement][headless-fe] Added search functionality to the field list in model editing.

* [improvement][headless-fe] fix the field list in model editing

* [improvement][headless-fe] Restructured the data for the dimension value settings interface.

* [improvement][headless-fe] Added dynamic variable functionality to model creation based on SQL scripts.
This commit is contained in:
tristanliu
2024-02-26 19:56:28 +08:00
committed by GitHub
parent bdf7df933b
commit afdf18398c
8 changed files with 49 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
webapp/packages/supersonic-fe/src/pages/SemanticModel/Datasource/components/DataSourceCreateForm.tsx
View File

@@ -18,6 +18,7 @@ export type CreateFormProps = {
dispatch: Dispatch; dispatch: Dispatch;
createModalVisible: boolean; createModalVisible: boolean;
sql?: string; sql?: string;
sqlParams?: IDataSource.ISqlParamsItem[];
databaseId?: number; databaseId?: number;
modelItem: ISemantic.IModelItem; modelItem: ISemantic.IModelItem;
onCancel?: () => void; onCancel?: () => void;
@@ -42,6 +43,7 @@ const DataSourceCreateForm: React.FC<CreateFormProps> = ({
createModalVisible, createModalVisible,
scriptColumns, scriptColumns,
sql = '', sql = '',
sqlParams,
onSubmit, onSubmit,
modelItem, modelItem,
databaseId, databaseId,
@@ -249,6 +251,7 @@ const DataSourceCreateForm: React.FC<CreateFormProps> = ({
queryType: basicInfoFormMode === 'fast' ? 'table_query' : 'sql_query', queryType: basicInfoFormMode === 'fast' ? 'table_query' : 'sql_query',
tableQuery: dbName && tableName ? `${dbName}.${tableName}` : '', tableQuery: dbName && tableName ? `${dbName}.${tableName}` : '',
sqlQuery: sql, sqlQuery: sql,
sqlVariables: sqlParams,
}, },
}; };
setQueryParamsState(queryParams); setQueryParamsState(queryParams);

13
webapp/packages/supersonic-fe/src/pages/SemanticModel/Datasource/components/SqlDetail.tsx
View File

@@ -33,6 +33,7 @@ export type DataSourceSubmitData = {
sql: string; sql: string;
databaseId: number; databaseId: number;
columns: any[]; columns: any[];
sqlParams: any[];
}; };
type IProps = { type IProps = {
@@ -428,9 +429,9 @@ const SqlDetail: React.FC<IProps> = ({
<Tooltip title="格式化SQL语句"> <Tooltip title="格式化SQL语句">
<EditOutlined className={styles.sqlOprIcon} onClick={formatSQL} /> <EditOutlined className={styles.sqlOprIcon} onClick={formatSQL} />
</Tooltip> </Tooltip>
{/* <Tooltip title="动态变量"> <Tooltip title="动态变量">
<ApiOutlined className={styles.sqlOprIcon} onClick={handleVariable} /> <ApiOutlined className={styles.sqlOprIcon} onClick={handleVariable} />
</Tooltip> */} </Tooltip>
<Tooltip title="改变主题"> <Tooltip title="改变主题">
<SwapOutlined className={styles.sqlOprIcon} onClick={handleThemeChange} /> <SwapOutlined className={styles.sqlOprIcon} onClick={handleThemeChange} />
</Tooltip> </Tooltip>
@@ -481,7 +482,12 @@ const SqlDetail: React.FC<IProps> = ({
className={variableCollapsed ? styles.hideSqlParams : styles.sqlParams} className={variableCollapsed ? styles.hideSqlParams : styles.sqlParams}
// style={{ height: sqlEditorHeight }} // style={{ height: sqlEditorHeight }}
> >
<SqlParams value={sqlParams} onChange={setSqlParams} /> <SqlParams
value={sqlParams}
onChange={(params) => {
setSqlParams(params);
}}
/>
</div> </div>
</div> </div>
</Pane> </Pane>
@@ -496,6 +502,7 @@ const SqlDetail: React.FC<IProps> = ({
columns: scriptColumns, columns: scriptColumns,
databaseId: currentDatabaseItem?.key || 0, databaseId: currentDatabaseItem?.key || 0,
sql, sql,
sqlParams,
}); });
}} }}
disabled={!runState} disabled={!runState}

20
webapp/packages/supersonic-fe/src/pages/SemanticModel/Datasource/components/SqlParams.tsx
View File

@@ -1,30 +1,28 @@
import { useState } from 'react'; import { useState } from 'react';
import type { FC } from 'react'; import type { FC } from 'react';
import type { SqlParamsItem, OprType } from '../data'; import type { OprType } from '../data';
import { IDataSource } from '../../data';
import styles from '../style.less'; import styles from '../style.less';
import { AppstoreAddOutlined, DeleteTwoTone, EditTwoTone } from '@ant-design/icons'; import { AppstoreAddOutlined, DeleteTwoTone, EditTwoTone } from '@ant-design/icons';
import SqlParamsDetailModal from './SqlParamsDetailModal'; import SqlParamsDetailModal from './SqlParamsDetailModal';
import { List } from 'antd'; import { List } from 'antd';
type Props = { type Props = {
value?: SqlParamsItem[]; value?: IDataSource.ISqlParamsItem[];
onChange?: (e: SqlParamsItem[]) => void; onChange?: (e: IDataSource.ISqlParamsItem[]) => void;
}; };
const defalutItem: SqlParamsItem = { const defalutItem: IDataSource.ISqlParamsItem = {
name: '', name: '',
type: 'query',
defaultValues: [], defaultValues: [],
valueType: 'string', valueType: 'STRING',
udf: false,
}; };
const SqlParams: FC<Props> = ({ value, onChange }) => { const SqlParams: FC<Props> = ({ value, onChange }) => {
const [oprType, setOprType] = useState<OprType>('add'); const [oprType, setOprType] = useState<OprType>('add');
const [visible, setVisible] = useState<boolean>(false); const [visible, setVisible] = useState<boolean>(false);
const [initValue, setInitValue] = useState<SqlParamsItem>(); const [initValue, setInitValue] = useState<IDataSource.ISqlParamsItem>();
const paramsChange = (params: SqlParamsItem[]) => { const paramsChange = (params: IDataSource.ISqlParamsItem[]) => {
if (onChange) { if (onChange) {
onChange(params); onChange(params);
} }
@@ -34,7 +32,7 @@ const SqlParams: FC<Props> = ({ value, onChange }) => {
setVisible(true); setVisible(true);
setInitValue(defalutItem); setInitValue(defalutItem);
}; };
const handleSave = async (values: SqlParamsItem) => { const handleSave = async (values: IDataSource.ISqlParamsItem) => {
const newValue = value ? [...value] : []; const newValue = value ? [...value] : [];
const { index, ...rest } = values; const { index, ...rest } = values;
if (index || index === 0) { if (index || index === 0) {

37
webapp/packages/supersonic-fe/src/pages/SemanticModel/Datasource/components/SqlParamsDetailModal.tsx
View File

@@ -8,17 +8,15 @@ import { IDataSource } from '../../data';
import TextArea from 'antd/lib/input/TextArea'; import TextArea from 'antd/lib/input/TextArea';
import ParamsSqlEditor from './SqlParamsSqlEditor'; import ParamsSqlEditor from './SqlParamsSqlEditor';
const EnumSqlParamsType = { // const EnumSqlParamsType = {
auth: '权限变量', // auth: '权限变量',
query: '查询变量', // query: '查询变量',
}; // };
const EnumSqlValueType = { const EnumSqlValueType = {
string: '字符串', STRING: '字符串',
sql: 'SQL表达式', NUMBER: '数字',
date: '日期', EXPR: 'SQL表达式',
boolean: '布尔',
number: '数字',
}; };
const { Option } = Select; const { Option } = Select;
@@ -56,7 +54,6 @@ const SqlParamsDetailModal: FC<IProps> = ({
nameList, nameList,
}) => { }) => {
const [valueType, setValueType] = useState<IDataSource.ISqlParamsValueType>(); const [valueType, setValueType] = useState<IDataSource.ISqlParamsValueType>();
const [udf, setUdf] = useState<boolean>();
const [oldName, setOldName] = useState<string>(); const [oldName, setOldName] = useState<string>();
const formLayout = { const formLayout = {
@@ -68,12 +65,11 @@ const SqlParamsDetailModal: FC<IProps> = ({
const submitSave = async () => { const submitSave = async () => {
const fieldsValue = await form.validateFields(); const fieldsValue = await form.validateFields();
onSave({ ...fieldsValue, index: initValue.index, udf }); onSave({ ...fieldsValue, index: initValue.index });
}; };
const handleCancel = async () => { const handleCancel = async () => {
if (onCancel && isFunction(onCancel)) { if (onCancel && isFunction(onCancel)) {
setUdf(false);
onCancel(oprType); onCancel(oprType);
} }
}; };
@@ -83,7 +79,6 @@ const SqlParamsDetailModal: FC<IProps> = ({
...initValue, ...initValue,
}); });
setValueType(initValue.valueType); setValueType(initValue.valueType);
setUdf(initValue.udf);
setOldName(initValue.name); setOldName(initValue.name);
}, [initValue]); }, [initValue]);
@@ -124,7 +119,7 @@ const SqlParamsDetailModal: FC<IProps> = ({
> >
<Input placeholder="请输入参数名称" /> <Input placeholder="请输入参数名称" />
</Form.Item> </Form.Item>
<Form.Item {/* <Form.Item
name="type" name="type"
label="参数类型" label="参数类型"
rules={[{ required: true, message: '请选择参数类型' }]} rules={[{ required: true, message: '请选择参数类型' }]}
@@ -136,7 +131,7 @@ const SqlParamsDetailModal: FC<IProps> = ({
</Option> </Option>
))} ))}
</Select> </Select>
</Form.Item> </Form.Item> */}
<Form.Item <Form.Item
name="valueType" name="valueType"
label="值类型" label="值类型"
@@ -146,7 +141,7 @@ const SqlParamsDetailModal: FC<IProps> = ({
placeholder="请选择值类型" placeholder="请选择值类型"
onChange={(e) => { onChange={(e) => {
setValueType(e as IDataSource.ISqlParamsValueType); setValueType(e as IDataSource.ISqlParamsValueType);
if (e === 'sql') { if (e === 'EXPR') {
form.setFieldsValue({ form.setFieldsValue({
defaultValues: undefined, defaultValues: undefined,
}); });
@@ -160,10 +155,10 @@ const SqlParamsDetailModal: FC<IProps> = ({
))} ))}
</Select> </Select>
</Form.Item> </Form.Item>
<Form.Item name="alias" label="别名"> {/* <Form.Item name="alias" label="别名">
<Input placeholder="请输入参数别名" /> <Input placeholder="请输入参数别名" />
</Form.Item> </Form.Item> */}
{valueType !== 'sql' && ( {/* {valueType !== 'sql' && (
<Form.Item name="udf" label="是否使用表达式"> <Form.Item name="udf" label="是否使用表达式">
<Checkbox <Checkbox
checked={udf} checked={udf}
@@ -177,8 +172,8 @@ const SqlParamsDetailModal: FC<IProps> = ({
}} }}
/> />
</Form.Item> </Form.Item>
)} )} */}
{valueType === 'sql' || udf ? ( {valueType === 'EXPR' ? (
<Form.Item name="defaultValues" label="表达式"> <Form.Item name="defaultValues" label="表达式">
<ParamsTextArea /> <ParamsTextArea />
</Form.Item> </Form.Item>

5
webapp/packages/supersonic-fe/src/pages/SemanticModel/View/components/ViewCreateFormModal.tsx
View File

@@ -7,13 +7,8 @@ import { createView, updateView, getDimensionList, queryMetric } from '../../ser
import { ISemantic } from '../../data'; import { ISemantic } from '../../data';
import { isString } from 'lodash'; import { isString } from 'lodash';
import FormItemTitle from '@/components/FormHelper/FormItemTitle'; import FormItemTitle from '@/components/FormHelper/FormItemTitle';
import SelectPartner from '@/components/SelectPartner';
import SelectTMEPerson from '@/components/SelectTMEPerson'; import SelectTMEPerson from '@/components/SelectTMEPerson';
import ViewModelConfigTransfer from './ViewModelConfigTransfer'; import ViewModelConfigTransfer from './ViewModelConfigTransfer';
import DefaultSettingForm from './DefaultSettingForm';
import SqlEditor from '@/components/SqlEditor';
import ProCard from '@ant-design/pro-card';
import { ChatConfigType } from '../../enum';
const FormItem = Form.Item; const FormItem = Form.Item;

23
webapp/packages/supersonic-fe/src/pages/SemanticModel/components/ClassDataSourceTypeModal.tsx
View File

@@ -41,6 +41,7 @@ const ClassDataSourceTypeModal: React.FC<Props> = ({
const [dataSourceEditOpen, setDataSourceEditOpen] = useState<boolean>(false); const [dataSourceEditOpen, setDataSourceEditOpen] = useState<boolean>(false);
const [currentDatabaseId, setCurrentDatabaseId] = useState<number>(); const [currentDatabaseId, setCurrentDatabaseId] = useState<number>();
const [scriptColumns, setScriptColumns] = useState<IDataSource.IExecuteSqlColumn[]>([]); const [scriptColumns, setScriptColumns] = useState<IDataSource.IExecuteSqlColumn[]>([]);
const [sqlParams, setSqlParams] = useState<IDataSource.ISqlParamsItem[]>([]);
useEffect(() => { useEffect(() => {
if (!dataSourceItem?.id || !open) { if (!dataSourceItem?.id || !open) {
@@ -88,23 +89,6 @@ const ClassDataSourceTypeModal: React.FC<Props> = ({
} }
}, [dataSourceItem]); }, [dataSourceItem]);
// const fetchTaskResult = (params) => {
// setScriptColumns(params.columns);
// };
// const queryTableColumnListByScript = async (dataSource: IDataSource.IDataSourceItem) => {
// if (!dataSource?.modelDetail?.sqlQuery) {
// return;
// }
// const { code, data } = await excuteSql({
// sql: dataSource.modelDetail?.sqlQuery,
// id: dataSource.databaseId,
// });
// if (code === 200) {
// fetchTaskResult(data);
// }
// };
return ( return (
<> <>
<Modal <Modal
@@ -187,6 +171,7 @@ const ClassDataSourceTypeModal: React.FC<Props> = ({
basicInfoFormMode="normal" basicInfoFormMode="normal"
modelItem={dataSourceItem} modelItem={dataSourceItem}
scriptColumns={scriptColumns} scriptColumns={scriptColumns}
sqlParams={sqlParams}
onCancel={() => { onCancel={() => {
setCreateModalVisible(false); setCreateModalVisible(false);
handleCancel(); handleCancel();
@@ -215,9 +200,11 @@ const ClassDataSourceTypeModal: React.FC<Props> = ({
<DataSource <DataSource
initialValues={dataSourceItem} initialValues={dataSourceItem}
onSubmitSuccess={(dataSourceInfo) => { onSubmitSuccess={(dataSourceInfo) => {
const { columns, sql, databaseId } = dataSourceInfo; console.log('onSubmitSuccess', dataSourceInfo);
const { columns, sql, databaseId, sqlParams } = dataSourceInfo;
setSql(sql); setSql(sql);
setScriptColumns(columns); setScriptColumns(columns);
setSqlParams(sqlParams);
setCurrentDatabaseId(databaseId); setCurrentDatabaseId(databaseId);
setDataSourceEditOpen(false); setDataSourceEditOpen(false);
}} }}

13
webapp/packages/supersonic-fe/src/pages/SemanticModel/data.d.ts vendored
View File

@@ -13,7 +13,6 @@ export type SensitiveLevel = 0 | 1 | 2 | null;
export type ToolBarSearchCallBack = (text: string) => void; export type ToolBarSearchCallBack = (text: string) => void;
export declare namespace IDataSource { export declare namespace IDataSource {
interface IExecuteSqlColumn { interface IExecuteSqlColumn {
name?: string; name?: string;
@@ -59,15 +58,15 @@ export declare namespace IDataSource {
fieldName: string; fieldName: string;
} }
type ISqlParamsValueType = 'string' | 'sql' | 'date' | 'boolean' | 'number'; type ISqlParamsValueType = 'STRING' | 'EXPR' | 'NUMBER';
interface ISqlParamsItem { interface ISqlParamsItem {
index?: number; index?: number;
defaultValues: (boolean | string | number)[]; defaultValues: (boolean | string | number)[];
name: string; name: string;
type: string; // type: string;
valueType: ISqlParamsValueType; valueType: ISqlParamsValueType;
udf?: boolean; udf?: boolean;
}; }
interface IDataSourceDetail { interface IDataSourceDetail {
queryType: string; queryType: string;
@@ -77,7 +76,7 @@ export declare namespace IDataSource {
fields: IDataSourceDetailFieldsItem[]; fields: IDataSourceDetailFieldsItem[];
dimensions: IDimensionsItem[]; dimensions: IDimensionsItem[];
measures: IMeasuresItem[]; measures: IMeasuresItem[];
sqlVariables: sqlVariables: ISqlParamsItem[];
} }
interface IDataSourceItem { interface IDataSourceItem {
@@ -344,7 +343,6 @@ export declare namespace ISemantic {
} }
type IDatabaseItemList = IDatabaseItem[]; type IDatabaseItemList = IDatabaseItem[];
interface IDictKnowledgeConfigItemConfig { interface IDictKnowledgeConfigItemConfig {
metricId?: number; metricId?: number;
blackList: string[]; blackList: string[];
@@ -394,7 +392,6 @@ export declare namespace IChatConfig {
}; };
} }
interface IKnowledgeInfosItem { interface IKnowledgeInfosItem {
itemId: number; itemId: number;
bizName: string; bizName: string;

1
webapp/packages/supersonic-fe/src/pages/SemanticModel/enum.ts
View File

@@ -20,6 +20,7 @@ export enum MetricTypeWording {
} }
export enum DictTaskState { export enum DictTaskState {
initial = '--',
error = '错误', error = '错误',
pending = '等待', pending = '等待',
running = '正在执行', running = '正在执行',