[improvement][docs]Update README to improve motivation part

This commit is contained in:
jerryjzhang
2023-10-08 14:08:48 +08:00
parent 69242f9f2d
commit e3b3e8861d
2 changed files with 4 additions and 4 deletions

View File

@@ -8,12 +8,12 @@
## Motivation
The emergence of Large Language Model (LLM) like ChatGPT is reshaping the way information is retrieved. In the field of data analytics, both academia and industry are primarily focused on leveraging LLM to convert natural language into SQL (so called text2sql or nl2sql). While some works show promising results, they are still not applicable to real-world scenarios. The biggest obstacle stems from the
The emergence of Large Language Model (LLM) like ChatGPT is reshaping the way information is retrieved. In the field of data analytics, both academia and industry are primarily focused on leveraging LLM to convert natural language into SQL (so called text2sql or nl2sql). While some works exhibit promising results, their **reliability** is inadequate for real-world applications.
From our perspective, the key to filling the real-world gap lies in three aspects:
1. Introduce a semantic layer encapsulating underlying data context(joins, formulas, etc) to reduce **complexity**.
2. Augment the LLM with schema mappers(as a kind of preprocessor) and semantic correctors(as a kind of postprocessor) to mitigate **hallucination**.
3. Utilize heuristic rules to improve **efficiency**(in terms of latency and cost).
3. Utilize heuristic rules when necessary to improve **efficiency**(in terms of latency and cost).
With these ideas in mind, we develop SuperSonic as a practical reference implementation and use it to power our real-world products. Additionally, to facilitate further development of ChatBI, we decide to open source SuperSonic as an extensible framework.

View File

@@ -6,12 +6,12 @@
## 项目动机
大型语言模型LLMs如ChatGPT的出现正在重塑信息检索的方式。在数据分析领域学术界和工业界主要关注利用深度学习模型将自然语言查询转换为SQL查询。虽然一些工作显示出有前景的结果但它们还并不适用于实际场景
大型语言模型LLMs如ChatGPT的出现正在重塑信息检索的方式。在数据分析领域学术界和工业界主要关注利用深度学习模型将自然语言查询转换为SQL查询。虽然一些工作显示出有前景的结果但它们的可靠性还达不到生产可用的要求
在我们看来,为了在实际场景发挥价值,有三个关键点:
1. 引入语义模型层封装底层数据的上下文关联、公式等降低SQL生成的**复杂度**。
2. 通过一前一后的模式映射器和语义修正器来缓解LLM常见的**幻觉**现象。
3. 利用基于规则的解析器,提升语义解析的**效率**。
3. 设计启发式的规则,在一些特定场景提升语义解析的**效率**。
为了验证上述想法,我们开发了超音数项目,并将其应用在实际的内部产品中。与此同时,我们将超音数作为一个可扩展的框架开源,希望能够促进数据问答对话领域的进一步发展。