Add feature to s2sql that allow few-shots example tied to data model. (#571)

* 1.refactor the retrieval module. 2.refactor the http service module. 3.upgrade text2sql output format the parse for absolute time related expression in query.

* fix bug.

* upgrade the config module, now support config llm suppoted by langchain.

* fix conflicts.

* update text2sql config reload to be compitable with new config format.

* modify default config.

* 1.add self-consistency feature for text2sql. 2.upgrade llm api call from sync to async. 3.refactor text2sql module. 4. refactor semantical retriever modules.

* merege with upstream master

* add general retrieve service.

* add api service for sql_agent for crud opereations of few-shots examples.

* modify requirements

* add auto-cot feature

* 1. output log to a fixed log file.  2.allow few-shots examples tied to data model, and add strategy that extend examples when retrieved examples tied to a data model is not enough. 3. fix misformat in s2ql args.

* add prior_ext to output.

---------

Co-authored-by: shaoweigong <shaoweigong@tencent.com>
This commit is contained in:
codescracker
2023-12-27 19:39:50 +08:00
committed by GitHub
parent cf2b4bfb5c
commit b706c4efb4
6 changed files with 512 additions and 68 deletions

View File

@@ -70,6 +70,7 @@ def transform_sql_example_autoCoT_run(examplar_list, min_window_size, max_window
transformed_sql_examplar = dict()
transformed_sql_examplar['question'] = question
transformed_sql_examplar['questionAugmented'] = question_augmented
transformed_sql_examplar['modelName'] = table_name
transformed_sql_examplar['dbSchema'] = db_schema
transformed_sql_examplar['sql'] = sql
transformed_sql_examplar['generatedSchemaLinkingCoT'] = generated_schema_linking_cot