Introduced a simple demo of mixed index and text2gremlin demonstration. #54

vichayturen · 2024-07-02T12:50:18Z

Main changes:

Modified the configuration method.
Added vector index for RAG, supported mixed-retrieving and reranking.
Introduced embedding configuration.
Introduced index operation and document operation.
Added task of transferring natural language to gremlin and its demonstration.
Simplified the use and modify the usage in README.

TODOs (add them later) 🔢

2. Graph RAG增加向量索引和混合索引加重排的两种方式的索引。 3. 增加自然语言转gremlin的任务及演示。

hugegraph-llm/src/hugegraph_llm/demo/gremlin_generate_web_demo.py

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py

TODO: lack some check now

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py

for debug RESTful API problem

2. added template input textbox 3. changed ernie llm to use qianfan sdk 4. added function of filtering id length

Also rename the same params (inner & outer) TODO: Non-Schema mode will throw Error now (we need fix it)

…id querying - Update BuildSemanticIndex class to match new file naming convention. - Modify code_format_and_analysis.sh to use a line length of 120. - Change logging format to use %s instead of {} for string formatting.

1. Introduce a clean-up function to remove index and content files, aiding in maintaining a clean file system after operations. 2. Introduce default demo of build kg. 3. Now it does not clean the kg before building kg by default. 4. Clean the stopwords files.

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py

…triples.

imbajin · 2024-07-10T07:13:07Z

Now we could use log.xx(msg) to print the log: ↓

…raph-ai into pr/54

ChenZiHong-Gavin · 2024-07-16T01:36:57Z

Now the .config file instead of json file can be generated with command like python -m hugegraph_llm.config.generate --dir_path=./

…y id. (#2)

init graphspace support

jasinliu · 2024-07-19T06:56:37Z

hugegraph-llm/src/hugegraph_llm/config/config.py

+
+dirname = os.path.dirname
+package_path = dirname(dirname(dirname(dirname(os.path.abspath(__file__)))))
+env_path = os.path.join(package_path, ".env")


在pip安装之后运行代码会在python所在路径下生成.env

There are some aspects that can be optimized in the future.

Add some configuration-related comments in the .env file

Rename .env filename to something related to hugegraph-llm

Unify the variables in the .env file and the variables in the Config class to be all uppercase

Differentiate between commonly used parameters and parameters that are secret keys

Give user a choice where to place their .env file

* support local openai env * fix env

…raph-ai into pr/54

…-only", "vector-only", "graph-vector"

# Conflicts: # hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py

…fix code style (#3) * fix: fix the triple extract * 修改格式，和属性图模式 * 修复代码样式，并增加错误抓取和传输到前端界面

1. 修改配置方式，整体配置简单易用。

671b9a8

2. Graph RAG增加向量索引和混合索引加重排的两种方式的索引。 3. 增加自然语言转gremlin的任务及演示。

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Jul 2, 2024

github-actions bot added the llm label Jul 2, 2024

dosubot bot added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 2, 2024

imbajin mentioned this pull request Jul 3, 2024

refact: support py 3.12 & improve the graph rag config experience #53

Merged

1 task

github-advanced-security bot found potential problems Jul 3, 2024

View reviewed changes

hugegraph-llm/src/hugegraph_llm/demo/gremlin_generate_web_demo.py Fixed Show fixed Hide fixed

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py Fixed Show fixed Hide fixed

imbajin added 2 commits July 4, 2024 10:46

Merge branch 'main' into pr/54

d6d7d2d

reafact: add conn check after applying configs (WIP)

33668f8

TODO: lack some check now

github-advanced-security bot found potential problems Jul 4, 2024

View reviewed changes

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py Fixed Show fixed Hide fixed

fix: use log to print some info

db2a8ff

github-advanced-security bot found potential problems Jul 4, 2024

View reviewed changes

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py Dismissed Show dismissed Hide dismissed

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py Fixed Show fixed Hide fixed

imbajin and others added 7 commits July 4, 2024 17:02

feat(llm): support qianfan platform for embedding & replace wenxin refer

65caea3

1. 修复了导入和搜索的bug

d0956ca

fix upload document

e1a5f3c

fix the non-disambiguate uploading

026a1b7

chore: exclude binary files & update params order

b72ab82

fix: openai params not match

875de1f

log: record error in py-client with orange color

19513d7

for debug RESTful API problem

github-actions bot added the python-client label Jul 5, 2024

vichayturen and others added 5 commits July 6, 2024 17:08

1. added triples and edges id setting

b3c0f6f

2. added template input textbox 3. changed ernie llm to use qianfan sdk 4. added function of filtering id length

refact: update the prompt template

adabef9

Also rename the same params (inner & outer) TODO: Non-Schema mode will throw Error now (we need fix it)

Add generating operation of config file in README.md

febfa85

github-advanced-security bot found potential problems Jul 8, 2024

View reviewed changes

hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py Dismissed Show dismissed Hide dismissed

imbajin and others added 4 commits July 8, 2024 16:56

fix: use lower case to compare str & enhance the extract prompt

fcd46c4

fix: Fix to add triples from extracted vertex, and fix the format of …

b4f9405

…triples.

doc: update readme

86fa022

feat: use custom_handler log color

3bf2252

ChenZiHong-Gavin and others added 4 commits July 10, 2024 20:12

fix: avoid FSRF security warning

2ac1b33

Merge branch 'main' of https://github.com/vichayturen/incubator-hugeg…

f156af4

…raph-ai into pr/54

doc: update readme

1e95975

feat: change config store method

3c3ae34

imbajin and others added 5 commits July 16, 2024 15:01

chore: use a flexible version for dependencies

a2b17d5

feat(hugegraph_llm): supporting property graph extract and primary ke…

27d785d

…y id. (#2)

fix: token type should be int (WIP)

f4aecdc

init graphspace support

fix: read int parameter from .env

d0f29e2

feat: use .env as default config file

0a5ca6f

jasinliu reviewed Jul 19, 2024

View reviewed changes

vichayturen and others added 9 commits July 20, 2024 14:07

fix: Add a 'copy' button to the output box

cc58220

Support local openai env (#1)

f1343bd

* support local openai env * fix env

refact: set gpt-4o-mini as the openai default type

3863829

Merge branch 'main' of https://github.com/vichayturen/incubator-hugeg…

1a2608c

…raph-ai into pr/54

feat: Add rag option for four output type including "llm-raw", "graph…

ec5f32d

…-only", "vector-only", "graph-vector"

feat: Add vector result output in cmd.

469a636

Merge remote-tracking branch 'origin/main'

1ee15c2

# Conflicts: # hugegraph-llm/src/hugegraph_llm/demo/rag_web_demo.py

feat: Add error catch and display it on the front-end web interface, …

28c97ee

…fix code style (#3) * fix: fix the triple extract * 修改格式，和属性图模式 * 修复代码样式，并增加错误抓取和传输到前端界面

revert: graphspace init for CI

129c9cb

ChenZiHong-Gavin approved these changes Jul 23, 2024

View reviewed changes

simon824 approved these changes Jul 23, 2024

View reviewed changes

simon824 merged commit 3a3698b into apache:main Jul 23, 2024
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduced a simple demo of mixed index and text2gremlin demonstration. #54

Introduced a simple demo of mixed index and text2gremlin demonstration. #54

vichayturen commented Jul 2, 2024 •

edited by imbajin

Loading

imbajin commented Jul 10, 2024

ChenZiHong-Gavin commented Jul 16, 2024 •

edited

Loading

jasinliu Jul 19, 2024

ChenZiHong-Gavin Jul 22, 2024

Introduced a simple demo of mixed index and text2gremlin demonstration. #54

Introduced a simple demo of mixed index and text2gremlin demonstration. #54

Conversation

vichayturen commented Jul 2, 2024 • edited by imbajin Loading

imbajin commented Jul 10, 2024

ChenZiHong-Gavin commented Jul 16, 2024 • edited Loading

jasinliu Jul 19, 2024

Choose a reason for hiding this comment

ChenZiHong-Gavin Jul 22, 2024

Choose a reason for hiding this comment

vichayturen commented Jul 2, 2024 •

edited by imbajin

Loading

ChenZiHong-Gavin commented Jul 16, 2024 •

edited

Loading