Skip to content

Commit

Permalink
NIFI-13097 Set project.version in Python Extension Processors
Browse files Browse the repository at this point in the history
- Enabled resource filtering for Python Extension modules

Signed-off-by: Pierre Villard <[email protected]>

This closes apache#8700.
  • Loading branch information
exceptionfactory authored and shubhluck committed Jun 1, 2024
1 parent abce11f commit 2fae910
Show file tree
Hide file tree
Showing 9 changed files with 9 additions and 7 deletions.
1 change: 1 addition & 0 deletions nifi-python-extensions/nifi-openai-module/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
<resources>
<resource>
<directory>src/main/python</directory>
<filtering>true</filtering>
<includes>
<include>**/</include>
</includes>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ class Java:
implements = ['org.apache.nifi.python.processor.FlowFileTransform']

class ProcessorDetails:
version = '2.0.0-SNAPSHOT'
version = '@project.version@'
description = "Submits a prompt to ChatGPT, writing the results either to a FlowFile attribute or to the contents of the FlowFile"
tags = ["text", "chatgpt", "gpt", "machine learning", "ML", "artificial intelligence", "ai", "document", "langchain"]
dependencies = ['langchain==0.1.2', 'openai==1.9.0', 'jsonpath-ng']
Expand Down
1 change: 1 addition & 0 deletions nifi-python-extensions/nifi-text-embeddings-module/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
<resources>
<resource>
<directory>src/main/python</directory>
<filtering>true</filtering>
<includes>
<include>**/</include>
</includes>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ class ChunkDocument(FlowFileTransform):
class Java:
implements = ['org.apache.nifi.python.processor.FlowFileTransform']
class ProcessorDetails:
version = '2.0.0-SNAPSHOT'
version = '@project.version@'
description = """Chunks incoming documents that are formatted as JSON Lines into chunks that are appropriately sized for creating Text Embeddings.
The input is expected to be in "json-lines" format, with each line having a 'text' and a 'metadata' element.
Each line will then be split into one or more lines in the output."""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ class Java:
implements = ["org.apache.nifi.python.processor.FlowFileTransform"]

class ProcessorDetails:
version = "2.0.0-SNAPSHOT"
version = '@project.version@'
description = """Parses incoming unstructured text documents and performs optical character recognition (OCR) in order to extract text from PDF and image files.
The output is formatted as "json-lines" with two keys: 'text' and 'metadata'.
Note that use of this Processor may require significant storage space and RAM utilization due to third-party dependencies necessary for processing PDF and image files.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ class Java:
implements = ['org.apache.nifi.python.processor.FlowFileTransform']

class ProcessorDetails:
version = '2.0.0-SNAPSHOT'
version = '@project.version@'
description = """Publishes JSON data to a Chroma VectorDB. The Incoming data must be in single JSON per Line format, each with two keys: 'text' and 'metadata'.
The text must be a string, while metadata must be a map with strings for values. Any additional fields will be ignored. If the collection name specified
does not exist, the Processor will automatically create the collection."""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ class Java:
implements = ['org.apache.nifi.python.processor.FlowFileTransform']

class ProcessorDetails:
version = '2.0.0-SNAPSHOT'
version = '@project.version@'
description = """Publishes JSON data to Pinecone. The Incoming data must be in single JSON per Line format, each with two keys: 'text' and 'metadata'.
The text must be a string, while metadata must be a map with strings for values. Any additional fields will be ignored."""
tags = ["pinecone", "vector", "vectordb", "vectorstore", "embeddings", "ai", "artificial intelligence", "ml", "machine learning", "text", "LLM"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ class Java:
implements = ['org.apache.nifi.python.processor.FlowFileTransform']

class ProcessorDetails:
version = '2.0.0-SNAPSHOT'
version = '@project.version@'
description = "Queries a Chroma Vector Database in order to gather a specified number of documents that are most closely related to the given query."
tags = ["chroma", "vector", "vectordb", "embeddings", "enrich", "enrichment", "ai", "artificial intelligence", "ml", "machine learning", "text", "LLM"]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ class Java:
implements = ['org.apache.nifi.python.processor.FlowFileTransform']

class ProcessorDetails:
version = '2.0.0-SNAPSHOT'
version = '@project.version@'
description = "Queries Pinecone in order to gather a specified number of documents that are most closely related to the given query."
tags = ["pinecone", "vector", "vectordb", "vectorstore", "embeddings", "ai", "artificial intelligence", "ml", "machine learning", "text", "LLM"]

Expand Down

0 comments on commit 2fae910

Please sign in to comment.