Skip to content

Commit

Permalink
Docs Lint
Browse files Browse the repository at this point in the history
  • Loading branch information
JingofXin committed Sep 10, 2024
1 parent 060091e commit b2abae6
Show file tree
Hide file tree
Showing 31 changed files with 501 additions and 433 deletions.
8 changes: 4 additions & 4 deletions docs/en/Advanced Topics/contribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,10 @@ Push your code to the remote repository. If it's the first time you're pushing,
- Create a pull request on the GitHub Pull request page
- Modify the PR description according to the guide, so that other developers can better understand your changes.

> **Note**:
>
> - The PR description should include the reason for the changes, the content of the changes, the impact of the changes, and link to related issues.
> - Once all reviewers agree to merge the PR, we will merge it into the main branch as soon as possible.
!!! Note

- The PR description should include the reason for the changes, the content of the changes, the impact of the changes, and link to related issues.
- Once all reviewers agree to merge the PR, we will merge it into the main branch as soon as possible.

#### Resolve Conflicts

Expand Down
64 changes: 40 additions & 24 deletions docs/en/Best Practice/flow.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
## Core of Application Building: Data Flow
# Core of Application Building: Data Flow

LazyLLM defines a multitude of data flow components that enable you to build complex large model applications using the tools and components provided by LazyLLM, much like building with blocks. This section will provide a detailed introduction to the usage of data flow.

### Definitions and API Documentation
## Definitions and API Documentation

[](){#use-flow}
The definitions and basic usage of data flow are described in [flow][lazyllm.flow.FlowBase].

### pipeline
## Pipeline

##### Basic Usage
#### Basic Usage

A Pipeline is a sequential data flow where the output of one stage becomes the input of the next stage. Pipelines support both functions and functors (or the type of functors). A typical pipeline is as follows:

Expand All @@ -26,7 +26,9 @@ f3 = Functor()
assert pipeline(f1, f2, f3, Functor)(1) == 256
```

> **Note**: Functions registered with LazyLLM's registration mechanism :[register][lazyllm.common.Register] can also be used directly by the pipeline. Below is an example:
!!! Note

Functions registered with LazyLLM's registration mechanism :[register][lazyllm.common.Register] can also be used directly by the pipeline. Below is an example:

```python
import lazyllm
Expand All @@ -43,7 +45,7 @@ def test2(input): return input * 3
assert pipeline(lazyllm.g1.test1, lazyllm.g1.test2(launcher=lazyllm.launchers.empty))(1) == 6
```

##### with Statement
#### with Statement

In addition to the basic usage, the pipeline also supports a more flexible usage with the ``with pipeline() as p`` statement to make the code more concise and clear. Here is an example:

Expand All @@ -65,9 +67,11 @@ with pipeline() as p:
assert p(1) == 16
```

> **Note**: Components such as ``parallel``, ``diverter``, ``switch``, ``loop`` etc., also support the with statement.
!!! Note

Components such as ``parallel``, ``diverter``, ``switch``, ``loop`` etc., also support the with statement.

##### Parameter Binding
#### Parameter Binding

[](){#use-bind}

Expand Down Expand Up @@ -99,10 +103,10 @@ assert p(1) == 'get [1], [f3-5], [5]'
In the example above, the ``bind`` function is used for parameter binding. Its basic usage is similar to C++'s ``std::bind``, where ``_0`` indicates the position of the new function's first parameter in the bound function's parameter list.
For the above case,The entire pipeline's input will be used as the first parameter of f4 (assuming we start counting from the first parameter). The output of f3 (i.e., the input to the new function) will be used as the second parameter of f4, and the output of f2 will be used as the third parameter of f4.

> **Note**:
>
> - Parameter binding is effective only within a single pipeline (note that when flows are nested, it does not apply in the subflow). It only allows downstream functions to bind the output of upstream functions as parameters.
> - When using parameter binding, any parameters passed in that are not referenced by ``placeholders`` such as ``_0``, ``_1``, etc., will be discarded.
!!! Note

- Parameter binding is effective only within a single pipeline (note that when flows are nested, it does not apply in the subflow). It only allows downstream functions to bind the output of upstream functions as parameters.
- When using parameter binding, any parameters passed in that are not referenced by ``placeholders`` such as ``_0``, ``_1``, etc., will be discarded.

The above method is already simple and clear enough. If you still find the function ``bind`` not intuitive, you can try the following approach. There is no difference between the two methods:

Expand All @@ -116,7 +120,9 @@ with pipeline() as p:
assert p(1) == 'get [1], [f3-5], [5]'
```

> **Note**: Please be careful with lambda functions! If you use a lambda function, make sure to enclose it in parentheses, for example: ``(lambda x, y: pass) | bind(1, _0)``
!!! Note

Please be careful with lambda functions! If you use a lambda function, make sure to enclose it in parentheses, for example: ``(lambda x, y: pass) | bind(1, _0)``

In addition to the C++ style bind method, as a Python library, we also provide parameter binding using ``kwargs``. You can mix ``kwargs`` with the C++ style binding method. Here's an example:

Expand All @@ -130,7 +136,9 @@ with pipeline() as p:
assert p(1) == 'get [1], [f3-5], [5]'
```

> **Note**: The values of parameters bound through ``kwargs`` cannot use ``_0`` and similar placeholders.
!!! Note

The values of parameters bound through ``kwargs`` cannot use ``_0`` and similar placeholders.

If the input to the pipeline is complex, you can directly perform a simple parsing of the ``input``. Here is an example:

Expand All @@ -152,7 +160,7 @@ assert p1([1, 2]) == '[[3 + 2] + 1]'
The example is a bit complex, so let's break it down step by step. First, the input list is processed by ``p1.f1`` which transforms it into a dictionary: ``dict(a=1, b=2)`` .This dictionary becomes the input for p2. After passing through ``p2.f2``, the output is ``3``,
Next, ``p2.f3`` is bound to the ``['b']`` value of the ``p2`` input, which is ``2``. Thus, the output of p2.f3 is ``[3 + 2]``. Finally, we return to ``p1.f3``, which is bound to the 0th element of the ``p1`` input. The final output is ``[[3 + 2] + 1]``.

##### pipeline.bind
#### pipeline.bind

When nesting pipelines (or pipelines with other flows), sometimes it's necessary to pass the outer layer's input to the inner layer. In such cases, you can use binding. Here's an example:

Expand All @@ -168,7 +176,7 @@ with pipeline() as p1:
assert p1([1, 2]) == '[[3 + 1] + 2]'
```

##### AutoCapture (Experimental Feature)
#### AutoCapture (Experimental Feature)

In order to further simplify the complexity of the code, we have introduced the ability to automatically capture variables defined within a with block. Here is an example:

Expand All @@ -189,15 +197,19 @@ with pipeline(auto_capture=True) as p:
assert p(1) == 'get [1], [f3-5]'
```

> **Note**: This capability is currently not very mature and is not recommended for use. Stay tuned for updates.
!!! Note

### parallel
This capability is currently not very mature and is not recommended for use. Stay tuned for updates.

## Parallel

All components of ``parallel`` share the input and merge the results for output. The definition method of ``parallel`` is similar to that of ``pipeline``. You can either initialize its elements directly when defining ``parallel`` or initialize its elements within a with block.

> **Note**: Since all modules in ``parallel`` share the input, the input to ``parallel`` does not support parameter binding.
!!! Note

Since all modules in ``parallel`` share the input, the input to ``parallel`` does not support parameter binding.

##### Result Post-Processing
#### Result Post-Processing

To further simplify the complexity of the process without introducing too many anonymous functions, the result of parallel can undergo simple post-processing (currently only supporting ``sum`` or ``asdict``) before being passed to the next stage. Here is an example:

Expand All @@ -222,9 +234,11 @@ with parallel().sum as p:
assert p(1) == 2
```

> **Note**: If using ``asdict``, you need to name the elements within ``parallel``. The returned ``dict`` will use these names as the ``key``.
!!! Note

##### Sequential Execution
If using ``asdict``, you need to name the elements within ``parallel``. The returned ``dict`` will use these names as the ``key``.

#### Sequential Execution

By default, ``parallel`` executes in parallel using multiple threads. In some special cases, you can change it to sequential execution as needed. Here is an example:

Expand All @@ -239,8 +253,10 @@ with parallel.sequential() as p:
assert p(1) == (1, 1)
```

> **Note**: ``diverter`` can also achieve sequential execution through ``.sequential``
!!! Note

``diverter`` can also achieve sequential execution through ``.sequential``

### Summary
## Summary

This article focused on ``pipeline`` and ``parallel``. It is hoped that you now have a basic understanding of how to use LazyLLM's flow to build complex applications. Other data flow components are not discussed in detail here; you can refer to [flow][lazyllm.flow.FlowBase] for their usage.
14 changes: 7 additions & 7 deletions docs/en/Best Practice/flowapp.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

### Building Applications Based on Internal Tools

> **Note**:
>
> Although it appears to be composed of modules combined into a data flow and then started uniformly, we are not a "static graph". The main manifestations are:
>
> - We use basic Python syntax to build applications, and the usage is very close to traditional programming methods.
> - You can still take advantage of Python's powerful and flexible features to change the topology of an already built application during runtime, and subsequent execution will follow the modified structure.
> - You can flexibly inject hook functions you wish to execute at the connection points of the modules. These functions can even be defined at runtime.
!!! Note

Although it appears to be composed of modules combined into a data flow and then started uniformly, we are not a "static graph". The main manifestations are:

- We use basic Python syntax to build applications, and the usage is very close to traditional programming methods.
- You can still take advantage of Python's powerful and flexible features to change the topology of an already built application during runtime, and subsequent execution will follow the modified structure.
- You can flexibly inject hook functions you wish to execute at the connection points of the modules. These functions can even be defined at runtime.

Here is an example of an application built based on LazyLLM:

Expand Down
16 changes: 8 additions & 8 deletions docs/en/Best Practice/functionCall.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,10 +158,10 @@ print(f"ret: {ret}")
# The current weather in Tokyo is 10 degrees Celsius, and in Paris, it is 22 degrees Celsius.
```

> Note:
>
> - When registering a function or tool, you must specify the default group `tool`, otherwise the model will not be able to use the corresponding tool.
> - When using the model, thers is no need to distinguish between [TrainableModule][lazyllm.module.TrainableModule] and [OnlineChatModule][lazyllm.module.onlineChatModule.OnlineChatModule], because the output types of [TrainableModule][lazyllm.module.TrainableModule] and [OnlineChatModule][lazyllm.module.onlineChatModule.OnlineChatModule] are designed to the same.
!!! Note

- When registering a function or tool, you must specify the default group `tool`, otherwise the model will not be able to use the corresponding tool.
- When using the model, thers is no need to distinguish between [TrainableModule][lazyllm.module.TrainableModule] and [OnlineChatModule][lazyllm.module.onlineChatModule.OnlineChatModule], because the output types of [TrainableModule][lazyllm.module.TrainableModule] and [OnlineChatModule][lazyllm.module.onlineChatModule.OnlineChatModule] are designed to the same.

## Design Concept of FunctionCall
The design process of [FunctionCall][lazyllm.tools.agent.FunctionCall] is carried out in a bottom-up manner. First, since [FunctionCall][lazyllm.tools.agent.FunctionCall] must call LLM, the output format of the model must be consistent. Therefore, the outputs of [TrainableModule][lazyllm.module.TrainableModule] and [OnlineChatModule][lazyllm.module.onlineChatModule.OnlineChatModule] are aligned. Then a single round of [FunctionCall][lazyllm.tools.agent.FunctionCall] is implemented, that is, LLM and tools are called once. Finally, the complete [FunctionCallAgent][lazyllm.tools.agent.FunctionCallAgent] is implemented, that is, [FunctionCall][lazyllm.tools.agent.FunctionCall] is iterated multiple times until the model iteration is completed or the maximum number of iterations is exceeded.
Expand Down Expand Up @@ -244,10 +244,10 @@ If the tool is not called, the output is of type str, which is the output of the
今天东京的天气温度是10摄氏度,而巴黎的天气温度是22摄氏度。
```

> Note:
>
> - The output format of the model is `content<|tool_calls|>tool_calls`, the delimiter is fixed, and the delimiter is used to determine whether it is a tool call.
> - The tool call information contains the tool's `name` and `arguments` fields as well as the `id`, `type` and `function` fields.
!!! Note

- The output format of the model is `content<|tool_calls|>tool_calls`, the delimiter is fixed, and the delimiter is used to determine whether it is a tool call.
- The tool call information contains the tool's `name` and `arguments` fields as well as the `id`, `type` and `function` fields.


### FunctionCall Output Flow
Expand Down
78 changes: 39 additions & 39 deletions docs/en/Best Practice/module.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ Module example deployed!
['[Module example get input: hello]', '[Module example get input: world]']
```

> **Note**: The test set is set by calling `evalset`, and there is no need to explicitly override any function. All `Modules` can have a test set.
!!! Note

The test set is set by calling `evalset`, and there is no need to explicitly override any function. All `Modules` can have a test set.

### Using the Built-in Registry

Expand All @@ -79,7 +81,7 @@ LazyLLM implements a registry for ``Modules``, which allows you to easily regist

#### Concept of Submodules

Similar to the ``Module`` class in ``pytorch``, the ``Module`` in LazyLLM also has a hierarchical concept, where a Module can have one or more ``Submodule``.
Similar to the ``Module`` class in PyTorch, the ``Module`` in LazyLLM also has a hierarchical concept, where a Module can have one or more ``Submodule``.
When using the ``update`` function to update a ``Module``, its ``Submodule`` will also be updated, unless explicitly set not to update the ``Submodule``.
Similarly, when using the ``start`` function to start the deployment task of a ``Module``, its ``Submodule`` will also be deployed, unless explicitly set not to deploy the ``Submodule``.
Here is an example:
Expand All @@ -99,38 +101,38 @@ You can make one ``Module`` a ``Submodule`` of another ``Module`` in the followi
>>> sm.submodules
[<Module type=MyModule name=m1>]

> **Note**:
>
> - When a flow is passed as a constructor argument to ``ActionModule`` or ``ServerModule``, any ``Module`` within it will also become a ``Submodule`` of the ``ActionModule`` or ``ServerModule``. Here's an example:
>
> >>> m1 = MyModule('m1')
> >>> m2 = MyModule('m2')
> >>> m3 = MyModule('m3')
> >>> am = lazyllm.ActionModule(lazyllm.pipeline(m1, lazyllm.parallel(m2, m3)))
> >>> am.submodules
> [<Module type=MyModule name=m1>, <Module type=MyModule name=m2>, <Module type=MyModule name=m3>]
> >>> sm = lazyllm.ServerModule(lazyllm.pipeline(m1, lazyllm.parallel(m2, m3)))
> >>> sm.submodules
> [<Module type=_ServerModuleImpl>]
> >>> sm.submodules[0].submodules
> [<Module type=Action return_trace=False sub-category=Flow type=Pipeline items=[]>
> └- <Flow type=Pipeline items=[]>
> |- <Module type=MyModule name=m1>
> └- <Flow type=Parallel items=[]>
> |- <Module type=MyModule name=m2>
> └- <Module type=MyModule name=m3>
> ]
>
> - When directly printing the ``repr`` of a ``Module``, it will display its hierarchical structure, including all its ``Submodules``. Continuing from the previous example:
>
> >>> sm
> <Module type=Server stream=False return_trace=False>
> └- <Module type=Action return_trace=False sub-category=Flow type=Pipeline items=[]>
> └- <Flow type=Pipeline items=[]>
> |- <Module type=MyModule name=m1>
> └- <Flow type=Parallel items=[]>
> |- <Module type=MyModule name=m2>
> └- <Module type=MyModule name=m3>
!!! Note

- When a flow is passed as a constructor argument to ``ActionModule`` or ``ServerModule``, any ``Module`` within it will also become a ``Submodule`` of the ``ActionModule`` or ``ServerModule``. Here's an example:

>>> m1 = MyModule('m1')
>>> m2 = MyModule('m2')
>>> m3 = MyModule('m3')
>>> am = lazyllm.ActionModule(lazyllm.pipeline(m1, lazyllm.parallel(m2, m3)))
>>> am.submodules
[<Module type=MyModule name=m1>, <Module type=MyModule name=m2>, <Module type=MyModule name=m3>]
>>> sm = lazyllm.ServerModule(lazyllm.pipeline(m1, lazyllm.parallel(m2, m3)))
>>> sm.submodules
[<Module type=_ServerModuleImpl>]
>>> sm.submodules[0].submodules
[<Module type=Action return_trace=False sub-category=Flow type=Pipeline items=[]>
└- <Flow type=Pipeline items=[]>
|- <Module type=MyModule name=m1>
└- <Flow type=Parallel items=[]>
|- <Module type=MyModule name=m2>
└- <Module type=MyModule name=m3>
]

- When directly printing the ``repr`` of a ``Module``, it will display its hierarchical structure, including all its ``Submodules``. Continuing from the previous example:

>>> sm
<Module type=Server stream=False return_trace=False>
└- <Module type=Action return_trace=False sub-category=Flow type=Pipeline items=[]>
└- <Flow type=Pipeline items=[]>
|- <Module type=MyModule name=m1>
└- <Flow type=Parallel items=[]>
|- <Module type=MyModule name=m2>
└- <Module type=MyModule name=m3>

2. Setting another ``Module`` as a member variable in a ``Module`` can make the other ``Module`` become its ``submodule``. Here is an example:

Expand Down Expand Up @@ -187,9 +189,7 @@ Module m2-2 m1-2 deployed!
Module m2-2 deployed!
```

> **Note**:
>
> It can be seen that when updating the ``ActionModule``, all its ``Submodules`` will be updated together. If there are deployment tasks, they will be executed after all the training/fine-tuning tasks are completed.
> Since parent modules may depend on submodules, submodules will be deployed first, followed by parent modules.
!!! Note

> **Note**: When the ``Redis`` service is configured, the lightweight gateway mechanism provided by LazyLLM can be used to achieve parallel deployment of all services.
- It can be seen that when updating the ``ActionModule``, all its ``Submodules`` will be updated together. If there are deployment tasks, they will be executed after all the training/fine-tuning tasks are completed. Since parent modules may depend on submodules, submodules will be deployed first, followed by parent modules.
- When the ``Redis`` service is configured, the lightweight gateway mechanism provided by LazyLLM can be used to achieve parallel deployment of all services.
Loading

0 comments on commit b2abae6

Please sign in to comment.