Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: refactor compile & installation #2251

Merged
merged 1 commit into from
Sep 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 2 additions & 37 deletions docs/directory.json
Original file line number Diff line number Diff line change
Expand Up @@ -87,26 +87,7 @@
},
{
"title": "安装与部署",
"path": "installation",
"children": [
{
"title": "编译安装",
"children": [
{
"title": "源码编译",
"path": "operation/compile/compile"
},
{
"title": "交叉编译",
"path": "operation/compile/cross-compile"
},
{
"title": "按需编译",
"path": "operation/compile/features"
}
]
}
]
"path": "installation"
},
{
"title": "规则",
Expand Down Expand Up @@ -783,23 +764,7 @@
},
{
"title": "Installation",
"path": "installation",
"children": [
{
"title": "Build from Source Code",
"path": "operation/compile/compile",
"children": [
{
"title": "Cross Compile",
"path": "operation/compile/cross-compile"
},
{
"title": "Compile selected features only",
"path": "operation/compile/features"
}
]
}
]
"path": "installation"
},
{
"title": "Rules",
Expand Down
4 changes: 3 additions & 1 deletion docs/en_US/extension/external/external_func.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ The json configuration file includes the following two parts:

- about: Used to describe the Meta-information of service, including author, detailed description, help document url, etc. For detailed usage, please refer to the example below.
- interfaces: Used to define a set of service interfaces. Services provided by the same server often have the same service address and can be used as a service interface. Each service interface contains the following attributes:
- protocol: The protocol used by the service. "grpc", "rest" are supported currently. The "msgpack-rpc" is not built by default, you need to build it with build tag "msgpack" by yourself. Please refer to [feature compilation](../../operation/compile/features.md#usage) for detail.
- protocol: The protocol used by the service. "grpc", "rest" are supported currently. The "msgpack-rpc" is not built
by default, you need to build it with build tag "msgpack" by yourself. Please refer
to [feature compilation](../../installation.md#compile-with-selected-features) for detail.
- address: Service address, which must be url. For example, typical rpc service address: "tcp://localhost:50000" or http service address "https://localhost:8000".
- schemaType: The type of service description file. Only "protobuf" is supported currently .
- schemaFile: service description file, currently only proto file is supported. The rest and msgpack services also need to be described in proto.
Expand Down
88 changes: 87 additions & 1 deletion docs/en_US/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,93 @@ When installing by package, the eKuiper folders are not in the same directory. T
$ bin/kuiperd
```

eKuiper allows to tailor the binary in compilation to get a customized feature set. As written by go, it also allows cross compilation. For detail, please check [compilation](./operation/compile/compile.md).
### Compile Packages and Docker Images

- Packages: `$ make pkg`
- Packages files that support EdgeX: `$ make pkg_with_edgex`
- Docker images: `$ make docker`

> Notice that: Docker images support EdgeX by default

### Cross-compile binaries

Go supports cross-compiling binaries for multiple platforms which applies to eKuiper as well. Because eKuiper depends on
sqlite, CGO_ENABLED must be set to 1 which requires to install and specify the gcc of the target system.

- Install the GNU toolchain/gcc of the target system.
- Modify the Makefile to specify `GOOS`, `GOARCH` and `CC` and then build.

For example, to cross build ARM64 binaries in AMD64 ubuntu/debian machine, do these steps:

1. Install the GNU toolchain/gcc of the target system ARM64

```shell
apt-get install gcc-aarch64-linux-gnu
```

2. Update the Makefile in the build command. Examples:

```shell
GO111MODULE=on CGO_ENABLED=1 GOOS=linux GOARCH=arm64 CC=aarch64-linux-gnu-gcc go build -trimpath -ldflags="-s -w -X main.Version=$(VERSION) -X main.LoadFileType=relative" -o kuiperd cmd/kuiperd/main.go
```

3. Run `make`

### Compile with Selected Features

eKuiper allows tailoring the binary in compilation to get a customized feature set.
As written by go, it also allows cross compilation.
Except core runtime and REST api,
there are some features
that are allowed
to be enabled or disabled during compilation
by [go build constraints](https://pkg.go.dev/go/build#hdr-Build_Constraints).
Uses can customize the built binary
to include only the desired features to reduce the binary size according to the limit of the target environment.

| Feature | Build Tag | Description |
|-----------------------------------------------------------------------------------------------|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
| Core | core | The core of eKuiper. It contains the processor and REST API for stream/table/rule, the configuration processing, the SQL parser, the rule runtime etc. |
| [CLI](./api/cli/overview.md) | rpc | The cli server |
| [EdgeX Foundry integration](./edgex/edgex_rule_engine_tutorial.md) | edgex | The built-in edgeX source, sink and connection |
| [Native plugin](./extension/native/overview.md) | plugin | The native plugin runtime, REST API, CLI API etc. |
| [Portable plugin](./extension/portable/overview.md) | portable | The portable plugin runtime, REST API, CLI API etc. |
| [External service](./extension/external/external_func.md) | service | The external service runtime, REST API, CLI API etc. |
| [Msgpack-rpc External service](./extension/external/external_func.md) | msgpack | Support msgpack-rpc protocol in external service |
| [UI Meta API](./operation/manager-ui/overview.md) | ui | The REST API of the metadata which is usually consumed by the ui |
| [Prometheus Metrics](./configuration/global_configurations.md#prometheus-configuration) | prometheus | Support to send metrics to prometheus |
| [Extended template functions](./guide/sinks/data_template.md#functions-supported-in-template) | template | Support additional data template function from sprig besides default go text/template functions |
| [Codecs with schema](./guide/serialization/serialization.md) | schema | Support schema registry and codecs with schema such as protobuf |

In makefile, we already provide three feature sets: standard, edgeX and core. The standard feature set include all
features in the list except edgeX; edgeX feature set include all features; And the core feature set is the minimal which
only has core feature. Build these feature sets with default makefile:

```shell
# standard
make
# EdgeX
make build_with_edgex
# core
make build_core
```

Feature selection is useful in a limited resource target which is unlikely to run as docker container. So we only
provide standard feature set in the docker images.

And users need to build from source to customize the feature sets. To build with the desired features:

```shell
go build --tags "<FEATURE>"
```

For example, to build with core and native plugin support:

```shell
go build --tags "core plugin"
```

Recommend updating the build command in the Makefile with tags and building from make.

## Installation structure

Expand Down
17 changes: 0 additions & 17 deletions docs/en_US/operation/compile/compile.md

This file was deleted.

22 changes: 0 additions & 22 deletions docs/en_US/operation/compile/cross-compile.md

This file was deleted.

48 changes: 0 additions & 48 deletions docs/en_US/operation/compile/features.md

This file was deleted.

3 changes: 2 additions & 1 deletion docs/zh_CN/extension/external/external_func.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ json 配置文件包括以下两个部分:

- about: 用于描述服务的元信息,包括作者,详细描述,帮助文档 url 等。详细用法请参考下面的范例。
- interfaces: 用于定义一组服务接口。同一个服务器提供的服务往往具有相同的服务地址,可作为一个服务接口。每一个服务接口包含下列属性:
- protocol: 服务采用的协议。目前支持 "grpc", "rest"。需要注意的是, "msgpack-rpc" 服务没有编译到默认的 eKuiper 中,需要添加 build tag "msgpack" 并自行编译。详情请参考[按需编译](../../operation/compile/features.md#使用)。
- protocol: 服务采用的协议。目前支持 "grpc", "rest"。需要注意的是, "msgpack-rpc" 服务没有编译到默认的 eKuiper 中,需要添加
build tag "msgpack" 并自行编译。详情请参考[按需编译](../../installation.md#按需编译功能)。
- adddress: 服务地址,必须为 url。例如,典型 rpc 服务地址:"tcp://localhost:50000" 或者 http 服务地址 "<https://localhost:8000"。>
- schemaType: 服务描述文件类型。目前仅支持 "protobuf"。
- schemaFile: 服务描述文件,目前仅支持 proto 文件。rest 和 msgpack 服务也需要采用 proto 描述。
Expand Down
Loading