Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support Cassandra with docker-compose in server #2307

Merged
merged 10 commits into from
Oct 23, 2023
Merged
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,5 @@ hs_err_pid*
.mtj.tmp/
# blueJ files
*.ctxt

*swagger-ui*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems no need to add here due to the master code has fixed it (#2277)

Copy link
Member

@imbajin imbajin Sep 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and also need handle the CI error (if we used refined github plugin)
image

5 changes: 2 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,10 @@ RUN set -x \
# 2. Init HugeGraph Sever
RUN set -e \
&& pwd && cd /hugegraph/ \
&& sed -i "s/^restserver.url.*$/restserver.url=http:\/\/0.0.0.0:8080/g" ./conf/rest-server.properties \
&& ./bin/init-store.sh
&& sed -i "s/^restserver.url.*$/restserver.url=http:\/\/0.0.0.0:8080/g" ./conf/rest-server.properties

EXPOSE 8080
VOLUME /hugegraph

ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["./bin/start-hugegraph.sh", "-d false -j $JAVA_OPTS -g zgc"]
CMD ["./bin/docker-entrypoint.sh"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we shouldn't put the docker files in general path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/JanusGraph/janusgraph/tree/master/janusgraph-dist/docker
maybe I can try to move the docker related file in a split folder in hugegraph-dist, like janusgraph?

44 changes: 44 additions & 0 deletions hugegraph-dist/docker/example/docker-compose-cassandra.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
version: "3"

services:
graph:
image: hugegraph/hugegraph
container_name: ca-graph
aroundabout marked this conversation as resolved.
Show resolved Hide resolved
ports:
- 18080:8080
environment:
hugegraph.backend: cassandra
hugegraph.serializer: cassandra
hugegraph.cassandra.host: ca-cassandra
aroundabout marked this conversation as resolved.
Show resolved Hide resolved
hugegraph.cassandra.port: 9042
networks:
- ca-network
depends_on:
- cassandra
healthcheck:
test: ["CMD", "bin/gremlin-console.sh", "--" ,"-e", "scripts/remote-connect.groovy"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still need -e?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://tinkerpop.apache.org/docs/3.6.2-SNAPSHOT/reference/#execution-mode
-e is for the gremlin console to use the execution mode to execute scripts.

interval: 10s
timeout: 30s
retries: 3

cassandra:
image: cassandra:3.11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try C* 4 for it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested, no problem
image

container_name: ca-cassandra
aroundabout marked this conversation as resolved.
Show resolved Hide resolved
ports:
- 7000:7000
- 9042:9042
security_opt:
- seccomp:unconfined
networks:
- ca-network
healthcheck:
test: ["CMD", "cqlsh", "--execute", "describe keyspaces;"]
interval: 10s
timeout: 30s
retries: 5

networks:
ca-network:

volumes:
hugegraph-data:
24 changes: 24 additions & 0 deletions hugegraph-dist/src/assembly/static/bin/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with this
# work for additional information regarding copyright ownership. The ASF
# licenses this file to You under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#


./bin/wait-storage.sh

./bin/init-store.sh

./bin/start-hugegraph.sh -d false -j "$JAVA_OPTS" -g zgc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the params here? could we get the value of $JAVA_OPTS?

Copy link
Contributor Author

@aroundabout aroundabout Sep 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JAVA_OPTS is an env var which is set in dockerfile. And this shell is only used in dockerfile, hence I think this way is available.

10 changes: 3 additions & 7 deletions hugegraph-dist/src/assembly/static/bin/gremlin-console.sh
Original file line number Diff line number Diff line change
Expand Up @@ -84,14 +84,10 @@ PROFILING_ENABLED=false

# Process options
MAIN_CLASS=org.apache.tinkerpop.gremlin.console.Console
while getopts "elpv" opt; do
while getopts "lpv" opt; do
case "$opt" in
e) MAIN_CLASS=org.apache.tinkerpop.gremlin.groovy.jsr223.ScriptExecutor
# Stop processing gremlin-console.sh arguments as soon as the -e switch
# is seen; everything following -e becomes arguments to the
# ScriptExecutor main class. This maintains compatibility with
# older deployments.
break;;
# class ScriptExecutor has been Deprecated.
# reference https://tinkerpop.apache.org/javadocs/3.2.3/full/org/apache/tinkerpop/gremlin/groovy/jsr223/ScriptExecutor.html
l) eval GREMLIN_LOG_LEVEL=\$$OPTIND
OPTIND="$(( $OPTIND + 1 ))"
if [ "$GREMLIN_LOG_LEVEL" = "TRACE" -o \
Expand Down
55 changes: 55 additions & 0 deletions hugegraph-dist/src/assembly/static/bin/wait-storage.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with this
# work for additional information regarding copyright ownership. The ASF
# licenses this file to You under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#

function abs_path() {
SOURCE="${BASH_SOURCE[0]}"
while [[ -h "$SOURCE" ]]; do
DIR="$(cd -P "$(dirname "$SOURCE")" && pwd)"
SOURCE="$(readlink "$SOURCE")"
[[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE"
done
cd -P "$(dirname "$SOURCE")" && pwd
}

BIN=$(abs_path)
TOP="$(cd "$BIN"/../ && pwd)"
GRAPH_PROP="$TOP/conf/graphs/hugegraph.properties"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer GRAPH_CONF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes have been applied on the var name

HUGE_STORAGE_TIMEOUT_S=60
TRY_STORAGE="$TOP/scripts/try-storage.groovy"

. "$BIN"/util.sh

# apply config from env
while IFS=' ' read -r envvar_key envvar_val; do
if [[ "${envvar_key}" =~ hugegraph\. ]] && [[ ! -z ${envvar_val} ]]; then
envvar_key=${envvar_key#"hugegraph."}
if grep -q -E "^\s*${envvar_key}\s*=\.*" ${GRAPH_PROP}; then
sed -ri "s#^(\s*${envvar_key}\s*=).*#\\1${envvar_val}#" ${GRAPH_PROP}
else
echo "${envvar_key}=${envvar_val}" >> ${GRAPH_PROP}
fi
else
continue
fi
done < <(env | sort -r | awk -F= '{ st = index($0, "="); print $1 " " substr($0, st+1) }')

# wait for storage
if ! [ -z "${HUGE_STORAGE_TIMEOUT_S:-}" ]; then
timeout "${HUGE_STORAGE_TIMEOUT_S}s" bash -c \
"until bin/gremlin-console.sh -- -e $TRY_STORAGE > /dev/null 2>&1; do echo \"waiting for storage...\"; sleep 5; done"
fi
19 changes: 19 additions & 0 deletions hugegraph-dist/src/assembly/static/scripts/remote-connect.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/

:remote connect tinkerpop.server conf/remote.yaml
:> hugegraph
14 changes: 14 additions & 0 deletions hugegraph-dist/src/assembly/static/scripts/try-storage.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import org.apache.hugegraph.HugeFactory
import org.apache.hugegraph.dist.RegisterUtil

// register all the backend to avoid changes if docker needs to support othre backend
RegisterUtil.registerPlugins()
RegisterUtil.registerRocksDB()
RegisterUtil.registerCassandra()
RegisterUtil.registerScyllaDB()
RegisterUtil.registerHBase()
RegisterUtil.registerMysql()
RegisterUtil.registerPalo()
RegisterUtil.registerPostgresql()

graph = HugeFactory.open('./conf/graphs/hugegraph.properties')