EOS dev env docs

[PVince81]

Adjust https://owncloud.github.io/ocis/eos/ with steps for anyone to setup EOS locally for debugging.

Using https://github.com/owncloud-docker/compose-playground/tree/master/examples/eos-compose/ or other.

This is crucial for everyone to be able to start debugging and testing against EOS.

• REQUIRES EOS in CI https://github.com/owncloud/ocis/issues/253 to be able to commit

@felixboehm @butonic @individual-it @micbar

[PVince81]

note: even if not fully complete it's ok to add some TODOs in the docs. maybe other people can help there as well

[butonic]

First step is #349 which lays the foundation using docker compose.

The idea is to use the eos-ocis-dev container to build the binary. Building a binary that has all necessary LDAP libs in place to look up a user using the os is tricky as it requires cgo. Getting rid of that is tracked in https://github.com/owncloud/ocis-reva/issues/331

Left tasks I see:

• update debugging.md to reflect how to use the eos-ocis-dev container to run the debug binary
  • for vscode
  • for the intellij go ide
  • for vim
• reorganize the debugging.md, development.md and eos.md under a common development section.

[PVince81]

Docs are in https://owncloud.github.io/ocis/eos/.

Some topics from me:

• clarify initial setup that one needs a local checkout of "ocis", "reva", etc

• clarify that one might need go.mod replaces and also the matching docker-compose volumes configured

• run make clean locally to make sure env is clean before running docker-compose to avoid compilation mix-up

• how to restart/recompile after changes ?

  • docker-compose exec ocis make (clean) build
  • docker-compose exec ocis ./bin/ocis kill reva-... then
  • docker-compose exec ocis ./bin/ocis run reva-

• part 2 "LDAP support" and part 3: is this optional or mandatory ? maybe add a sentence "if you need a different namespace for the frontend, do this". => seems to be required

• add section for running acceptance tests locally (API tests and Phoenix tests) against the containers

• add section about checking logs docker-compose logs -f ocis

• simplify steps, many commands to run manually (for technical reasons) could be improved later, integrate them into the booting process

• how to custom config, for example restarting with a specific chunk size: REVA_FRONTEND_UPLOAD_MAX_CHUNK_SIZE=$(( 10 * 1024 * 1024 )). I guess one needs to kill the matching reva service and restart it with the matching env var

• for proper cleanup, make sure to delete the "e" directory => redundant with Use separate volumes for EOS in docker-compose #477

[PVince81]

fancy idea: add targets to the makefile that runs "docker-compose up" and other commands like rebuilding inside the container :-)

[PVince81]

• cache go modules from deps to accelerate rebuilds (need the go cache folder on a volume?)

[PVince81]

@butonic @felixboehm see checkboxes above. Please raise tickets accordingly for the non-trivial tasks that need more time and future scheduling.

for the others I guess it could be done right away

[PVince81]

with latest ocis master + ocis-reva master + reva master there is some strange behavior:

• upload doesn't work and fails with a 500 error due to permission denied, see client upload of 5.7 GB fails with internal server error 500 product#128 (comment)
• when I login as einstein, I see the Webdav requests being done to 4c510ada-c86b-4815-8820-42cdf82c3d51, however in the web UI I see the contents of docker-compose exec ocis eos ls -l /eos/dockertest/reva/users/e/einstein despite the existence of the docker-compose exec ocis eos ls -l /eos/dockertest/reva/users/4/4c510ada-c86b-4815-8820-42cdf82c3d51/ folder. But the latter is empty.

So maybe there's a user / user id mismatch somewhere ??

[PVince81]

never mind, I just noticed that the docs have changed so I need to change my script as well to use the new layout...

[PVince81]

• document how to access the meta folders like the ones where the chunks are stored.
  the docker-compose exec mgm-master eos only gives access to the same files like in the web UI but not the meta ones.

=> uploads will land in the ocis docker inside the "/tmp" folder. can be checked with docker-compose exec ocis bash and going into "/tmp".
=> uploads are copied from that tmp folder with xrdcopy into EOS

[PVince81]

• the current EOS dev setup replicates the data multiple times so need enough free space (or TODO: fix the env!)

[PVince81]

• add hint that the ocis container will be compiling the go binaries the first time it is run, so need to wait before proceeding to the LDAP step

[PVince81]

• have the "e" directory and other local data be in another docker volume, because the local FS (like BTRFS) might cause unwanted effects with EOS => Use separate volumes for EOS in docker-compose #477

[PVince81]

I managed to solve my BTRFS issues by using docker volumes for all the EOS stuff, raised #477

[jnweiger]

* [ ]  add hint that the ocis container will be compiling the go binaries the first time it is run, so need to wait before proceeding to the LDAP step

I have good success with the following wait functions:

wait_for_ocis () {
  ## it compiles code upon first start. this can take ca 6 minutes.
  while true; do
    # expect to see "Starting server ... 0.0.0.0:9200" in the logs.
    docker-compose logs --tail=10 ocis
    if [ -n "\$(docker-compose logs ocis | grep 'Starting server' | grep 0.0.0.0:9200)" ]; then
      break
    fi
    echo " ... waiting for 0.0.0.0:9200 ..."
    sleep 15;
  done
}

wait_for_ldap () {
  # expect nslcd is running.
  while test -z "\$(docker-compose exec ocis ps -ef | grep nslcd | grep -v grep)"; do
    echo "waiting for nslcd ...";
    sleep 3;
  done
}

wait_for_eos_fst () {
  # expect (at least) four fst entries.
  while [ "\$(docker-compose exec ocis eos fs ls -m | grep host=fst | wc -l)" -lt 4 ]; do
    echo "waiting for four fst to appear in 'eos fs ls' ..."
    sleep 3;
    docker-compose exec ocis eos fs ls
  done
}

wait_for_eos_health () {
  echo "Expect to see 'online', 'ok', 'fine', 'default.0' here:"
  for i in 1 2 3 4 5 6 7 8 9 0; do
    # immediately after start, default.0 is shown as 0B free and 'full'
    docker-compose exec ocis eos health -a
    if [ -z "\$(docker-compose exec ocis eos health -m | grep status=full)" ]; then
      break     # nothing is full, carry on.
    fi
    sleep 2
  done
}

wait_for_ocis should be called before starting ldap. Then call wait_for_ldap before using ldap accounts, and so on....

[jnweiger]

With ocis-1.0.0rc3 and the instrutions at https://owncloud.github.io/ocis/eos/
eos does not work by default. We still need to manually bring it from offline state to online:

wait_for_eos_fst
# expect to see stat.active=online four times!
while [ "\$(docker-compose -f $compose_yml exec ocis eos fs ls -m | grep stat.active=online | wc -l)" -lt 4 ]; do
  docker-compose -f $compose_yml exec mgm-master eos space set default on
  sleep 5 
done
wait_for_eos_health

I just discovered that the rendered version and the github source are out of sync:

• https://owncloud.github.io/ocis/eos/ has 4 steps before 'verification'
• https://github.com/owncloud/ocis/blob/master/docs/ocis/eos.md has 6 steps before 'verification'

@butonic @micbar which of both is correct for use with docker-compose-eos-test.yml in v1.0.0-rc3 ?

[phil-davis]

@dpakach do you know the current working EOS setup and can help get docs up-to-date?

[jnweiger]

As discussed today with @butonic, both versions of the docs are outdated. We have identified a list of steps that I should not do, although they were documented. 😱 -> #815

[refs]

superseded by #815