[WIP] Hackathon merge #3013

garagatyi · 2016-11-08T09:54:33Z

What does this PR do?

Refactors workspace bootstrapping code:
- Addition of projects volumes to ws-machines and applying of agents moved to infrastructure provisioner. It can be overriden in any implementation of Che and will be responsible for providing all infrastructure needed for Che implementatipn functioning.
- Allow to configure environment network driver.
- Allow to add Agents from the code instead of special files. These agents can override agents configured in files if identifiers are equal.
- Adds possibility to host agents binaries from CHE master
- Reworks agents in a way when they can be downloaded from CHE master if they are not found on FS.
- Adds workspace ID to internal representation of environment to allow to use it for internal purposes of machine bootstrapping.
Renames some properties.
Refactors AgentConfigApplier to incapsulate additional code that was used with this class before.

Change-Id: I8383847bbba3b8932e4a10492642c67d37d4efb8 Signed-off-by: Florent BENOIT <[email protected]>

Change-Id: Ie21c57bf60bd482b2752e00bc1a8b08db5e71afd Signed-off-by: Florent BENOIT <[email protected]>

…r "mounted" path Change-Id: I5004b5344a708dc6985e76ded80edd033a747fc6 Signed-off-by: Florent BENOIT <[email protected]>

Change-Id: I4f1c3474387fe90f07a30a1af036773883e6c67e Signed-off-by: Florent BENOIT <[email protected]>

Signed-off-by: Oleksii Kurinnyi <[email protected]>

Signed-off-by: Alexander Garagatyi <[email protected]>

benoitf · 2016-11-08T10:00:51Z

...main/java/org/eclipse/che/plugin/docker/machine/local/LocalCheInfrastructureProvisioner.java

+import static java.lang.String.format;
+
+/**
+ * @author Alexander Garagatyi


There is nothing about what this class does

Yes, I'll fix that. It is because this PR is still in WIP

benoitf · 2016-11-08T10:02:54Z

.../che-core-api-agent/src/main/java/org/eclipse/che/api/agent/server/model/impl/AgentImpl.java

@@ -44,10 +44,10 @@ public AgentImpl(String id,
        this.name = name;
        this.version = version;
        this.description = description;
-        this.dependencies = dependencies;
-        this.properties = properties;
+        this.dependencies = dependencies != null ? dependencies : new ArrayList<>();


getters are doing this check so it looks odd to have it at both places
if it's immutable it should be Collection.emptyList()

firstNotNull?

Getters have a bug, I'll rewrite them. The bug is that they return mutable collection that doesn't influence their state.

codenvy-ci · 2016-11-08T12:03:54Z

Build # 951 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/951/ to view the results.

Signed-off-by: Alexander Garagatyi <[email protected]>

garagatyi · 2016-11-08T14:59:45Z

@skabashnyuk @evoevodin @mshaposhnik @sleshchenko Please review

mshaposhnik · 2016-11-08T15:09:05Z

.../che-core-commons-lang/src/main/java/org/eclipse/che/commons/lang/os/WindowsPathEscaper.java

+            // check and replace only occurrence of ":" after disk label on Windows host (e.g. C:/)
+            // but keep other occurrences it can be marker for docker mount volumes
+            // (e.g. /path/dir/from/host:/name/of/dir/in/container                                               )
+            esc = path.replaceFirst(":", "").replace('\\', '/');


Doesn't like File.separator :) ?

It's not my code. And I don't want to introduce here additional changes in this PR because it will delay merge significantly.

CHE-2454: add replacing of double underscores Add replacing of double underscores in environment variable name with single underscore in che application variable name. Signed-off-by: Alexander Garagatyi <[email protected]>

skabashnyuk · 2016-11-08T15:15:33Z

assembly/assembly-main/src/assembly/tomcat/conf/server.xml

@@ -139,6 +139,10 @@
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="false">

+
+        <!-- Provide Che Agent binaries (like workspace agent, terminal) -->
+        <Context docBase="${che.home}/lib" path="/agent-binaries" />


I propose to use separate folder for that. Let's call it /agent-binaries

separate folder by duplicating the existing binaries ? because agents are inside CHE_HOME/lib for now

I suppose that we can move agents binaries to any folder, no?

I propose to move agents away from /lib

I'm ok but it might take some time to validate this change no ?

@skabashnyuk are you ok if we postpone that and do after merging that PR?

skabashnyuk · 2016-11-08T15:17:43Z

.../che-core-commons-lang/src/main/java/org/eclipse/che/commons/lang/os/WindowsPathEscaper.java

+package org.eclipse.che.commons.lang.os;
+
+/**
+ * Escapes Windows path to unix-style path.


It is used in several places in around docker usage in both Che and Codenvy. I think it is time to move that code in a single place since it is bad practice to use the same private methods in several classes.

skabashnyuk · 2016-11-08T15:20:19Z

.../che-core-api-agent/src/main/java/org/eclipse/che/api/agent/server/model/impl/AgentImpl.java

@@ -44,10 +44,10 @@ public AgentImpl(String id,
        this.name = name;
        this.version = version;
        this.description = description;
-        this.dependencies = dependencies;
-        this.properties = properties;
+        this.dependencies = dependencies != null ? dependencies : new ArrayList<>();


firstNotNull?

codenvy-ci · 2016-11-08T16:54:03Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/957/

codenvy-ci · 2016-11-08T23:25:36Z

Build # 962 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/962/ to view the results.

Signed-off-by: Alexander Garagatyi <[email protected]>

codenvy-ci · 2016-11-09T12:09:47Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/966/

Signed-off-by: Vitalii Parfonov <[email protected]>

codenvy-ci · 2016-11-09T22:39:08Z

Build # 976 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/976/ to view the results.

Signed-off-by: Alexander Garagatyi <[email protected]>

codenvy-ci · 2016-11-10T08:36:12Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/977/

Signed-off-by: Alexander Garagatyi <[email protected]>

codenvy-ci · 2016-11-10T15:53:29Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/983/

Signed-off-by: Alexander Garagatyi <[email protected]>

codenvy-ci · 2016-11-11T16:25:48Z

Build # 997 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/997/ to view the results.

Refactors workspace bootstrapping code: Addition of projects volumes to ws-machines and applying of agents moved to infrastructure provisioner. It can be overwritten in any implementation of Che and will be responsible for providing all infrastructure needed for Che implementation functioning. Allow to configure environment network driver. Allow to add Agents from the code instead of special files. These agents can override agents configured in files if identifiers are equal. Adds possibility to host agents binaries from CHE master. Reworks agents in a way when they can be downloaded from CHE master if they are not found on FS. Adds workspace ID to internal representation of environment to allow to use it for internal purposes of machine bootstrapping. Renames some properties. Refactors AgentConfigApplier to encapsulate additional code that was used with this class before. Signed-off-by: Alexander Garagatyi <[email protected]>

benoitf and others added 16 commits October 16, 2016 10:20

introduce hackathon branch and version

2b3537e

Change-Id: I8383847bbba3b8932e4a10492642c67d37d4efb8 Signed-off-by: Florent BENOIT <[email protected]>

Provide agents on WSMaster with context /agent-binaries

338e8d9

Change-Id: Ie21c57bf60bd482b2752e00bc1a8b08db5e71afd Signed-off-by: Florent BENOIT <[email protected]>

Rework agent scripts to download the agents if agents are not on thei…

56c09bd

…r "mounted" path Change-Id: I5004b5344a708dc6985e76ded80edd033a747fc6 Signed-off-by: Florent BENOIT <[email protected]>

Use of CHE_API_ENDPOINT to guess url of WS master url to grab agents

07a4fcb

Change-Id: I4f1c3474387fe90f07a30a1af036773883e6c67e Signed-off-by: Florent BENOIT <[email protected]>

Merge branch 'master' into hackathon

366b94f

add rsync agent

ecad6a0

sort poms

b6fe453

fix CodeMirror version

19979af

Signed-off-by: Oleksii Kurinnyi <[email protected]>

Merge branch 'master' into hackathon

70c3e2f

Signed-off-by: Alexander Garagatyi <[email protected]>

Merge branch 'master' into hackathon

6276380

Signed-off-by: Alexander Garagatyi <[email protected]>

Merge branch 'master' into hackathon-fix

a1606e6

Signed-off-by: Alexander Garagatyi <[email protected]>

add small fixes

cd108e7

Signed-off-by: Alexander Garagatyi <[email protected]>

fix agents

c600fc8

Signed-off-by: Alexander Garagatyi <[email protected]>

fix agents

df59b26

Signed-off-by: Alexander Garagatyi <[email protected]>

CODENVY-1027: fix property names for docker registry

d09dd3c

Refactor workspace botstrapping infrastructure

d89ff97

Signed-off-by: Alexander Garagatyi <[email protected]>

garagatyi changed the title ~~Hackathon merge~~ [WIP] Hackathon merge Nov 8, 2016

benoitf reviewed Nov 8, 2016

View reviewed changes

garagatyi self-assigned this Nov 8, 2016

benoitf added the status/in-progress This issue has been taken by an engineer and is under active development. label Nov 8, 2016

fixup! Refactor workspace botstrapping infrastructure

bf966bc

Signed-off-by: Alexander Garagatyi <[email protected]>

mshaposhnik reviewed Nov 8, 2016

View reviewed changes

CHE-2454: add replacing of double underscores (#2664)

8ce1e4f

CHE-2454: add replacing of double underscores Add replacing of double underscores in environment variable name with single underscore in che application variable name. Signed-off-by: Alexander Garagatyi <[email protected]>

skabashnyuk reviewed Nov 8, 2016

View reviewed changes

Alexander Garagatyi added 2 commits November 9, 2016 01:11

Add fixes

4037072

Merge branch 'master' into hackathon-merge

84f57e7

skabashnyuk approved these changes Nov 9, 2016

View reviewed changes

skabashnyuk mentioned this pull request Nov 9, 2016

cannot use codenvy.sh via symlink codenvy/codenvy#1118

Closed

Alexander Garagatyi and others added 2 commits November 9, 2016 13:01

add fix

5f12a55

Signed-off-by: Alexander Garagatyi <[email protected]>

change version to docker

4b30906

Fix build: moving DTO classes to ide-api

a916b01

Signed-off-by: Vitalii Parfonov <[email protected]>

l0rd mentioned this pull request Nov 9, 2016

OpenShift API implementation #3031

Closed

Alexander Garagatyi added 2 commits November 9, 2016 23:40

Merge branch 'docker' into hackathon-merge

2b896cb

Merge branch 'master' into hackathon-merge

6427a41

Alexander Garagatyi added 3 commits November 10, 2016 09:49

fxi

0f423e8

Signed-off-by: Alexander Garagatyi <[email protected]>

set version

6b7edad

Signed-off-by: Alexander Garagatyi <[email protected]>

Merge branch 'docker' into hackathon-merge

d3923c2

Signed-off-by: Alexander Garagatyi <[email protected]>

benoitf mentioned this pull request Nov 10, 2016

Problem with mounting in docker while che starts from the external physical hard drive #3027

Closed

add fix

59b3816

Signed-off-by: Alexander Garagatyi <[email protected]>

fix

703fb33

Signed-off-by: Alexander Garagatyi <[email protected]>

garagatyi merged commit 0c81935 into master Nov 11, 2016

garagatyi deleted the hackathon-merge branch November 11, 2016 16:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Hackathon merge #3013

[WIP] Hackathon merge #3013

garagatyi commented Nov 8, 2016

benoitf Nov 8, 2016

garagatyi Nov 8, 2016

benoitf Nov 8, 2016

skabashnyuk Nov 8, 2016

garagatyi Nov 8, 2016

codenvy-ci commented Nov 8, 2016

garagatyi commented Nov 8, 2016

mshaposhnik Nov 8, 2016

garagatyi Nov 8, 2016 •

edited

Loading

skabashnyuk Nov 8, 2016

benoitf Nov 8, 2016

garagatyi Nov 8, 2016

skabashnyuk Nov 8, 2016

benoitf Nov 8, 2016

garagatyi Nov 8, 2016

skabashnyuk Nov 9, 2016

skabashnyuk Nov 8, 2016

garagatyi Nov 8, 2016

skabashnyuk Nov 8, 2016

codenvy-ci commented Nov 8, 2016

codenvy-ci commented Nov 8, 2016

codenvy-ci commented Nov 9, 2016

codenvy-ci commented Nov 9, 2016

codenvy-ci commented Nov 10, 2016

codenvy-ci commented Nov 10, 2016

codenvy-ci commented Nov 11, 2016

[WIP] Hackathon merge #3013

[WIP] Hackathon merge #3013

Conversation

garagatyi commented Nov 8, 2016

What does this PR do?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codenvy-ci commented Nov 8, 2016

garagatyi commented Nov 8, 2016

Choose a reason for hiding this comment

garagatyi Nov 8, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codenvy-ci commented Nov 8, 2016

codenvy-ci commented Nov 8, 2016

codenvy-ci commented Nov 9, 2016

codenvy-ci commented Nov 9, 2016

codenvy-ci commented Nov 10, 2016

codenvy-ci commented Nov 10, 2016

codenvy-ci commented Nov 11, 2016

garagatyi Nov 8, 2016 •

edited

Loading