Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-1474. ozone.scm.datanode.id config should take path for a dir #792

Merged
merged 5 commits into from
May 9, 2019
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Fix review comments
  • Loading branch information
vivekratnavel committed May 8, 2019

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 8645e4312ad7ec8af3f457f2dd3fcbaae24a32d1
Original file line number Diff line number Diff line change
@@ -299,7 +299,7 @@ public final class ScmConfigKeys {
// The path where datanode ID is to be written to.
// if this value is not set then container startup will fail.
public static final String OZONE_SCM_DATANODE_ID_DIR =
"ozone.scm.datanode.id";
"ozone.scm.datanode.id.dir";

public static final String OZONE_SCM_DB_CACHE_SIZE_MB =
"ozone.scm.db.cache.size.mb";
2 changes: 1 addition & 1 deletion hadoop-hdds/common/src/main/resources/ozone-default.xml
Original file line number Diff line number Diff line change
@@ -883,7 +883,7 @@
</description>
</property>
<property>
<name>ozone.scm.datanode.id</name>
<name>ozone.scm.datanode.id.dir</name>
<value/>
<tag>OZONE, MANAGEMENT</tag>
<description>The path that datanodes will use to store the datanode ID.
Original file line number Diff line number Diff line change
@@ -397,7 +397,7 @@ public void testDatanodeStateMachineWithInvalidConfiguration()
/** Port out of range **/
confList.add(Maps.immutableEntry(
ScmConfigKeys.OZONE_SCM_NAMES, "scm:123456"));
// Invalid ozone.scm.datanode.id
// Invalid ozone.scm.datanode.id.dir
/** Empty **/
confList.add(Maps.immutableEntry(
ScmConfigKeys.OZONE_SCM_DATANODE_ID_DIR, ""));
4 changes: 2 additions & 2 deletions hadoop-hdds/docs/content/Settings.md
Original file line number Diff line number Diff line change
@@ -105,14 +105,14 @@ Here is an example,
</property>
{{< /highlight >}}

* **ozone.scm.datanode.id** Data nodes generate a Unique ID called Datanode
* **ozone.scm.datanode.id.dir** Data nodes generate a Unique ID called Datanode
ID. This identity is written to the file datanode.id in a directory specified by this path. *Data nodes
will create this path if it doesn't exist already.*

Here is an example,
{{< highlight xml >}}
<property>
<name>ozone.scm.datanode.id</name>
<name>ozone.scm.datanode.id.dir</name>
<value>/data/disk1/meta/node/datanode.id</value>
</property>
{{< /highlight >}}
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.scm.client.address=scm
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@ OZONE-SITE.XML_ozone.om.address.om3=om3
OZONE-SITE.XML_ozone.om.ratis.enable=true
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.handler.type=distributed
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.recon.db.dir=/data/metadata/recon
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozone/docker-config
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.handler.type=distributed
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.handler.type=distributed
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozonefs/docker-config
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.handler.type=distributed
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozoneperf/docker-config
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.scm.client.address=scm
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/main/compose/ozones3/docker-config
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.scm.client.address=scm
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@ CORE-SITE.XML_fs.defaultFS=hdfs://namenode:9000
OZONE-SITE.XML_ozone.ksm.address=ksm
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.block.client.address=scm
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.handler.type=distributed
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.om.http-address=om:9874
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=True
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.handler.type=distributed
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@
OZONE-SITE.XML_ozone.om.address=om
OZONE-SITE.XML_ozone.scm.names=scm
OZONE-SITE.XML_ozone.enabled=true
OZONE-SITE.XML_ozone.scm.datanode.id=/data
OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data
OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.scm.client.address=scm
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ metadata:
data:

OZONE-SITE.XML_hdds.datanode.dir: "/data/storage"
OZONE-SITE.XML_ozone.scm.datanode.id: "/data"
OZONE-SITE.XML_ozone.scm.datanode.id.dir: "/data"
OZONE-SITE.XML_ozone.metadata.dirs: "/data/metadata"
OZONE-SITE.XML_ozone.scm.block.client.address: "scm-0.scm"
OZONE-SITE.XML_ozone.om.address: "om-0.om"
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ metadata:
name: config
data:
OZONE-SITE.XML_hdds.datanode.dir: /data/storage
OZONE-SITE.XML_ozone.scm.datanode.id: /data/datanode.id
OZONE-SITE.XML_ozone.scm.datanode.id.dir: /data/datanode.id
Copy link
Contributor

@anuengineer anuengineer May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to comment so late, is this wrong ? Shouldn't this be /data instead of /data/datanode.id ?

OZONE-SITE.XML_ozone.metadata.dirs: /data/metadata
OZONE-SITE.XML_ozone.scm.block.client.address: scm-0.scm
OZONE-SITE.XML_ozone.om.address: om-0.om
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ metadata:
name: config
data:
OZONE-SITE.XML_hdds.datanode.dir: /data/storage
OZONE-SITE.XML_ozone.scm.datanode.id: /data/datanode.id
OZONE-SITE.XML_ozone.scm.datanode.id.dir: /data/datanode.id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same , /data instead of /data/datanode.id?

OZONE-SITE.XML_ozone.metadata.dirs: /data/metadata
OZONE-SITE.XML_ozone.scm.block.client.address: scm-0.scm
OZONE-SITE.XML_ozone.om.address: om-0.om
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ metadata:
name: config
data:
OZONE-SITE.XML_hdds.datanode.dir: /data/storage
OZONE-SITE.XML_ozone.scm.datanode.id: /data/datanode.id
OZONE-SITE.XML_ozone.scm.datanode.id.dir: /data/datanode.id
OZONE-SITE.XML_ozone.metadata.dirs: /data/metadata
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here ..thx

OZONE-SITE.XML_ozone.scm.block.client.address: scm-0.scm
OZONE-SITE.XML_ozone.om.address: om-0.om