HdfsHadoop

Setup fully distributed Hbase cluster

Build Images:

1/ Hdfs:

docker build -f Dockerfile_hdfs -t foodytechnologies/hdfs-openjdk8-alpine .

2/ Hbase:

docker build -f Dockerfile_hbase -t foodytechnologies/hbase-openjdk8-alpine .

prerequisites:

• Create a new network :
  docker network create -d bridge hdfshadoop_network
  
• Create a Volume to keep data and meta-data persistant:
  docker volume create hdfs_data
  
• Launch zookeeper servers: they will be used for Hdfs and Hbase components
  docker-compose -f composers/hdfs/zookeeper.yml up -d

Launch Hdfs and Hbase:

1. We have to launch Hdfs components in the order

• Launch journal nodes:
  docker-compose -f composers/hdfs/journalnode.yml up -d
  
• Launch name nodes:
  docker-compose -f composers/hdfs/namenodes up -d
  
• Launch data nodes:
  docker-compose -f composers/hdfs/datanodenode.yml up -d
  

2. Quick Check that namenodes are started
docker exec -ti NN1 hdfs getconf -confKey dfs.ha.namenodes.databootcamp
(shall display: n1,n2,n3,n4)
docker exec -ti NN1 hdfs getconf -confKey dfs.ha.namenodes.databootcamp
(shall display: n1,n2,n3,n4)
docker exec -ti NN1 hdfs haadmin -getServiceState nn1
(shall display: active)
docker exec -ti NN1 hdfs haadmin -getServiceState nn2 (same for n3 and n4)
(shall display: standby)

3. Quick Check that services are started: by executin jps commands on each started container

4. We have to launch Hbase components
docker-compose -f composers/hbase/hbase.yml up -d

5. Check that all is started correctly by connecting to hbase using shell and verifying status

$ docker exec -ti rs1 bash
bash-4.3$ hbase shell
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.2.0, rUnknown, Tue Jun 11 04:30:30 UTC 2019
Took 0.0035 seconds
hbase(main):001:0> status
1 active master, 1 backup masters, 3 servers, 0 dead, 1.6667 average load
Took 0.4977 seconds