Runtime PMC Meeting Notes 04.26.2018

Meeting Agenda:

1. ODPi Runtime v3 specification, next steps

From BigSQL team (Robin Noble) In terms of HDP 3.0 and Big SQL we are in the early stages of getting it to compile and run. The biggest change for us is in Hive 3.0 with ACID tables there are API changes, deprecated code, and package name changes that we had to make changes to Big SQL code to get it to build. Otherwise most of the base hadoop seems compatible with our existing code At this point we are not exploiting any of the new features in hadoop but we will be looking at Ozone filesystem New Features?

What are pieces they feel we need to cover in the spec. Here is best practices in deploying it. How to find the binaries?

New features: Ozone, container management in yarn, eraser coding in hdfs (traditionally makes 3 copies) a way to effectively put raid in your code 1.5 times instead of 3 but much less space.

How do you go about upgraded from v2 to v3.

2. What components to include? Ranger/Spark? Can we contain? Apache Hadoop® 3.0, including all maintenance releases. Apache Hadoop® 3.0, compatible filesystems (HCFS). Apache Hive™.

Hive, spark + security.

Maybe kafka after that.

3. HDP v3 new features Covered in #1

Questions

1. Ganesh, had been contributing to bigtop now toward odpi.
2. Looking into ambari as well.

Attendees:

• David (Saas)
• Alan Gates (Hortonworks)
• Raj Desai (IBM)
• Ganesh Raju (Linaro)