Hive 3 support

[massdosage]

Is your feature request related to a problem? Please describe.
Upgrade Waggle Dance to use Hive 3 (currently it supports Hive 1.x and 2.x).

Describe the solution you'd like
Waggle Dance able to proxy a Hive 3 metastore.

[massdosage]

If anyone would like to contribute support for Hive 3 we have a branch at https://github.com/HotelsDotCom/waggle-dance/tree/hive-3.x that has the Hive and Hadoop version numbers set to 3.x versions. Currently the code doesn't even compile but this could be a starting point for anyone who wants to get it working, we would happily review any pull requests against that branch that help get this all working.

[logikv]

Did you mean Apache Hive version or Cloudera Apache Hive version?
Apache Ambari Hive version is 3.1.2 (3.1.3rc0) and Cloudera already release CDH 7.1.4 with hive 3.1.3 version. Or you mean some other distr?
https://mvnrepository.com/artifact/org.apache.hive/hive-exec/3.1.3000.7.1.4.0-203
https://mvnrepository.com/artifact/org.apache.hive/hive-exec/3.1.2

[massdosage]

Waggle Dance uses Hive via the Apache Hive artifacts that are publicly available with the Maven Group id "org.apache.hive". It's all set up in the Maven POM files.

[LulaV14]

@massdosage is this still a valid issue?
If so I would like to give it a try on my free time if possible.

[massdosage]

@LulaV14 yes this is definitely still valid, it would be great if you could take a look. I can suggest some starting steps:

• Merge main into the hive-3.x branch I created.
• Update BeeJU to 5.0.0 once Upgrade Hive from 2 to 3 beeju#50 is merged and a release done (this should hopefully fix a set of issues related to test dependencies).
• Take a look through these changes which have upgraded various projects from Hive 2 to 3 to see the kinds of issues you may find (I'm sure WaggleDance will have plenty new ones too!):
  • Upgrade Hive from 2.3.7 to 3.1.2 HiveRunner/HiveRunner#126
  • Address hive 3.x related test issues HiveRunner/HiveRunner#125
  • Upgrade Hive from 2 to 3 beeju#50
  • https://github.com/ExpediaGroup/apiary-extensions/tree/hive-3.x

I'm happy to help and also to review PRs for changes into hive-3.x, the first two steps above should be fairly straightforward and will improve things in that branch already.

[LulaV14]

Nice! Thanks for the help @massdosage, I'll start with that 👍