Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite parameter generator in Julia #83

Closed
6 of 10 tasks
szarnyasg opened this issue May 18, 2019 · 2 comments
Closed
6 of 10 tasks

Rewrite parameter generator in Julia #83

szarnyasg opened this issue May 18, 2019 · 2 comments

Comments

@szarnyasg
Copy link
Member

szarnyasg commented May 18, 2019

I am currently learning Julia and will use it to take a stab at issue #51, i.e. I'll rewrite the parameter generator (currently implemented in Python 2) in Julia.

I understand that Julia is still a niche language, so users might not be able to trivially install it on their systems. To mitigate this, I'll update the Docker image to support Julia.

The current Docker image is based on Alpine Linux which doesn't have a maintainer for recent Julia releases. Alpine uses musl libc, which has only Tier 3 support in Julia, which might include test failures (e.g. see the issue that has only been resolved 11 days ago). Therefore I'd just move from openjdk:8-jdk-alpine to the Debian-based openjdk:8-jdk-stretch. This also has an old Julia version, but it's possible to install with

wget -q https://julialang-s3.julialang.org/bin/linux/x64/1.1/julia-1.1.1-linux-x86_64.tar.gz
tar xf julia-1.1.1-linux-x86_64.tar.gz
export PATH=$PATH:`pwd`/julia-1.1.1/bin
julia

The work happens on the rework-paramgen branch.

@szarnyasg
Copy link
Member Author

Since the generated parameters for BI queries have changed since switching to Hadoop 3.2. and parameter tuning comes at a later stage of benchmark design, we can take a bit more relaxed approach with this task: Interactive parameters should be same (or as close as possible) but BI parameters are subject to change.

@szarnyasg
Copy link
Member Author

Paramgen is going to be reimplemented in SQL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant