Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some validation for remove_job_data in the executor server #468

Merged
merged 2 commits into from
Nov 1, 2022
Merged

Add some validation for remove_job_data in the executor server #468

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
daf9a10
Add some validation for remove_job_data in the executor server
kyotoYaho Oct 27, 2022
08886aa
Fix for comments of subdirectory check
kyotoYaho Oct 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 36 additions & 4 deletions ballista/executor/src/executor_server.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -720,13 +720,45 @@ impl<T: 'static + AsLogicalPlan, U: 'static + AsExecutionPlan> ExecutorGrpc
request: Request<RemoveJobDataParams>,
) -> Result<Response<RemoveJobDataResult>, Status> {
let job_id = request.into_inner().job_id;
info!("Remove data for job {:?}", job_id);

// Verify whether it's a legal job id
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the redundant scope?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unnecessary. Just for making the verification steps more clearly

if job_id.is_empty() {
return Err(Status::internal(
"Job id should not be empty!!!".to_string(),
));
}
if job_id.contains('.') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to prevent files with extensions? Or parent directories?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's for prevent deleting parent directory. Generally, a directory should not contain '.'

return Err(Status::internal(format!(
"Job id {} should not contain char '.'!!!",
job_id
)));
}
}

let work_dir = self.executor.work_dir.clone();
let mut path = PathBuf::from(work_dir);
path.push(job_id.clone());
if path.is_dir() {
info!("Remove data for job {:?}", job_id);
std::fs::remove_dir_all(&path)?;
path.push(job_id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the work_dir subdirectory check... that's the critical vulnerability here.


// Verify whether the path is for an existing directory
{
if !path.exists() {
return Err(Status::internal(format!(
"Path {:?} does not exist!!!",
path
)));
}
if !path.is_dir() {
return Err(Status::internal(format!(
"Path {:?} is not for a directory!!!",
path
)));
}
}

std::fs::remove_dir_all(&path)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the docs we have a vulnerability here:

if self has a verbatim prefix (e.g. \\?\C:\windows) and path is not empty, the new path is normalized: all references to . and .. are removed.

I can bypass the . string check on Windows by doing a \\?. It is impossible to validate directories at a string level. The missing check is to fully normalize this directory, then check if the normalized directory is a subdirectory of the work_dir.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. I'll refine it to use a more standard and robust way for the subdirectory check.


Ok(Response::new(RemoveJobDataResult {}))
}
}