box_rescale function to rescale bounding boxes to a new image shape

[tflahaul]

Added a small function (+ doc) in the ops module to rescale bounding boxes to a new image shape.

(It's my first time contributing so I hope I'm doing it right)

[facebook-github-bot]

Hi @tflahaul!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

[datumbox]

Thanks for the PR @tflahaul, welcome to TorchVision!

Since it's your first contribution to the project, I'll just mention a couple of best-practices that maximize your chances of getting your PRs merged:

• When introducing a new major feature, it's often good to create first an issue and provide more information on your use-case. This will help other maintainers give feedback, offer alternatives or help you design the API.
• All new operators and public methods need to be covered by unit-tests, have proper documentation (which you do) and make use of proper typing (which again you do fine).

Given the above, could you add a brief description on your PR to explain what you would like to do? We are in the process of updating our detection transforms so noting your use-case will be helpful.

[tflahaul]

hey thanks for your feedback @datumbox ! And sorry for the messy PR haha, I'll follow your advices next time.

Here's some context: I made this function for my detection script because the model needed square images as input. But the bounding boxes outputted by the model were then based on the squared images and I had to rescale them to their original image sizes before drawing them. It's a very short & simple function but I thought it would be useful to many other.

The function multiplies the x and y's by their respective width and height ratios

Hope this makes more sense, have a great day!

[datumbox]

@tflahaul No worries! Thanks for the context.

TorchVision already has a similar method here:

vision/torchvision/models/detection/transform.py

Lines 279 to 293 in b060163

	def resize_boxes(boxes, original_size, new_size):
	# type: (Tensor, List[int], List[int]) -> Tensor
	ratios = [
	torch.tensor(s, dtype=torch.float32, device=boxes.device) /
	torch.tensor(s_orig, dtype=torch.float32, device=boxes.device)
	for s, s_orig in zip(new_size, original_size)
	]
	ratio_height, ratio_width = ratios
	xmin, ymin, xmax, ymax = boxes.unbind(1)
	xmin = xmin * ratio_width
	xmax = xmax * ratio_width
	ymin = ymin * ratio_height
	ymax = ymax * ratio_height
	return torch.stack((xmin, ymin, xmax, ymax), dim=1)

It also offers transforms that handle masks, boxes, keypoints etc:

vision/torchvision/models/detection/transform.py

Lines 150 to 173 in b060163

	def resize(self,
	image: Tensor,
	target: Optional[Dict[str, Tensor]] = None,
	) -> Tuple[Tensor, Optional[Dict[str, Tensor]]]:
	h, w = image.shape[-2:]
	if self.training:
	size = float(self.torch_choice(self.min_size))
	else:
	# FIXME assume for now that testing uses the largest scale
	size = float(self.min_size[-1])
	image, target = _resize_image_and_masks(image, size, float(self.max_size), target, self.fixed_size)
	if target is None:
	return image, target
	bbox = target["boxes"]
	bbox = resize_boxes(bbox, (h, w), image.shape[-2:])
	target["boxes"] = bbox
	if "keypoints" in target:
	keypoints = target["keypoints"]
	keypoints = resize_keypoints(keypoints, (h, w), image.shape[-2:])
	target["keypoints"] = keypoints
	return image, target

Perhaps this covers your use-case?

[tflahaul]

Oh I had no idea this function existed but it does cover my case, thanks for letting me know!

Would it still be interesting to use my implementation for resize_boxes ? It needs a few tweaks but after running some tests I noticed an average speedup of about 28% on my CPU. And from looking at the code I guess using the slices would use a bit less memory too.

[datumbox]

@tflahaul Hmm, it's a tough call on this one...

At the moment, we are in the process of reviewing ways to improve the transforms for Detection, so having your use-case in mind is helpful. The problem with adding this PR in is that we already have a way to do the same thing. Our API is far from perfect and that's why we want to improve it but the functionality is already there. I think I'm weighting towards closing this PR and pinging you once we are ready to implement part of the Detection Transform API in case you want to contribute. This will give us time to finalize the API and reduce the BC-breaking changes. Thoughts?

BTW if you have time and you would like to contribute to the project have a look for issues tagged with "good first issue" and "help wanted". 😃

[tflahaul]

Yes I understand, this seems like the best thing to do right now.
Please notify me when you're ready and I'll have a look at the open issues in the meantime.

[datumbox]

@tflahaul Awesome, sounds like a plan!