Add Rich image editing capabilities to Gutenberg #21024

ajlende · 2020-03-19T15:43:51Z

Description

Copied from #20637
Fixes #13748
Props to @johngodley and @jasmussen for the heavy lifting.

Adds new image-editor API endpoints in lib/
Adds new "block" that extends image capabilities in the image block

How has this been tested?

Add image
Click on image and try different editing tools

Screenshots

New toolbar items

Zoom crop mode

Types of changes

New features: Adds crop, flip, rotate capabilities to images.

Checklist:

My code is tested.
My code follows the WordPress code style.
My code follows the accessibility standards.
My code has proper inline documentation.
I've included developer documentation if appropriate.
I've updated all React Native files affected by any refactorings/renamings in this PR.

packages/block-library/src/rich-image/constants.js

ellatrix · 2020-06-03T09:19:41Z

As the feature was also flagged as experimental

I see the feature added by default in the Gutenberg plugin, or am I missing something?

If it helps, I expect more review to happen in the much smaller and easier to digest PRs that will be coming in the next couple weeks for the issues opened and linked to this PR.

That's good if the general direction is agreed upon. The problem is that there's no easy, small PR to go from editing on the server to editing in the browser. The PR will need to replace it almost entirely.

This is already being tracked in #22566 and #22579. The first previews the change by editing the image in JS, and the second makes a single request to the server once all image edits are complete.

I'm not talking about previews in the browser, but rather editing it in JS and sending the edited image to the server on save or autosave or some other trigger. There's no for the server to edit the images.

I any case, thanks for working on this!

ellatrix · 2020-06-03T10:12:15Z

packages/block-library/src/rich-image/rich-image/index.js

+		const { url } = attributes;
+		const isEditing = ! isCrop && isSelected && url;
+
+		if ( ! isSelected ) {


This is causing the block to rerender when it is selected, which is not a great user experience esp. with an image that needs to load.

ellatrix · 2020-06-03T10:12:51Z

packages/block-library/src/rich-image/rich-image/index.js

+			<Fragment>
+				{ noticeUI }
+
+				<div className={ classes }>


Why the div wrapper? I don't see why image editing tools should affect the structure of a block.

The in-progress spinner needs a container to be centered properly on top of the image

Let's rethink the spinner so we don't have to have the div. Can be a fading in and out image which I believe we use to indicate transience elsewhere. Happy to work with you on this 👍

ajlende · 2020-06-03T19:56:26Z

I see the feature added by default in the Gutenberg plugin, or am I missing something?

Experimental, as in Gutenberg phase 2 (#21024 (comment)) as to not ship with core. @talldan also mentioned adding it to the experiments page, which I can go ahead and switch to instead since this is a bigger feature.

EDIT: PR #22870 is now open

I'm not talking about previews in the browser, but rather editing it in JS and sending the edited image to the server on save or autosave or some other trigger.

Which benefits do you think we'll get from editing in the browser instead? Using WP_Image_Editor still seems more straightforward to me. Either way you're going to need to download the saved image since images are saved on the server. In #22579 (batched edits) I was hoping to only do the edit once, ideally on save if I can hook into that event. And I think the CSS animations for previewing from #22566 would be easier and just as smooth as doing them on a canvas.

TimothyBJacobs · 2020-06-04T01:03:06Z

lib/class-wp-rest-image-editor-controller.php

+					'permission_callback' => array( $this, 'permission_callback' ),
+					'args'                => array(
+						'direction' => array(
+							'type'     => 'enum',


enum isn't a type, it should instead be string.

TimothyBJacobs · 2020-06-04T01:03:22Z

lib/class-wp-rest-image-editor-controller.php

+					'permission_callback' => array( $this, 'permission_callback' ),
+					'args'                => array(
+						'cropX'      => array(
+							'type'     => 'float',


float is not a json schema type, it should be number.

TimothyBJacobs · 2020-06-04T01:04:35Z

lib/image-editor/class-image-editor-crop.php

+	 * @param integer $height Height in pixels for the crop.
+	 */
+	public function __construct( $crop_x, $crop_y, $width, $height ) {
+		$this->crop_x = floatval( $crop_x );


Why are we using floatval is the types are documented as integer? Which is correct?

TimothyBJacobs · 2020-06-04T01:07:04Z

lib/image-editor/class-image-editor-crop.php

+	 * @return array Updated metadata.
+	 */
+	public function apply_to_meta( $meta ) {
+		$meta['cropX']      = $this->crop_x;


Are these based off an existing pattern that means these need to be camelCase? Core uses snake_case in PHP land.

TimothyBJacobs · 2020-06-04T01:09:46Z

Instead of using WP_REST_Request::get_params() and operating off of that, the code should just use $request['parameter'] directly.

The REST API also uses snake_case not camelCase for parameter names. Those should be updated.

The REST API meets every Thursday at 18:00 UTC in slack at #core-restapi. If you aren't seeing feedback from the REST API team, drop a link to the PR during our open floor.

gziolo · 2020-06-04T05:27:12Z

One note, this PR doesn't add a new block but some of the functionality added here makes the impression that a new experimental block is being registered. Given that this functionality is added using a filter, it should be moved to a different package, @wordpress/editor might be a good fit if this is planned to work on all edit pages. @wordpress/block-library should contain only folders with core blocks.

mtias · 2020-06-04T14:27:03Z

If we can address the outstanding issues promptly, I don't think we need to add it to experiments. The discussion on how much to do on the client vs server is a bit nuanced. It makes sense to me to do all manipulation client side and then commit the updates as a new image on "save". Stacking new images on the server every time would be taxing on storage space and memory usage for shared hosts.

TimothyBJacobs · 2020-06-04T14:59:06Z

Related ticket: https://core.trac.wordpress.org/ticket/49096

ellatrix · 2020-06-04T16:38:45Z

Ok, even if you could do previewing in JS or even CSS, why do a round trip with an image to the server at all then, if it can be done on the client side? That's double work. Even more than double because the image is being transferred over the network for each edit. And regardless of that, the preview might not be exactly the same. The algorithm on the server might be different. Things like filters suddenly become more difficult to do, while if we just have to handle the client, it’s fairly easy.

ajlende · 2020-06-04T18:17:47Z

Doing the edits on the server was under the assumption that #22579 would be implemented so the request only happens once on save, so you wouldn't be downloading an image per edit. You'd POST the instructions for the edits and only download the final image.

When editing in JS, since you send the image to the server, aren't you going to want to replace the image in the editor with the URL of the saved image to make sure the save worked properly for the preview? That was my assumption, and that way you're both uploading and downloading the image.

The algorithms for crop, rotate, and flip are very straightforward, so I don't think there would be much of a chance for the algorithms to be different for them.

I was thinking that filters would continue to be done in CSS like they are in Automattic/block-experiments, but I forgot that CSS filters aren't supported in IE11. It could still be done with SVG filters which are supported across browsers—I started a demo for a duotone filter a while ago in Automattic/block-experiments#88.

Still, it might be nice to save the filtered image as well. In that case filters would have more opportunity for differences since you'd have to replicate the SVG filter on the server.

ellatrix · 2020-06-04T19:13:27Z

I think filters are not done in CSS for the same reasons rotate and flip are not done in CSS on the front end? The image itself should be adjusted because the content might not include the CSS or might be distributed by email, RSS, social networks, etc. Additionally, if you download the image from the website, you'll get the original image. Generally people are not expecting anyone to have access to the original images.

With the client already doing the edits, I don't see why we should add the complexity of doing the same thing on the server, if we can just upload the image on save/publish.

I could also imagine Gutenberg being used with another API of a CMS that doesn't support image editing. Editing in JS means using the common API of creating/uploading an image, while editing on the server means additional endpoints are required.

azaozz · 2020-06-05T02:27:52Z

With the client already doing the edits, I don't see why we should add the complexity of doing the same thing on the server, if we can just upload the image on save/publish.

It would be ideal if that works :) Editing images in the browser (also before uploading them) is the best option. Been looking at that for quite some time, but seems "we're not there yet", the browsers still aren't particularly good image editors.

Known problems with HTML canvas based editors (afaik):

Insufficient memory/resources on phones and possibly old/slow computers when attempting to edit larger images (the default base size in WP currently is 2560px). See https://github.com/fengyuanchen/cropperjs#known-issues.
Not able to edit/preserve EXIF data.
Image quality and/or compression may not be as good as server based editors like ImageMagick.

Would be great if some js based image editors can be evaluated again, there seem to be few that are more popular.

jasmussen · 2020-06-05T06:36:34Z

I think it might be worth separating the conversation on filters. These were experimental, and might not even be a fit for the core project, regardless of previous implementation. If they are (even just making a photo black and white and adjusting contrast would be delightful), it feels a separate conversation. One which I look forward to having, by the way :)

ellatrix · 2020-06-05T09:13:05Z

Regarding EXIF data on JPG images, the edited image would have access to the original image data on the server, so we could adjust the function to get the EXIF data to get it from the original, if so desired. Not sure if that would be considered an ugly fix. Alternatively we can try to preserve it in JS, I would need to research it.

Regarding the quality, are there any articles about this? I don't really see a problem at first sight.

Yes, that's really annoying for phones and old/slow computers... what is the limit on a server?

RickorDD · 2020-06-06T19:32:32Z

When i add the Imageblock the Fontbar hidden the Text.

Remove this Bar to the Top?

jasmussen · 2020-06-08T08:09:58Z

When i add the Imageblock the Fontbar hidden the Text.

This is a bug with re-rendering, that will be fixed. The caption field should not even be visible in the setup state. Thanks for the report, @RickorDD

joppuyo · 2020-06-09T13:16:11Z

About editing images in JavaScript:

There are tons of features that depend on native WordPress image editor that are completely sidestepped by implementing image editing in JavaScript using canvas, the first thing that comes to mind is jpeg_quality filter that allows users and plugins to alter the JPEG quality.

WordPress also allows hooking into the image processing so plugins like image optimizers can process the images. It's also possible to completely replace the WP_Image_Editor with another class to customize the image processing, for example using an image processing library like vips instead of GD or ImageMagick.

If a canvas is used for image editing instead of predictable quality and scaling settings, the results will vary depending on which browser or OS was used when editing the image. Replacing native server-side WordPress functionality with JavaScript may seem trivial at first but it has a lot of consequences that need to be taken into account.

TimothyBJacobs · 2020-06-09T17:39:59Z

Is there a PR somewhere for addressing the REST API related issues? As well as an overall tracking ticket for image editing?

The REST API team is concerned about the amount of server side REST API code that is needed with this current approach so late in the cycle if we want the feature to land in 5.5. There is already a large amount of complex REST API integrations that are slated for 5.5.

The only way I can see this landing in 5.5 is by doing editing in JS and adding support for https://core.trac.wordpress.org/ticket/49096.

ajlende · 2020-06-09T18:55:48Z

@TimothyBJacobs I didn't get to your edits because I was going to do a batch editing endpoint instead of all the individual ones in #22959. It's taken longer than expected, so I opened up #23037 for those edits now.

koraysels · 2021-12-13T10:50:10Z

can we add more aspect ratios to the crop feature? and how do we go about doing this?

paaljoachim · 2023-04-03T17:26:36Z

I am adding in this associated issue
Hover color over image block
#33939

mtias reviewed Mar 19, 2020

View reviewed changes

packages/block-library/src/rich-image/constants.js Outdated Show resolved Hide resolved

mtias added [Block] Image Affects the Image Block [Feature] Media Anything that impacts the experience of managing media labels Mar 19, 2020

ajlende force-pushed the try/13748-rich-image branch 3 times, most recently from 7895448 to c440834 Compare March 20, 2020 19:11

mkaz and others added 21 commits March 24, 2020 10:59

Init commit of adding rich image editing to Gutenberg

5e8769b

Remove image filters

720e57f

Replace custom notice with snackbar

62e168d

Add react-easy-crop

f3656b1

Move dropdowns to open down and right

03025d5

Center crop icon

baab84e

Replace image cropping tool

cc476db

Add crop aspect ratios

11a940a

Keep original image aspect ratio when cropping

4d79072

Clean up cropper functions

cc32ad5

Remove error_log for debug

5366dfd

Fix sending percentages for cropping

c07405c

Add range control for zoom

a1bb170

Finish removing filters

9c895cd

Remove ReactCrop CSS

d372d3c

Style full width rather than natural width

b96eeef

Fix package.json and lockfile

a1cf03e

Remove more filter scss

6db2f6c

Remove unused SCSS variables

d5c73ee

Name zoom constants

0de6203

Remove redundant inProgress

da7c016

ajlende force-pushed the try/13748-rich-image branch from c440834 to e15c736 Compare March 24, 2020 16:15

Fix missing block light wrapper for crop

c35716d

ajlende force-pushed the try/13748-rich-image branch from e15c736 to c35716d Compare March 24, 2020 16:16

ellatrix reviewed Jun 3, 2020

View reviewed changes

ajlende mentioned this pull request Jun 3, 2020

Image: Add rich image editing option to experiments page #22870

Merged

6 tasks

TimothyBJacobs reviewed Jun 4, 2020

View reviewed changes

oandregal added [Type] Feature New feature to highlight in changelogs. and removed [Type] Enhancement A suggestion for improvement. labels Jun 8, 2020

ajlende mentioned this pull request Jun 9, 2020

Image Edit: REST API Code Cleanup #23037

Merged

6 tasks

ellatrix mentioned this pull request Jun 16, 2020

WP 5.5: merge all PHP changes #23210

Closed

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Rich image editing capabilities to Gutenberg #21024

Add Rich image editing capabilities to Gutenberg #21024

ajlende commented Mar 19, 2020 •

edited

Loading

ellatrix commented Jun 3, 2020

ellatrix Jun 3, 2020

ellatrix Jun 3, 2020

ajlende Jun 4, 2020

jasmussen Jun 4, 2020

ajlende commented Jun 3, 2020 •

edited

Loading

TimothyBJacobs Jun 4, 2020

TimothyBJacobs Jun 4, 2020

TimothyBJacobs Jun 4, 2020

TimothyBJacobs Jun 4, 2020

TimothyBJacobs commented Jun 4, 2020

gziolo commented Jun 4, 2020

mtias commented Jun 4, 2020

TimothyBJacobs commented Jun 4, 2020

ellatrix commented Jun 4, 2020 •

edited

Loading

ajlende commented Jun 4, 2020

ellatrix commented Jun 4, 2020 •

edited

Loading

azaozz commented Jun 5, 2020

jasmussen commented Jun 5, 2020

ellatrix commented Jun 5, 2020

RickorDD commented Jun 6, 2020

jasmussen commented Jun 8, 2020

joppuyo commented Jun 9, 2020 •

edited

Loading

TimothyBJacobs commented Jun 9, 2020

ajlende commented Jun 9, 2020

koraysels commented Dec 13, 2021

paaljoachim commented Apr 3, 2023

Add Rich image editing capabilities to Gutenberg #21024

Add Rich image editing capabilities to Gutenberg #21024

Conversation

ajlende commented Mar 19, 2020 • edited Loading

Description

How has this been tested?

Screenshots

Types of changes

Checklist:

ellatrix commented Jun 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ajlende commented Jun 3, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimothyBJacobs commented Jun 4, 2020

gziolo commented Jun 4, 2020

mtias commented Jun 4, 2020

TimothyBJacobs commented Jun 4, 2020

ellatrix commented Jun 4, 2020 • edited Loading

ajlende commented Jun 4, 2020

ellatrix commented Jun 4, 2020 • edited Loading

azaozz commented Jun 5, 2020

jasmussen commented Jun 5, 2020

ellatrix commented Jun 5, 2020

RickorDD commented Jun 6, 2020

jasmussen commented Jun 8, 2020

joppuyo commented Jun 9, 2020 • edited Loading

TimothyBJacobs commented Jun 9, 2020

ajlende commented Jun 9, 2020

koraysels commented Dec 13, 2021

paaljoachim commented Apr 3, 2023

ajlende commented Mar 19, 2020 •

edited

Loading

ajlende commented Jun 3, 2020 •

edited

Loading

ellatrix commented Jun 4, 2020 •

edited

Loading

ellatrix commented Jun 4, 2020 •

edited

Loading

joppuyo commented Jun 9, 2020 •

edited

Loading