Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[camerax] Implement resolution configuration #3799

Merged
merged 43 commits into from
Sep 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
44d7828
Add resolution configuration
camsim99 Apr 22, 2023
6281eb5
dart side of impl
bparrishMines Apr 26, 2023
9a74cd1
java impls
bparrishMines Apr 26, 2023
03ef376
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 May 15, 2023
2a5c0fc
Fix implementation
camsim99 May 15, 2023
9001ab0
Small cleanup
camsim99 May 15, 2023
fd19de6
Fixing tests and cleanup minus plugin dart impl test
camsim99 May 17, 2023
c9b62ef
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 Jul 18, 2023
725be10
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 Jul 18, 2023
6389ae1
Work on Dart side -- tests, cleanup, bug id
camsim99 Jul 18, 2023
16770de
Fix java unit tests
camsim99 Jul 18, 2023
a8144fa
Fix typo
camsim99 Jul 18, 2023
aba4c34
Fix dart tests
camsim99 Jul 19, 2023
ca5604a
Add integration tests
camsim99 Jul 20, 2023
474745c
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 Aug 16, 2023
b778894
Add res support
camsim99 Aug 16, 2023
8d2cb3a
correct overrides
camsim99 Aug 16, 2023
a145afe
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 Aug 16, 2023
fd3ade2
Self review
camsim99 Aug 16, 2023
1dc6fbd
formatting
camsim99 Aug 16, 2023
fad7cbc
Undo strange change
camsim99 Aug 16, 2023
08edc4c
Fix dart unit tests
camsim99 Aug 16, 2023
1def519
Fix integration test
camsim99 Aug 17, 2023
5bab620
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 Aug 17, 2023
caf819e
Nits
camsim99 Aug 17, 2023
c775756
Update readme
camsim99 Aug 17, 2023
fabaab7
Update packages/camera/camera_android_camerax/lib/src/android_camera_…
camsim99 Aug 22, 2023
810cb54
Update packages/camera/camera_android_camerax/lib/src/android_camera_…
camsim99 Aug 22, 2023
60a47ed
Update packages/camera/camera_android_camerax/lib/src/android_camera_…
camsim99 Aug 22, 2023
474d84c
Update packages/camera/camera_android_camerax/lib/src/android_camera_…
camsim99 Aug 22, 2023
6f45ecb
Add comments to integration tests
camsim99 Aug 22, 2023
5c31de4
Merge branch 'camx_resconfig' of github.com:camsim99/packages into ca…
camsim99 Aug 22, 2023
e4134bd
Updated integration test comment
camsim99 Aug 23, 2023
bb35a61
start quality re-write
camsim99 Aug 29, 2023
5ac1c7c
Replace vid qual const with quality/qualitydata
camsim99 Aug 29, 2023
860b4db
Replace Quality indices with VideoQuality/VideoQualityData class
camsim99 Aug 29, 2023
5ea328e
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 Aug 29, 2023
a163f0e
Fix versionining
camsim99 Aug 29, 2023
2a9671e
Format
camsim99 Aug 29, 2023
0178493
run tests
camsim99 Aug 29, 2023
ff9bd3a
nits
camsim99 Sep 7, 2023
396091c
Merge remote-tracking branch 'upstream/main' into camx_resconfig
camsim99 Sep 7, 2023
206fbb6
Fix changelog
camsim99 Sep 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/camera/camera_android_camerax/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 0.5.0+17

* Implements resolution configuration for all camera use cases.

## 0.5.0+16

* Adds pub topics to package metadata.
Expand Down
10 changes: 6 additions & 4 deletions packages/camera/camera_android_camerax/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,18 @@ dependencies:

## Missing features and limitations

### Resolution configuration \[[Issue #120462][120462]\]

Any specified `ResolutionPreset` wll go unused in favor of CameraX defaults and
`onCameraResolutionChanged` is unimplemented.
### 240p resolution configuration for video recording

240p resolution configuration for video recording is unsupported by CameraX,
and thus, the plugin will fall back to 480p if configured with a
`ResolutionPreset`.

### Locking/Unlocking capture orientation \[[Issue #125915][125915]\]

`lockCaptureOrientation` & `unLockCaptureOrientation` are unimplemented.

### Flash mode configuration \[[Issue #120715][120715]\]
### Torch mode \[[Issue #120715][120715]\]

Calling `setFlashMode` with mode `FlashMode.torch` currently does nothing.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import androidx.camera.video.FallbackStrategy;
import androidx.camera.video.Quality;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.FallbackStrategyHostApi;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoQualityConstraint;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoQuality;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoResolutionFallbackRule;

/**
Expand All @@ -28,20 +28,18 @@ public class FallbackStrategyHostApiImpl implements FallbackStrategyHostApi {
public static class FallbackStrategyProxy {
/** Creates an instance of {@link FallbackStrategy}. */
public @NonNull FallbackStrategy create(
@NonNull VideoQualityConstraint videoQualityConstraint,
@NonNull VideoResolutionFallbackRule fallbackRule) {
Quality videoQuality =
QualitySelectorHostApiImpl.getQualityFromVideoQualityConstraint(videoQualityConstraint);
@NonNull VideoQuality videoQuality, @NonNull VideoResolutionFallbackRule fallbackRule) {
Quality quality = QualitySelectorHostApiImpl.getQualityFromVideoQuality(videoQuality);

switch (fallbackRule) {
case HIGHER_QUALITY_OR_LOWER_THAN:
return FallbackStrategy.higherQualityOrLowerThan(videoQuality);
return FallbackStrategy.higherQualityOrLowerThan(quality);
case HIGHER_QUALITY_THAN:
return FallbackStrategy.higherQualityThan(videoQuality);
return FallbackStrategy.higherQualityThan(quality);
case LOWER_QUALITY_OR_HIGHER_THAN:
return FallbackStrategy.lowerQualityOrHigherThan(videoQuality);
return FallbackStrategy.lowerQualityOrHigherThan(quality);
case LOWER_QUALITY_THAN:
return FallbackStrategy.lowerQualityThan(videoQuality);
return FallbackStrategy.lowerQualityThan(quality);
}
throw new IllegalArgumentException(
"Specified fallback rule " + fallbackRule + " unrecognized.");
Expand Down Expand Up @@ -75,9 +73,8 @@ public FallbackStrategyHostApiImpl(@NonNull InstanceManager instanceManager) {
@Override
public void create(
@NonNull Long identifier,
@NonNull VideoQualityConstraint videoQualityConstraint,
@NonNull VideoQuality videoQuality,
@NonNull VideoResolutionFallbackRule fallbackRule) {
instanceManager.addDartCreatedInstance(
proxy.create(videoQualityConstraint, fallbackRule), identifier);
instanceManager.addDartCreatedInstance(proxy.create(videoQuality, fallbackRule), identifier);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ private LiveDataSupportedType(final int index) {
*
* <p>See https://developer.android.com/reference/androidx/camera/video/Quality.
*/
public enum VideoQualityConstraint {
public enum VideoQuality {
SD(0),
HD(1),
FHD(2),
Expand All @@ -122,12 +122,16 @@ public enum VideoQualityConstraint {

final int index;

private VideoQualityConstraint(final int index) {
private VideoQuality(final int index) {
this.index = index;
}
}

/** Fallback rules for selecting video resolution. */
/**
* Fallback rules for selecting video resolution.
*
* <p>See https://developer.android.com/reference/androidx/camera/video/FallbackStrategy.
*/
public enum VideoResolutionFallbackRule {
HIGHER_QUALITY_OR_LOWER_THAN(0),
HIGHER_QUALITY_THAN(1),
Expand Down Expand Up @@ -472,6 +476,59 @@ ArrayList<Object> toList() {
}
}

/**
* Convenience class for sending lists of [Quality]s.
*
* <p>Generated class from Pigeon that represents data sent in messages.
*/
public static final class VideoQualityData {
private @NonNull VideoQuality quality;

public @NonNull VideoQuality getQuality() {
return quality;
}

public void setQuality(@NonNull VideoQuality setterArg) {
if (setterArg == null) {
throw new IllegalStateException("Nonnull field \"quality\" is null.");
}
this.quality = setterArg;
}

/** Constructor is non-public to enforce null safety; use Builder. */
VideoQualityData() {}

public static final class Builder {

private @Nullable VideoQuality quality;

public @NonNull Builder setQuality(@NonNull VideoQuality setterArg) {
this.quality = setterArg;
return this;
}

public @NonNull VideoQualityData build() {
VideoQualityData pigeonReturn = new VideoQualityData();
pigeonReturn.setQuality(quality);
return pigeonReturn;
}
}

@NonNull
ArrayList<Object> toList() {
ArrayList<Object> toListResult = new ArrayList<Object>(1);
toListResult.add(quality == null ? null : quality.index);
return toListResult;
}

static @NonNull VideoQualityData fromList(@NonNull ArrayList<Object> list) {
VideoQualityData pigeonResult = new VideoQualityData();
Object quality = list.get(0);
pigeonResult.setQuality(quality == null ? null : VideoQuality.values()[(int) quality]);
return pigeonResult;
}
}

public interface Result<T> {
@SuppressWarnings("UnknownNullness")
void success(T result);
Expand Down Expand Up @@ -2991,6 +3048,8 @@ protected Object readValueOfType(byte type, @NonNull ByteBuffer buffer) {
switch (type) {
case (byte) 128:
return ResolutionInfo.fromList((ArrayList<Object>) readValue(buffer));
case (byte) 129:
return VideoQualityData.fromList((ArrayList<Object>) readValue(buffer));
default:
return super.readValueOfType(type, buffer);
}
Expand All @@ -3001,6 +3060,9 @@ protected void writeValue(@NonNull ByteArrayOutputStream stream, Object value) {
if (value instanceof ResolutionInfo) {
stream.write(128);
writeValue(stream, ((ResolutionInfo) value).toList());
} else if (value instanceof VideoQualityData) {
stream.write(129);
writeValue(stream, ((VideoQualityData) value).toList());
} else {
super.writeValue(stream, value);
}
Expand All @@ -3012,12 +3074,11 @@ public interface QualitySelectorHostApi {

void create(
@NonNull Long identifier,
@NonNull List<Long> videoQualityConstraintIndexList,
@NonNull List<VideoQualityData> videoQualityDataList,
@Nullable Long fallbackStrategyId);

@NonNull
ResolutionInfo getResolution(
@NonNull Long cameraInfoId, @NonNull VideoQualityConstraint quality);
ResolutionInfo getResolution(@NonNull Long cameraInfoId, @NonNull VideoQuality quality);

/** The codec used by QualitySelectorHostApi. */
static @NonNull MessageCodec<Object> getCodec() {
Expand All @@ -3039,12 +3100,13 @@ static void setup(
ArrayList<Object> wrapped = new ArrayList<Object>();
ArrayList<Object> args = (ArrayList<Object>) message;
Number identifierArg = (Number) args.get(0);
List<Long> videoQualityConstraintIndexListArg = (List<Long>) args.get(1);
List<VideoQualityData> videoQualityDataListArg =
(List<VideoQualityData>) args.get(1);
Number fallbackStrategyIdArg = (Number) args.get(2);
try {
api.create(
(identifierArg == null) ? null : identifierArg.longValue(),
videoQualityConstraintIndexListArg,
videoQualityDataListArg,
(fallbackStrategyIdArg == null) ? null : fallbackStrategyIdArg.longValue());
wrapped.add(0, null);
} catch (Throwable exception) {
Expand All @@ -3069,8 +3131,8 @@ static void setup(
ArrayList<Object> wrapped = new ArrayList<Object>();
ArrayList<Object> args = (ArrayList<Object>) message;
Number cameraInfoIdArg = (Number) args.get(0);
VideoQualityConstraint qualityArg =
args.get(1) == null ? null : VideoQualityConstraint.values()[(int) args.get(1)];
VideoQuality qualityArg =
args.get(1) == null ? null : VideoQuality.values()[(int) args.get(1)];
try {
ResolutionInfo output =
api.getResolution(
Expand All @@ -3094,7 +3156,7 @@ public interface FallbackStrategyHostApi {

void create(
@NonNull Long identifier,
@NonNull VideoQualityConstraint quality,
@NonNull VideoQuality quality,
@NonNull VideoResolutionFallbackRule fallbackRule);

/** The codec used by FallbackStrategyHostApi. */
Expand All @@ -3117,8 +3179,8 @@ static void setup(
ArrayList<Object> wrapped = new ArrayList<Object>();
ArrayList<Object> args = (ArrayList<Object>) message;
Number identifierArg = (Number) args.get(0);
VideoQualityConstraint qualityArg =
args.get(1) == null ? null : VideoQualityConstraint.values()[(int) args.get(1)];
VideoQuality qualityArg =
args.get(1) == null ? null : VideoQuality.values()[(int) args.get(1)];
VideoResolutionFallbackRule fallbackRuleArg =
args.get(2) == null
? null
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
import androidx.camera.video.QualitySelector;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.QualitySelectorHostApi;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.ResolutionInfo;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoQualityConstraint;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoQuality;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoQualityData;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
Expand All @@ -34,12 +35,12 @@ public class QualitySelectorHostApiImpl implements QualitySelectorHostApi {
public static class QualitySelectorProxy {
/** Creates an instance of {@link QualitySelector}. */
public @NonNull QualitySelector create(
@NonNull List<Long> videoQualityConstraintIndexList,
@NonNull List<VideoQualityData> videoQualityDataList,
@Nullable FallbackStrategy fallbackStrategy) {
// Convert each index of VideoQualityConstraint to Quality.
// Convert each index of VideoQuality to Quality.
List<Quality> qualityList = new ArrayList<Quality>();
for (Long qualityIndex : videoQualityConstraintIndexList) {
qualityList.add(getQualityConstant(qualityIndex));
for (VideoQualityData videoQualityData : videoQualityDataList) {
qualityList.add(getQualityFromVideoQuality(videoQualityData.getQuality()));
}

boolean fallbackStrategySpecified = fallbackStrategy != null;
Expand All @@ -57,12 +58,6 @@ public static class QualitySelectorProxy {
? QualitySelector.fromOrderedList(qualityList, fallbackStrategy)
: QualitySelector.fromOrderedList(qualityList);
}

/** Converts from index of {@link VideoQualityConstraint} to {@link Quality}. */
private Quality getQualityConstant(@NonNull Long qualityIndex) {
VideoQualityConstraint quality = VideoQualityConstraint.values()[qualityIndex.intValue()];
return getQualityFromVideoQualityConstraint(quality);
}
}

/**
Expand Down Expand Up @@ -93,11 +88,11 @@ public QualitySelectorHostApiImpl(@NonNull InstanceManager instanceManager) {
@Override
public void create(
@NonNull Long identifier,
@NonNull List<Long> videoQualityConstraintIndexList,
@NonNull List<VideoQualityData> videoQualityDataList,
@Nullable Long fallbackStrategyIdentifier) {
instanceManager.addDartCreatedInstance(
proxy.create(
videoQualityConstraintIndexList,
videoQualityDataList,
fallbackStrategyIdentifier == null
? null
: Objects.requireNonNull(instanceManager.getInstance(fallbackStrategyIdentifier))),
Expand All @@ -110,24 +105,23 @@ public void create(
*/
@Override
public @NonNull ResolutionInfo getResolution(
@NonNull Long cameraInfoIdentifier, @NonNull VideoQualityConstraint quality) {
@NonNull Long cameraInfoIdentifier, @NonNull VideoQuality quality) {
final Size result =
QualitySelector.getResolution(
Objects.requireNonNull(instanceManager.getInstance(cameraInfoIdentifier)),
getQualityFromVideoQualityConstraint(quality));
getQualityFromVideoQuality(quality));
return new ResolutionInfo.Builder()
.setWidth(Long.valueOf(result.getWidth()))
.setHeight(Long.valueOf(result.getHeight()))
.build();
}

/**
* Converts the specified {@link VideoQualityConstraint} to a {@link Quality} that is understood
* Converts the specified {@link VideoQuality to a {@link Quality} that is understood
* by CameraX.
*/
public static @NonNull Quality getQualityFromVideoQualityConstraint(
@NonNull VideoQualityConstraint videoQualityConstraint) {
switch (videoQualityConstraint) {
public static @NonNull Quality getQualityFromVideoQuality(@NonNull VideoQuality videoQuality) {
switch (videoQuality) {
case SD:
return Quality.SD;
case HD:
Expand All @@ -142,8 +136,6 @@ public void create(
return Quality.HIGHEST;
}
throw new IllegalArgumentException(
"VideoQualityConstraint "
+ videoQualityConstraint
+ " is unhandled by QualitySelectorHostApiImpl.");
"VideoQuality " + videoQuality + " is unhandled by QualitySelectorHostApiImpl.");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

import androidx.camera.video.FallbackStrategy;
import androidx.camera.video.Quality;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoQualityConstraint;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoQuality;
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.VideoResolutionFallbackRule;
import org.junit.After;
import org.junit.Before;
Expand Down Expand Up @@ -48,11 +48,11 @@ public void hostApiCreate_makesCallToCreateExpectedFallbackStrategy() {

try (MockedStatic<FallbackStrategy> mockedFallbackStrategy =
mockStatic(FallbackStrategy.class)) {
for (VideoQualityConstraint videoQualityConstraint : VideoQualityConstraint.values()) {
for (VideoQuality videoQuality : VideoQuality.values()) {
for (VideoResolutionFallbackRule fallbackRule : VideoResolutionFallbackRule.values()) {
// Determine expected Quality based on videoQualityConstraint being tested.
// Determine expected Quality based on videoQuality being tested.
Quality convertedQuality = null;
switch (videoQualityConstraint) {
switch (videoQuality) {
case SD:
convertedQuality = Quality.SD;
break;
Expand All @@ -72,10 +72,7 @@ public void hostApiCreate_makesCallToCreateExpectedFallbackStrategy() {
convertedQuality = Quality.HIGHEST;
break;
default:
fail(
"The VideoQualityConstraint "
+ videoQualityConstraint.toString()
+ "is unhandled by this test.");
fail("The VideoQuality " + videoQuality.toString() + "is unhandled by this test.");
}
// Set Quality as final local variable to avoid error about using non-final (or effecitvely final) local variables in lambda expressions.
final Quality expectedQuality = convertedQuality;
Expand Down Expand Up @@ -108,7 +105,7 @@ public void hostApiCreate_makesCallToCreateExpectedFallbackStrategy() {
+ fallbackRule.toString()
+ "is unhandled by this test.");
}
hostApi.create(instanceIdentifier, videoQualityConstraint, fallbackRule);
hostApi.create(instanceIdentifier, videoQuality, fallbackRule);
assertEquals(instanceManager.getInstance(instanceIdentifier), mockFallbackStrategy);

// Clear/reset FallbackStrategy mock and InstanceManager.
Expand Down
Loading