Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xiaomi Android 14] Light grays appearing as white only for scrcpy #4756

Closed
2 tasks done
carstenhag opened this issue Mar 12, 2024 · 24 comments
Closed
2 tasks done

[Xiaomi Android 14] Light grays appearing as white only for scrcpy #4756

carstenhag opened this issue Mar 12, 2024 · 24 comments

Comments

@carstenhag
Copy link

carstenhag commented Mar 12, 2024

  • I have read the FAQ.
  • I have searched in existing issues.

Environment

  • OS: macOS 14.3.1
  • scrcpy version: 2.4
  • installation method: brew
  • device model: MBP 16" M3 / Poco F5
  • Android version: 14

Describe the bug

Light grays are not shown. They are displayed as white. ADB and Android Studio render the colors correctly.

I am pretty sure this is happening since I upgraded the phone from Android 13 to Android 14 some weeks ago.

With a [Google] google Pixel 6 (Android 14) the issue is not occurring, which means the device is somehow influencing this.

/opt/homebrew/Cellar/scrcpy/2.4/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 71.2 MB/s (69007 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] POCO 23049PCD8G (Android 14)

AS:
as
ADB:
adb-screencapture-output
scrpcy:
scrcpy

@rom1v
Copy link
Collaborator

rom1v commented Mar 12, 2024

ADB

What do you mean by adb here? screenrecord (adb shell screenrecord /sdcard/file.mp4)?

If you record then play the file (with VLC or Firefox for example), does the problem occur?

scrcpy --record=file.mp4 --no-playback
@carstenhag
Copy link
Author

Yep, I meant adb shell screenrecord /sdcard/file.mp4.

With scrcpy --record=file.mp4 --no-playback it does work correctly.
Screenshot 2024-03-12 at 15 58 40

@carstenhag
Copy link
Author

    val brush = Brush.horizontalGradient(listOf(Color.Black, Color.White))
    Canvas(
        modifier = Modifier.size(400.dp),
        onDraw = {
            drawCircle(brush)
        }
    )

With this one can see that after some specific gray value it just outputs white

Screenshot 2024-03-12 at 16 00 21
@rom1v
Copy link
Collaborator

rom1v commented Mar 12, 2024

In the console output, which renderer is used?

For example:

INFO: Renderer: opengl

I guess this is metal for you.

If so, could you try other values:

scrcpy --render-driver=opengl
scrcpy --render-driver=opengles2
scrcpy --render-driver=software
@carstenhag
Copy link
Author

carstenhag commented Mar 12, 2024

All of them look the same (partially horrible) way, even software, which was unexpected to me 😆

 %  ~  scrcpy
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  3d97ed47                        device  23049PCD8G
/opt/homebrew/Cellar/scrcpy/2.4/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 111.3 MB/s (69007 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] POCO 23049PCD8G (Android 14)
INFO: Renderer: metal
INFO: Texture: 1080x2400
^[[A^CWARN: Device disconnected
 %  ~  USAGE  scrcpy --render-driver=opengl
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  3d97ed47                        device  23049PCD8G
/opt/homebrew/Cellar/scrcpy/2.4/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 93.2 MB/s (69007 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] POCO 23049PCD8G (Android 14)
INFO: Renderer: opengl
INFO: OpenGL version: 4.1 Metal - 88
INFO: Trilinear filtering enabled
INFO: Texture: 1080x2400
^CWARN: Device disconnected
 %  ~  USAGE  scrcpy --render-driver=opengles2
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  3d97ed47                        device  23049PCD8G
/opt/homebrew/Cellar/scrcpy/2.4/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 111.9 MB/s (69007 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] POCO 23049PCD8G (Android 14)
INFO: Renderer: metal
INFO: Texture: 1080x2400
[server] WARN: Could not get initial audio timestamp
^CWARN: Device disconnected
 %  ~  USAGE  scrcpy --render-driver=software
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  3d97ed47                        device  23049PCD8G
/opt/homebrew/Cellar/scrcpy/2.4/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 221.6 MB/s (69007 bytes in 0.000s)
[server] INFO: Device: [Xiaomi] POCO 23049PCD8G (Android 14)
INFO: Renderer: software
INFO: Texture: 1080x2400
^CWARN: Device disconnected
 %  ~  USAGE  scrcpy --render-driver=metal
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     -->   (usb)  3d97ed47                        device  23049PCD8G
/opt/homebrew/Cellar/scrcpy/2.4/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 230.9 MB/s (69007 bytes in 0.000s)
[server] INFO: Device: [Xiaomi] POCO 23049PCD8G (Android 14)
INFO: Renderer: metal
INFO: Texture: 1080x2400

@ssalenik
Copy link

I've experienced the same issue before (not with scrcpy) and it was the encoder which was at fault (switching from the hardware encoder to x264 fixed it). I don't know if its possible the device is somehow defaulting to a different h264 encoder on A14? Though not sure why recording to a .mp4 would fix it.

@carstenhag
Copy link
Author

scrcpy --video-codec=h265 and scrcpy --video-codec=h264 results in the mentioned error, with scrcpy --video-codec=av1 the colors look fine.

@rom1v
Copy link
Collaborator

rom1v commented Mar 12, 2024

I don't know if its possible the device is somehow defaulting to a different h264 encoder on A14?

You can list and change the encoder.

it was the encoder which was at fault (switching from the hardware encoder to x264 fixed it). […] Though not sure why recording to a .mp4 would fix it.

Maybe your hardware encoder encodes with different parameters and set some metadata to handle color, and scrcpy does not handle them (either during the YUV-to-RGB conversion, either during rendering) while VLC does.

If you could try the same device on another computer with another OS (on Linux for example), that might help (or not).

@yume-chan
Copy link
Contributor

yume-chan commented Mar 13, 2024

Same as #4282

Maybe the device captures in HDR but SDL can't render them correctly?

@pain414410797
Copy link

I had the same problem, scrcpy and android studio mirror screens have a noticeable color difference
😂
image

@carstenhag
Copy link
Author

@pain414410797 what Xiaomi phone are you using?

@pain414410797
Copy link

Phone:Redmi K50 Gaming
OS:Xiaomi HyperOS 1.0.1.0
1710749352016

@yume-chan
Copy link
Contributor

yume-chan commented Mar 18, 2024

Some newer XiaoMi devices encode in full color range. I found this fix, but updating it in every frame might not be the most efficient method: (apparently libsdl-org/SDL#7083 is not true, FFMpeg and SDL both use JPEG to refer full color range)

diff --git a/app/src/display.c b/app/src/display.c
index c8df615d..ccc65902 100644
--- a/app/src/display.c
+++ b/app/src/display.c
@@ -199,6 +199,13 @@ sc_display_set_texture_size(struct sc_display *display, struct sc_size size) {
 static bool
 sc_display_update_texture_internal(struct sc_display *display,
                                    const AVFrame *frame) {
+    if (frame->color_range == AVCOL_RANGE_JPEG){
+      LOGE("JPEG color range detected");
+      SDL_SetYUVConversionMode(SDL_YUV_CONVERSION_JPEG);
+    } else {
+      SDL_SetYUVConversionMode(SDL_YUV_CONVERSION_AUTOMATIC);
+    }
+
     int ret = SDL_UpdateYUVTexture(display->texture, NULL,
                                    frame->data[0], frame->linesize[0],
                                    frame->data[1], frame->linesize[1],

AVCodecContext also has a color_range field, but seems that's not populated when decoding.

Recoding from Mi 11 (color is correct in Scrcpy 2.4): limited.zip

MediaInfo:
Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.2
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 s 100 ms
Width                                    : 1 440 pixels
Height                                   : 3 200 pixels
Display aspect ratio                     : 0.450
Frame rate mode                          : Constant
Frame rate                               : 10.000 FPS
Standard                                 : NTSC
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Default                                  : No
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.601 PAL
Transfer characteristics                 : BT.601
Matrix coefficients                      : BT.470 System B/G

Recording from Redmi K60 (color not correct in Scrcpy 2.4): full.zip

MediaInfo:
Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.2
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 s 100 ms
Width                                    : 1 440 pixels
Height                                   : 3 200 pixels
Display aspect ratio                     : 0.450
Frame rate mode                          : Constant
Frame rate                               : 10.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Default                                  : No
Forced                                   : No
Color range                              : Full
Color primaries                          : BT.601 PAL
Transfer characteristics                 : BT.601
Matrix coefficients                      : BT.470 System B/G
@rom1v
Copy link
Collaborator

rom1v commented Mar 18, 2024

👍

updating it in every frame might not be the most efficient method

Although SDL_SetYUVConversionMode() just writes the provided value (so it should be fast), the color range should be known in AVCodecContext, which is given here:

scrcpy/app/src/screen.c

Lines 281 to 283 in 79968a0

static bool
sc_screen_frame_sink_open(struct sc_frame_sink *sink,
const AVCodecContext *ctx) {

@yume-chan
Copy link
Contributor

yume-chan commented Mar 18, 2024

@rom1v I have tried exactly there, but ctx->color_range is 0. When sc_screen_frame_sink_open was called, ctx is only initialized with codec ID. color_range should come from the config packet, so I added this log:

diff --git a/app/src/decoder.c b/app/src/decoder.c
index 5d42b8b0..123f9d44 100644
--- a/app/src/decoder.c
+++ b/app/src/decoder.c
@@ -43,6 +43,10 @@ sc_decoder_push(struct sc_decoder *decoder, const AVPacket *packet) {
         return true;
     }
 
+    if (decoder->ctx->width != 0) {
+        LOGE("color range before avcodec_send_packet: %d", decoder->ctx->color_range);
+    }
+
     int ret = avcodec_send_packet(decoder->ctx, packet);
     if (ret < 0 && ret != AVERROR(EAGAIN)) {
         LOGE("Decoder '%s': could not send video packet: %d",
@@ -50,6 +54,10 @@ sc_decoder_push(struct sc_decoder *decoder, const AVPacket *packet) {
         return false;
     }
 
+    if (decoder->ctx->width != 0) {
+        LOGE("color range after avcodec_send_packet: %d", decoder->ctx->color_range);
+    }
+
     for (;;) {
         ret = avcodec_receive_frame(decoder->ctx, decoder->frame);
         if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {

Output:

> ./scrcpy -s af176a0b
scrcpy 2.4 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:           (usb)  73294bba                        device  M2011K2C
INFO:           (usb)  M01046406Y2031500412            device  mblu10
INFO:     -->   (usb)  af176a0b                        device  23013RK75C
D:\dev\yume-chan\scrcpy-devcontainer\scrcpy\dist\scrcpy-wi... file pushed, 0 skipped. 86.4 MB/s (69007 bytes in 0.001s)
[server] INFO: Device: [Xiaomi] Redmi 23013RK75C (Android 14)
INFO: Renderer: direct3d
INFO: Texture: 1440x3200
ERROR: color range before avcodec_send_packet: 0
ERROR: color range after avcodec_send_packet: 2
ERROR: color range before avcodec_send_packet: 2
ERROR: color range after avcodec_send_packet: 2
ERROR: color range before avcodec_send_packet: 2
ERROR: color range after avcodec_send_packet: 2
ERROR: color range before avcodec_send_packet: 2
ERROR: color range after avcodec_send_packet: 2

It was changed in the first avcodec_send_packet call.

@yume-chan
Copy link
Contributor

Or always request the encoder to encode in limited range?

diff --git a/server/src/main/java/com/genymobile/scrcpy/SurfaceEncoder.java b/server/src/main/java/com/genymobile/scrcpy/SurfaceEncoder.java
index 28435c09..467406fd 100644
--- a/server/src/main/java/com/genymobile/scrcpy/SurfaceEncoder.java
+++ b/server/src/main/java/com/genymobile/scrcpy/SurfaceEncoder.java
@@ -219,6 +219,9 @@ public class SurfaceEncoder implements AsyncProcessor {
         format.setInteger(MediaFormat.KEY_BIT_RATE, bitRate);
         // must be present to configure the encoder, but does not impact the actual frame rate, which is variable
         format.setInteger(MediaFormat.KEY_FRAME_RATE, 60);
+        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
+            format.setInteger(MediaFormat.KEY_COLOR_RANGE, MediaFormat.COLOR_RANGE_LIMITED);
+        }
         format.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);
         format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, DEFAULT_I_FRAME_INTERVAL);
         // display the very first frame, and recover from bad quality when no new frames
@rom1v
Copy link
Collaborator

rom1v commented Mar 18, 2024

Why not 😄

@carstenhag Can you confirm that this command fixes your issue:

scrcpy --video-codec-options=color-range=2

?

(btw, I'm able to reproduce the issue with my device using scrcpy --video-codec-options=color-range=1)

@carstenhag
Copy link
Author

Yep works fine with scrcpy --video-codec-options=color-range=2

@carstenhag carstenhag changed the title [Poco F5 Android 14] Light grays appearing as white only for scrcpy Mar 18, 2024
@rom1v
Copy link
Collaborator

rom1v commented Mar 19, 2024

@yume-chan Does the patch from your comment work for you?

I tested with --video-codec-options=color-range=1, I get the log JPEG color range detected, but visually it's incorrect (the same as without the patch). 🤔

@yume-chan
Copy link
Contributor

It works on Redmi K60 (Android 14), but not on Mi 11 (Android 13) with --video-codec-options=color-range=1.

If I record Mi 11 with ./scrcpy -s 73294bba --video-codec-options=color-range=1 -r m11_full.mp4 and play it in Windows Media Player, the color is incorrect, but also same as Scrcpy with my patch applied (#ffffff becomes #ebebeb. However Chrome can play the recording with correct color (white is still white).

image

Here is Redmi K60, recorded with ./scrcpy -s af176a0b -r k60_full.mp4, both the video and Scrcpy window (with my patch) have the correct color:

image

@rom1v
Copy link
Collaborator

rom1v commented Mar 19, 2024

both the video and Scrcpy window (with my patch) have the correct color:

OK, so if your patch fixes the problem, calling SDL_SetYUVConversionMode() fixes the problem on your machine, but it does nothing on mine. So I guess the YUV conversion is not properly applied on Linux/X11/OpenGL (SDL bug?)…

EDIT: it works for me with your patch and:

scrcpy --no-audio --video-codec-options=color-range=1 --render=driver=opengles2
scrcpy --no-audio --video-codec-options=color-range=1 --render=driver=software

But not the default:

scrcpy --no-audio --video-codec-options=color-range=1 --render=driver=opengl
rom1v added a commit that referenced this issue Mar 30, 2024
Take the color range (full vs limited) into account to render the
picture.

Note that with the current version of SDL, it has no impact with the SDL
opengl render driver.

Fixes #4756 <#4756>
Refs <#4756 (comment)>
Refs libusb/#9311 <libsdl-org/SDL#9311>

Suggested-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
rom1v pushed a commit that referenced this issue Mar 30, 2024
Most devices currently use limited color range, but some recent devices
encode in full color range, which is currently not supported by the SDL
opengl render driver.

Fixes #4756 <#4756>
Refs <#4756 (comment)>
Refs libusb/#9311 <libsdl-org/SDL#9311>

Signed-off-by: Romain Vimont <rom@rom1v.com>
@rom1v
Copy link
Collaborator

rom1v commented Mar 30, 2024

@yume-chan I suggest to apply both changes (branch yuv):

  • db55edb: fix the rendering (even if it is currently not supported by the SDL opengl render driver)
  • bf62579: request limited range by default (unless the user explicitly requests --video-codec-options=color-range=1
@yume-chan
Copy link
Contributor

I suggest to apply both changes

Sounds good to me.

@rom1v
Copy link
Collaborator

rom1v commented Apr 1, 2024

Merged into dev 🚀

@rom1v rom1v closed this as completed Apr 1, 2024
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this issue Aug 2, 2024
Take the color range (full vs limited) into account to render the
picture.

Note that with the current version of SDL, it has no impact with the SDL
opengl render driver.

Fixes Genymobile#4756 <Genymobile#4756>
Refs <Genymobile#4756 (comment)>
Refs libusb/#9311 <libsdl-org/SDL#9311>

Suggested-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this issue Aug 2, 2024
Most devices currently use limited color range, but some recent devices
encode in full color range, which is currently not supported by the SDL
opengl render driver.

Fixes Genymobile#4756 <Genymobile#4756>
Refs <Genymobile#4756 (comment)>
Refs libusb/#9311 <libsdl-org/SDL#9311>

Signed-off-by: Romain Vimont <rom@rom1v.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
5 participants