Compare commits

..

23 Commits

Author SHA1 Message Date
Romain Vimont
a92a712bf0 Enforce monotonical PTS
Audio PTS may come from two sources:
 - recorder.getTimestamp() if the call works;
 - an estimation from the previous PTS and the packet size as a
   fallback.

Therefore, the property that PTS are monotonically increasing is no
guaranteed in corner cases, so enforce it.
2023-02-03 13:08:29 +01:00
Romain Vimont
bf7eba97e6 fixup! AudioEncoder WIP 2023-02-03 13:04:31 +01:00
Romain Vimont
709c24bfc3 fixup! AudioEncoder WIP 2023-02-03 13:04:19 +01:00
Romain Vimont
ad5a02e93a fixup! AudioEncoder WIP 2023-02-03 13:04:06 +01:00
Romain Vimont
5e3e2cdd02 AudioEncoder WIP 2023-02-03 12:59:38 +01:00
Simon Chan
aa30eadc7b Capture device audio WIP
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-02-03 12:59:16 +01:00
Romain Vimont
eb1830449e Use FakeContext for Application instance
This will expose the correct package name and UID to the application
context.
2023-02-03 12:59:01 +01:00
Romain Vimont
56df8cd48f Use shell package name for workarounds
For consistency.
2023-02-03 12:58:57 +01:00
Romain Vimont
6c406c9319 Use PACKAGE_NAME from FakeContext
Remove duplicated constant.
2023-02-03 12:58:54 +01:00
Romain Vimont
44e697cd9a Use AttributionSource from FakeContext
FakeContext already provides an AttributeSource instance.

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-02-03 12:58:52 +01:00
Simon Chan
132be1a81b Add a fake Android Context
Since scrcpy-server is not an Android application (it's a java
executable), it has no Context.

Some features will require a Context instance to get the package name
and the UID. Add a FakeContext for this purpose.

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-02-03 12:58:45 +01:00
Romain Vimont
5138ce75aa Use Process.ROOT_UID
Replace ServiceManager.USER_ID by existing constant Process.ROOT_UID.
2023-02-03 12:58:36 +01:00
Romain Vimont
ff3c3670b2 Add support for AV1
Add option --codec=av1.
2023-02-03 12:54:44 +01:00
Romain Vimont
e50e409405 Add support for H265
Add option --codec=h265.
2023-02-03 12:54:44 +01:00
Romain Vimont
07b65038a1 Add option to select video codec
Introduce the selection mechanism. Alternative codecs will be added in
further commits.
2023-02-03 12:54:27 +01:00
Romain Vimont
f70f6cdd3e Simplify server info initialization
Use sc_read16be() to read 16-bit integer fields.
2023-02-03 12:31:28 +01:00
Romain Vimont
87972e2022 Extract video streaming to a separate class
ScreenEncoder handled both capture/encoding and sending over the
network.

Move the streaming part to a separate VideoStreamer.
2023-02-03 12:31:28 +01:00
Romain Vimont
3aac74e9e9 Move variable assignment
Computing eof flag is not necessary if rotation changed.
2023-02-03 12:31:28 +01:00
Romain Vimont
1c82c3923d Compute relative PTS on the client-side
The PTS received from MediaCodec are expressed relative to an arbitrary
clock origin. We consider the PTS of the first frame to be 0, and the
PTS of every other frame is relative to this first PTS (note that the
PTS is only used for recording, it is ignored for mirroring).

For simplicity, this relative PTS was computed on the server-side.

To prepare support for multiple stream (video and audio), send the
packet with its original PTS, and handle the PTS offset on the
client-side (by the recorder).

Since we can't know in advance which stream will produce the first
packet with the lowest PTS (a packet received later on one stream may
have a PTS lower than a packet received earlier on another stream),
computing the PTS on the server-side would require unnecessary waiting.
2023-02-03 12:31:28 +01:00
Romain Vimont
36d656e91f Improve workarounds call comments 2023-02-03 12:31:28 +01:00
Romain Vimont
bdbf1f4eb7 Move Workarounds call
Workarounds are not specific to the screen encoder.

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-02-03 12:31:28 +01:00
Romain Vimont
4177de5880 Do not expose controller threads
The way the controller executes its events asynchronously is an
implementation detail.
2023-02-03 12:31:28 +01:00
Romain Vimont
6a07e3d470 Fix manpage formatting
Only the option arguments must be underlined.
2023-02-03 12:31:28 +01:00
24 changed files with 658 additions and 170 deletions

View File

@@ -194,6 +194,18 @@ The other dimension is computed so that the Android device aspect ratio is
preserved. That way, a device in 1920×1080 will be mirrored at 1024×576. preserved. That way, a device in 1920×1080 will be mirrored at 1024×576.
#### Select codec
The video codec can be selected. The possible values are `h264` (default),
`h265` and `av1`:
```bash
scrcpy --codec=h264 # default
scrcpy --codec=h265
scrcpy --codec=av1
```
#### Change bit-rate #### Change bit-rate
The default bit-rate is 8 Mbps. To change the video bitrate (e.g. to 2 Mbps): The default bit-rate is 8 Mbps. To change the video bitrate (e.g. to 2 Mbps):
@@ -254,8 +266,8 @@ The [window may also be rotated](#rotation) independently.
#### Encoder #### Encoder
Some devices have more than one encoder, and some of them may cause issues or Some devices have more than one encoder for a specific codec, and some of them
crash. It is possible to select a different encoder: may cause issues or crash. It is possible to select a different encoder:
```bash ```bash
scrcpy --encoder=OMX.qcom.video.encoder.avc scrcpy --encoder=OMX.qcom.video.encoder.avc
@@ -268,6 +280,8 @@ error will give the available encoders:
scrcpy --encoder=_ scrcpy --encoder=_
``` ```
Note that you can also select a different [codec](#select-codec).
### Capture ### Capture
#### Recording #### Recording

View File

@@ -26,7 +26,11 @@ Encode the video at the given bit\-rate, expressed in bits/s. Unit suffixes are
Default is 8000000. Default is 8000000.
.TP .TP
.BI "\-\-codec\-options " key[:type]=value[,...] .BI "\-\-codec " name
Select a video codec (h264, h265 or av1).
.TP
.BI "\-\-codec\-options " key\fR[:\fItype\fR]=\fIvalue\fR[,...]
Set a list of comma-separated key:type=value options for the device encoder. Set a list of comma-separated key:type=value options for the device encoder.
The possible values for 'type' are 'int' (default), 'long', 'float' and 'string'. The possible values for 'type' are 'int' (default), 'long', 'float' and 'string'.
@@ -117,7 +121,7 @@ Inject computer clipboard text as a sequence of key events on Ctrl+v (like MOD+S
This is a workaround for some devices not behaving as expected when setting the device clipboard programmatically. This is a workaround for some devices not behaving as expected when setting the device clipboard programmatically.
.TP .TP
.BI "\-\-lock\-video\-orientation[=value] \fB\-\-lock\-video\-orientation\fR[=\fIvalue\fR]
Lock video orientation to \fIvalue\fR. Possible values are "unlocked", "initial" (locked to the initial orientation), 0, 1, 2 and 3. Natural device orientation is 0, and each increment adds a 90 degrees rotation counterclockwise. Lock video orientation to \fIvalue\fR. Possible values are "unlocked", "initial" (locked to the initial orientation), 0, 1, 2 and 3. Natural device orientation is 0, and each increment adds a 90 degrees rotation counterclockwise.
Default is "unlocked". Default is "unlocked".
@@ -199,7 +203,7 @@ It may only work over USB.
See \fB\-\-hid\-keyboard\fR and \fB\-\-hid\-mouse\fR. See \fB\-\-hid\-keyboard\fR and \fB\-\-hid\-mouse\fR.
.TP .TP
.BI "\-p, \-\-port " port[:port] .BI "\-p, \-\-port " port\fR[:\fIport\fR]
Set the TCP port (range) used by the client to listen. Set the TCP port (range) used by the client to listen.
Default is 27183:27199. Default is 27183:27199.
@@ -260,7 +264,7 @@ Set the initial display rotation. Possibles values are 0, 1, 2 and 3. Each incre
The device serial number. Mandatory only if several devices are connected to adb. The device serial number. Mandatory only if several devices are connected to adb.
.TP .TP
.BI "\-\-shortcut\-mod " key[+...]][,...] .BI "\-\-shortcut\-mod " key\fR[+...]][,...]
Specify the modifiers to use for scrcpy shortcuts. Possible keys are "lctrl", "rctrl", "lalt", "ralt", "lsuper" and "rsuper". Specify the modifiers to use for scrcpy shortcuts. Possible keys are "lctrl", "rctrl", "lalt", "ralt", "lsuper" and "rsuper".
A shortcut can consist in several keys, separated by '+'. Several shortcuts can be specified, separated by ','. A shortcut can consist in several keys, separated by '+'. Several shortcuts can be specified, separated by ','.
@@ -270,7 +274,7 @@ For example, to use either LCtrl+LAlt or LSuper for scrcpy shortcuts, pass "lctr
Default is "lalt,lsuper" (left-Alt or left-Super). Default is "lalt,lsuper" (left-Alt or left-Super).
.TP .TP
.BI "\-\-tcpip[=ip[:port]] .BI "\-\-tcpip\fR[=\fIip\fR[:\fIport\fR]]
Configure and reconnect the device over TCP/IP. Configure and reconnect the device over TCP/IP.
If a destination address is provided, then scrcpy connects to this address before starting. The device must listen on the given TCP port (default is 5555). If a destination address is provided, then scrcpy connects to this address before starting. The device must listen on the given TCP port (default is 5555).

View File

@@ -57,6 +57,7 @@
#define OPT_NO_CLEANUP 1037 #define OPT_NO_CLEANUP 1037
#define OPT_PRINT_FPS 1038 #define OPT_PRINT_FPS 1038
#define OPT_NO_POWER_ON 1039 #define OPT_NO_POWER_ON 1039
#define OPT_CODEC 1040
struct sc_option { struct sc_option {
char shortopt; char shortopt;
@@ -105,6 +106,12 @@ static const struct sc_option options[] = {
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n" "Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is " STR(DEFAULT_BIT_RATE) ".", "Default is " STR(DEFAULT_BIT_RATE) ".",
}, },
{
.longopt_id = OPT_CODEC,
.longopt = "codec",
.argdesc = "name",
.text = "Select a video codec (h264, h265 or av1).",
},
{ {
.longopt_id = OPT_CODEC_OPTIONS, .longopt_id = OPT_CODEC_OPTIONS,
.longopt = "codec-options", .longopt = "codec-options",
@@ -1377,6 +1384,24 @@ guess_record_format(const char *filename) {
return 0; return 0;
} }
static bool
parse_codec(const char *optarg, enum sc_codec *codec) {
if (!strcmp(optarg, "h264")) {
*codec = SC_CODEC_H264;
return true;
}
if (!strcmp(optarg, "h265")) {
*codec = SC_CODEC_H265;
return true;
}
if (!strcmp(optarg, "av1")) {
*codec = SC_CODEC_AV1;
return true;
}
LOGE("Unsupported codec: %s (expected h264, h265 or av1)", optarg);
return false;
}
static bool static bool
parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[], parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
const char *optstring, const struct option *longopts) { const char *optstring, const struct option *longopts) {
@@ -1610,6 +1635,11 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
case OPT_PRINT_FPS: case OPT_PRINT_FPS:
opts->start_fps_counter = true; opts->start_fps_counter = true;
break; break;
case OPT_CODEC:
if (!parse_codec(optarg, &opts->codec)) {
return false;
}
break;
case OPT_OTG: case OPT_OTG:
#ifdef HAVE_USB #ifdef HAVE_USB
opts->otg = true; opts->otg = true;
@@ -1718,6 +1748,13 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
} }
} }
if (opts->record_format == SC_RECORD_FORMAT_MP4
&& opts->codec == SC_CODEC_AV1) {
LOGE("Could not mux AV1 stream into MP4 container "
"(record to mkv or select another video codec)");
return false;
}
if (!opts->control) { if (!opts->control) {
if (opts->turn_screen_off) { if (opts->turn_screen_off) {
LOGE("Could not request to turn screen off if control is disabled"); LOGE("Could not request to turn screen off if control is disabled");

View File

@@ -17,6 +17,31 @@
#define SC_PACKET_PTS_MASK (SC_PACKET_FLAG_KEY_FRAME - 1) #define SC_PACKET_PTS_MASK (SC_PACKET_FLAG_KEY_FRAME - 1)
static enum AVCodecID
sc_demuxer_recv_codec_id(struct sc_demuxer *demuxer) {
uint8_t data[4];
ssize_t r = net_recv_all(demuxer->socket, data, 4);
if (r < 4) {
return false;
}
#define SC_CODEC_ID_H264 UINT32_C(0x68323634) // "h264" in ASCII
#define SC_CODEC_ID_H265 UINT32_C(0x68323635) // "h265" in ASCII
#define SC_CODEC_ID_AV1 UINT32_C(0x00617631) // "av1" in ASCII
uint32_t codec_id = sc_read32be(data);
switch (codec_id) {
case SC_CODEC_ID_H264:
return AV_CODEC_ID_H264;
case SC_CODEC_ID_H265:
return AV_CODEC_ID_HEVC;
case SC_CODEC_ID_AV1:
return AV_CODEC_ID_AV1;
default:
LOGE("Unknown codec id 0x%08" PRIx32, codec_id);
return AV_CODEC_ID_NONE;
}
}
static bool static bool
sc_demuxer_recv_packet(struct sc_demuxer *demuxer, AVPacket *packet) { sc_demuxer_recv_packet(struct sc_demuxer *demuxer, AVPacket *packet) {
// The video stream contains raw packets, without time information. When we // The video stream contains raw packets, without time information. When we
@@ -171,7 +196,13 @@ static int
run_demuxer(void *data) { run_demuxer(void *data) {
struct sc_demuxer *demuxer = data; struct sc_demuxer *demuxer = data;
const AVCodec *codec = avcodec_find_decoder(AV_CODEC_ID_H264); enum AVCodecID codec_id = sc_demuxer_recv_codec_id(demuxer);
if (codec_id == AV_CODEC_ID_NONE) {
// Error already logged
goto end;
}
const AVCodec *codec = avcodec_find_decoder(codec_id);
if (!codec) { if (!codec) {
LOGE("H.264 decoder not found"); LOGE("H.264 decoder not found");
goto end; goto end;
@@ -188,7 +219,7 @@ run_demuxer(void *data) {
goto finally_free_codec_ctx; goto finally_free_codec_ctx;
} }
demuxer->parser = av_parser_init(AV_CODEC_ID_H264); demuxer->parser = av_parser_init(codec_id);
if (!demuxer->parser) { if (!demuxer->parser) {
LOGE("Could not initialize parser"); LOGE("Could not initialize parser");
goto finally_close_sinks; goto finally_close_sinks;

View File

@@ -13,6 +13,7 @@ const struct scrcpy_options scrcpy_options_default = {
.v4l2_device = NULL, .v4l2_device = NULL,
#endif #endif
.log_level = SC_LOG_LEVEL_INFO, .log_level = SC_LOG_LEVEL_INFO,
.codec = SC_CODEC_H264,
.record_format = SC_RECORD_FORMAT_AUTO, .record_format = SC_RECORD_FORMAT_AUTO,
.keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT, .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT,
.port_range = { .port_range = {

View File

@@ -23,6 +23,12 @@ enum sc_record_format {
SC_RECORD_FORMAT_MKV, SC_RECORD_FORMAT_MKV,
}; };
enum sc_codec {
SC_CODEC_H264,
SC_CODEC_H265,
SC_CODEC_AV1,
};
enum sc_lock_video_orientation { enum sc_lock_video_orientation {
SC_LOCK_VIDEO_ORIENTATION_UNLOCKED = -1, SC_LOCK_VIDEO_ORIENTATION_UNLOCKED = -1,
// lock the current orientation when scrcpy starts // lock the current orientation when scrcpy starts
@@ -93,6 +99,7 @@ struct scrcpy_options {
const char *v4l2_device; const char *v4l2_device;
#endif #endif
enum sc_log_level log_level; enum sc_log_level log_level;
enum sc_codec codec;
enum sc_record_format record_format; enum sc_record_format record_format;
enum sc_keyboard_input_mode keyboard_input_mode; enum sc_keyboard_input_mode keyboard_input_mode;
enum sc_mouse_input_mode mouse_input_mode; enum sc_mouse_input_mode mouse_input_mode;

View File

@@ -130,8 +130,6 @@ sc_recorder_write(struct sc_recorder *recorder, AVPacket *packet) {
return true; return true;
} }
LOGI("==== %" PRIu64, packet->pts);
sc_recorder_rescale_packet(recorder, packet); sc_recorder_rescale_packet(recorder, packet);
return av_write_frame(recorder->ctx, packet) >= 0; return av_write_frame(recorder->ctx, packet) >= 0;
} }

View File

@@ -315,6 +315,7 @@ scrcpy(struct scrcpy_options *options) {
.select_usb = options->select_usb, .select_usb = options->select_usb,
.select_tcpip = options->select_tcpip, .select_tcpip = options->select_tcpip,
.log_level = options->log_level, .log_level = options->log_level,
.codec = options->codec,
.crop = options->crop, .crop = options->crop,
.port_range = options->port_range, .port_range = options->port_range,
.tunnel_host = options->tunnel_host, .tunnel_host = options->tunnel_host,

View File

@@ -8,6 +8,7 @@
#include <SDL2/SDL_platform.h> #include <SDL2/SDL_platform.h>
#include "adb/adb.h" #include "adb/adb.h"
#include "util/binary.h"
#include "util/file.h" #include "util/file.h"
#include "util/log.h" #include "util/log.h"
#include "util/net_intr.h" #include "util/net_intr.h"
@@ -155,6 +156,20 @@ sc_server_sleep(struct sc_server *server, sc_tick deadline) {
return !stopped; return !stopped;
} }
static const char *
sc_server_get_codec_name(enum sc_codec codec) {
switch (codec) {
case SC_CODEC_H264:
return "h264";
case SC_CODEC_H265:
return "h265";
case SC_CODEC_AV1:
return "av1";
default:
return NULL;
}
}
static sc_pid static sc_pid
execute_server(struct sc_server *server, execute_server(struct sc_server *server,
const struct sc_server_params *params) { const struct sc_server_params *params) {
@@ -202,6 +217,9 @@ execute_server(struct sc_server *server,
ADD_PARAM("log_level=%s", log_level_to_server_string(params->log_level)); ADD_PARAM("log_level=%s", log_level_to_server_string(params->log_level));
ADD_PARAM("bit_rate=%" PRIu32, params->bit_rate); ADD_PARAM("bit_rate=%" PRIu32, params->bit_rate);
if (params->codec != SC_CODEC_H264) {
ADD_PARAM("codec=%s", sc_server_get_codec_name(params->codec));
}
if (params->max_size) { if (params->max_size) {
ADD_PARAM("max_size=%" PRIu16, params->max_size); ADD_PARAM("max_size=%" PRIu16, params->max_size);
} }
@@ -398,10 +416,9 @@ device_read_info(struct sc_intr *intr, sc_socket device_socket,
buf[SC_DEVICE_NAME_FIELD_LENGTH - 1] = '\0'; buf[SC_DEVICE_NAME_FIELD_LENGTH - 1] = '\0';
memcpy(info->device_name, (char *) buf, sizeof(info->device_name)); memcpy(info->device_name, (char *) buf, sizeof(info->device_name));
info->frame_size.width = (buf[SC_DEVICE_NAME_FIELD_LENGTH] << 8) unsigned char *fields = &buf[SC_DEVICE_NAME_FIELD_LENGTH];
| buf[SC_DEVICE_NAME_FIELD_LENGTH + 1]; info->frame_size.width = sc_read16be(fields);
info->frame_size.height = (buf[SC_DEVICE_NAME_FIELD_LENGTH + 2] << 8) info->frame_size.height = sc_read16be(&fields[2]);
| buf[SC_DEVICE_NAME_FIELD_LENGTH + 3];
return true; return true;
} }

View File

@@ -25,6 +25,7 @@ struct sc_server_params {
uint32_t uid; uint32_t uid;
const char *req_serial; const char *req_serial;
enum sc_log_level log_level; enum sc_log_level log_level;
enum sc_codec codec;
const char *crop; const char *crop;
const char *codec_options; const char *codec_options;
const char *encoder_name; const char *encoder_name;

View File

@@ -0,0 +1,207 @@
package com.genymobile.scrcpy;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import android.annotation.SuppressLint;
import android.annotation.TargetApi;
import android.content.ComponentName;
import android.content.Intent;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.AudioTimestamp;
import android.media.MediaCodec;
import android.media.MediaFormat;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.Handler;
import android.os.HandlerThread;
import android.os.SystemClock;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.concurrent.Semaphore;
import java.util.concurrent.atomic.AtomicBoolean;
public final class AudioEncoder {
private static final String MIMETYPE = MediaFormat.MIMETYPE_AUDIO_OPUS;
private static final int SAMPLE_RATE = 48000;
private static final int CHANNELS = 2;
private static final int BIT_RATE = 128000;
private static int BUFFER_MS = 15; // milliseconds
private static final int BUFFER_SIZE = SAMPLE_RATE * CHANNELS * BUFFER_MS / 1000;
private AudioRecord recorder;
private MediaCodec mediaCodec;
private HandlerThread thread;
private final AtomicBoolean interrupted = new AtomicBoolean();
private final Semaphore endSemaphore = new Semaphore(0); // blocks until encoding is ended
private static AudioFormat createAudioFormat() {
AudioFormat.Builder builder = new AudioFormat.Builder();
builder.setEncoding(AudioFormat.ENCODING_PCM_16BIT);
builder.setSampleRate(SAMPLE_RATE);
builder.setChannelMask(CHANNELS == 2 ? AudioFormat.CHANNEL_IN_STEREO : AudioFormat.CHANNEL_IN_MONO);
return builder.build();
}
@TargetApi(Build.VERSION_CODES.M)
@SuppressLint({"WrongConstant", "MissingPermission"})
private static AudioRecord createAudioRecord() {
AudioRecord.Builder builder = new AudioRecord.Builder();
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
// On older APIs, Workarounds.fillAppInfo() must be called beforehand
builder.setContext(FakeContext.get());
}
builder.setAudioSource(MediaRecorder.AudioSource.REMOTE_SUBMIX);
builder.setAudioFormat(createAudioFormat());
builder.setBufferSizeInBytes(1024 * 1024);
return builder.build();
}
private static MediaFormat createFormat() {
MediaFormat format = new MediaFormat();
format.setString(MediaFormat.KEY_MIME, MIMETYPE);
format.setInteger(MediaFormat.KEY_BIT_RATE, BIT_RATE);
format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, CHANNELS);
format.setInteger(MediaFormat.KEY_SAMPLE_RATE, SAMPLE_RATE);
return format;
}
@TargetApi(Build.VERSION_CODES.M)
public void start() throws IOException {
mediaCodec = MediaCodec.createEncoderByType(MIMETYPE); // may throw IOException
recorder = createAudioRecord();
MediaFormat format = createFormat();
mediaCodec.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
recorder.startRecording();
thread = new HandlerThread("AudioEncoder");
thread.start();
class AudioEncoderCallbacks extends MediaCodec.Callback {
private final AudioTimestamp timestamp = new AudioTimestamp();
private long previousPts;
private long nextPts;
private boolean eofSignaled;
private boolean ended;
private void notifyEnded() {
assert !ended;
ended = true;
endSemaphore.release();
}
@TargetApi(Build.VERSION_CODES.N)
@Override
public void onInputBufferAvailable(MediaCodec codec, int index) {
if (eofSignaled) {
return;
}
ByteBuffer inputBuffer = codec.getInputBuffer(index);
int r = recorder.read(inputBuffer, BUFFER_SIZE);
long pts;
int ret = recorder.getTimestamp(timestamp, AudioTimestamp.TIMEBASE_MONOTONIC);
if (ret == AudioRecord.SUCCESS) {
pts = timestamp.nanoTime / 1000;
} else {
if (nextPts == 0) {
Ln.w("Could not get any audio timestamp");
}
// compute from previous timestamp and packet size
pts = nextPts;
}
long durationMs = r * 1000 / CHANNELS / SAMPLE_RATE;
nextPts = pts + durationMs;
int flags = 0;
if (interrupted.get()) {
flags = flags | MediaCodec.BUFFER_FLAG_END_OF_STREAM;
eofSignaled = true;
}
if (previousPts != 0 && pts < previousPts) {
// Audio PTS may come from two sources:
// - recorder.getTimestamp() if the call works;
// - an estimation from the previous PTS and the packet size as a fallback.
//
// Therefore, the property that PTS are monotonically increasing is no guaranteed in corner cases, so enforce it.
pts = previousPts + 1;
}
codec.queueInputBuffer(index, 0, r, pts, flags);
previousPts = pts;
}
@Override
public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo bufferInfo) {
if (ended) {
return;
}
ByteBuffer codecBuffer = codec.getOutputBuffer(index);
try {
boolean isConfig = (bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0;
long pts = bufferInfo.presentationTimeUs;
Ln.i("Audio packet: pts=" + pts + " " + codecBuffer.remaining() + " bytes");
} finally {
codec.releaseOutputBuffer(index, false);
}
boolean eof = (bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0;
if (eof) {
notifyEnded();
}
}
@Override
public void onError(MediaCodec codec, MediaCodec.CodecException e) {
Ln.e("MediaCodec error", e);
if (!ended) {
notifyEnded();
}
}
@Override
public void onOutputFormatChanged(MediaCodec codec, MediaFormat format) {
// ignore
}
}
mediaCodec.setCallback(new AudioEncoderCallbacks(), new Handler(thread.getLooper()));
mediaCodec.start();
}
private void waitEnded() {
try {
endSemaphore.acquire();
} catch (InterruptedException e) {
// ignore
}
}
public void stop() {
Ln.i("==== STOP");
if (thread != null) {
interrupted.set(true);
waitEnded();
thread.interrupt();
thread = null;
mediaCodec.stop();
mediaCodec.release();
recorder.stop();
Ln.i("==== STOPPED");
}
}
}

View File

@@ -24,6 +24,8 @@ public class Controller {
private static final ScheduledExecutorService EXECUTOR = Executors.newSingleThreadScheduledExecutor(); private static final ScheduledExecutorService EXECUTOR = Executors.newSingleThreadScheduledExecutor();
private Thread thread;
private final Device device; private final Device device;
private final DesktopConnection connection; private final DesktopConnection connection;
private final DeviceMessageSender sender; private final DeviceMessageSender sender;
@@ -62,7 +64,7 @@ public class Controller {
} }
} }
public void control() throws IOException { private void control() throws IOException {
// on start, power on the device // on start, power on the device
if (powerOn && !Device.isScreenOn()) { if (powerOn && !Device.isScreenOn()) {
device.pressReleaseKeycode(KeyEvent.KEYCODE_POWER, Device.INJECT_MODE_ASYNC); device.pressReleaseKeycode(KeyEvent.KEYCODE_POWER, Device.INJECT_MODE_ASYNC);
@@ -82,6 +84,27 @@ public class Controller {
} }
} }
public void start() {
thread = new Thread(() -> {
try {
control();
} catch (IOException e) {
// this is expected on close
Ln.d("Controller stopped");
}
});
thread.start();
sender.start();
}
public void stop() {
if (thread != null) {
thread.interrupt();
thread = null;
}
sender.stop();
}
public DeviceMessageSender getSender() { public DeviceMessageSender getSender() {
return sender; return sender;
} }

View File

@@ -6,6 +6,8 @@ public final class DeviceMessageSender {
private final DesktopConnection connection; private final DesktopConnection connection;
private Thread thread;
private String clipboardText; private String clipboardText;
private long ack; private long ack;
@@ -24,7 +26,7 @@ public final class DeviceMessageSender {
notify(); notify();
} }
public void loop() throws IOException, InterruptedException { private void loop() throws IOException, InterruptedException {
while (!Thread.currentThread().isInterrupted()) { while (!Thread.currentThread().isInterrupted()) {
String text; String text;
long sequence; long sequence;
@@ -49,4 +51,22 @@ public final class DeviceMessageSender {
} }
} }
} }
public void start() {
thread = new Thread(() -> {
try {
loop();
} catch (IOException | InterruptedException e) {
// this is expected on close
Ln.d("Device message sender stopped");
}
});
thread.start();
}
public void stop() {
if (thread != null) {
thread.interrupt();
thread = null;
}
}
} }

View File

@@ -0,0 +1,40 @@
package com.genymobile.scrcpy;
import android.annotation.TargetApi;
import android.content.AttributionSource;
import android.content.ContextWrapper;
import android.os.Build;
import android.os.Process;
public final class FakeContext extends ContextWrapper {
public static final String PACKAGE_NAME = "com.android.shell";
private static final FakeContext INSTANCE = new FakeContext();
public static FakeContext get() {
return INSTANCE;
}
private FakeContext() {
super(null);
}
@Override
public String getPackageName() {
return PACKAGE_NAME;
}
@Override
public String getOpPackageName() {
return PACKAGE_NAME;
}
@TargetApi(Build.VERSION_CODES.S)
@Override
public AttributionSource getAttributionSource() {
AttributionSource.Builder builder = new AttributionSource.Builder(Process.SHELL_UID);
builder.setPackageName(PACKAGE_NAME);
return builder.build();
}
}

View File

@@ -5,9 +5,12 @@ import android.graphics.Rect;
import java.util.List; import java.util.List;
public class Options { public class Options {
private static final String VIDEO_CODEC_H264 = "h264";
private Ln.Level logLevel = Ln.Level.DEBUG; private Ln.Level logLevel = Ln.Level.DEBUG;
private int uid = -1; // 31-bit non-negative value, or -1 private int uid = -1; // 31-bit non-negative value, or -1
private int maxSize; private int maxSize;
private VideoCodec codec = VideoCodec.H264;
private int bitRate = 8000000; private int bitRate = 8000000;
private int maxFps; private int maxFps;
private int lockVideoOrientation = -1; private int lockVideoOrientation = -1;
@@ -29,6 +32,7 @@ public class Options {
private boolean sendDeviceMeta = true; // send device name and size private boolean sendDeviceMeta = true; // send device name and size
private boolean sendFrameMeta = true; // send PTS so that the client may record properly private boolean sendFrameMeta = true; // send PTS so that the client may record properly
private boolean sendDummyByte = true; // write a byte on start to detect connection issues private boolean sendDummyByte = true; // write a byte on start to detect connection issues
private boolean sendCodecId = true; // write the codec ID (4 bytes) before the stream
public Ln.Level getLogLevel() { public Ln.Level getLogLevel() {
return logLevel; return logLevel;
@@ -54,6 +58,14 @@ public class Options {
this.maxSize = maxSize; this.maxSize = maxSize;
} }
public VideoCodec getCodec() {
return codec;
}
public void setCodec(VideoCodec codec) {
this.codec = codec;
}
public int getBitRate() { public int getBitRate() {
return bitRate; return bitRate;
} }
@@ -205,4 +217,12 @@ public class Options {
public void setSendDummyByte(boolean sendDummyByte) { public void setSendDummyByte(boolean sendDummyByte) {
this.sendDummyByte = sendDummyByte; this.sendDummyByte = sendDummyByte;
} }
public boolean getSendCodecId() {
return sendCodecId;
}
public void setSendCodecId(boolean sendCodecId) {
this.sendCodecId = sendCodecId;
}
} }

View File

@@ -12,7 +12,6 @@ import android.os.IBinder;
import android.os.SystemClock; import android.os.SystemClock;
import android.view.Surface; import android.view.Surface;
import java.io.FileDescriptor;
import java.io.IOException; import java.io.IOException;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
import java.util.ArrayList; import java.util.ArrayList;
@@ -22,6 +21,10 @@ import java.util.concurrent.atomic.AtomicBoolean;
public class ScreenEncoder implements Device.RotationListener { public class ScreenEncoder implements Device.RotationListener {
public interface Callbacks {
void onPacket(ByteBuffer codecBuffer, MediaCodec.BufferInfo bufferInfo) throws IOException;
}
private static final int DEFAULT_I_FRAME_INTERVAL = 10; // seconds private static final int DEFAULT_I_FRAME_INTERVAL = 10; // seconds
private static final int REPEAT_FRAME_DELAY_US = 100_000; // repeat after 100ms private static final int REPEAT_FRAME_DELAY_US = 100_000; // repeat after 100ms
private static final String KEY_MAX_FPS_TO_ENCODER = "max-fps-to-encoder"; private static final String KEY_MAX_FPS_TO_ENCODER = "max-fps-to-encoder";
@@ -30,25 +33,20 @@ public class ScreenEncoder implements Device.RotationListener {
private static final int[] MAX_SIZE_FALLBACK = {2560, 1920, 1600, 1280, 1024, 800}; private static final int[] MAX_SIZE_FALLBACK = {2560, 1920, 1600, 1280, 1024, 800};
private static final int MAX_CONSECUTIVE_ERRORS = 3; private static final int MAX_CONSECUTIVE_ERRORS = 3;
private static final long PACKET_FLAG_CONFIG = 1L << 63;
private static final long PACKET_FLAG_KEY_FRAME = 1L << 62;
private final AtomicBoolean rotationChanged = new AtomicBoolean(); private final AtomicBoolean rotationChanged = new AtomicBoolean();
private final ByteBuffer headerBuffer = ByteBuffer.allocate(12);
private final String videoMimeType;
private final String encoderName; private final String encoderName;
private final List<CodecOption> codecOptions; private final List<CodecOption> codecOptions;
private final int bitRate; private final int bitRate;
private final int maxFps; private final int maxFps;
private final boolean sendFrameMeta;
private final boolean downsizeOnError; private final boolean downsizeOnError;
private boolean firstFrameSent; private boolean firstFrameSent;
private int consecutiveErrors; private int consecutiveErrors;
public ScreenEncoder(boolean sendFrameMeta, int bitRate, int maxFps, List<CodecOption> codecOptions, String encoderName, public ScreenEncoder(String videoMimeType, int bitRate, int maxFps, List<CodecOption> codecOptions, String encoderName, boolean downsizeOnError) {
boolean downsizeOnError) { this.videoMimeType = videoMimeType;
this.sendFrameMeta = sendFrameMeta;
this.bitRate = bitRate; this.bitRate = bitRate;
this.maxFps = maxFps; this.maxFps = maxFps;
this.codecOptions = codecOptions; this.codecOptions = codecOptions;
@@ -65,20 +63,9 @@ public class ScreenEncoder implements Device.RotationListener {
return rotationChanged.getAndSet(false); return rotationChanged.getAndSet(false);
} }
public void streamScreen(Device device, FileDescriptor fd) throws IOException { public void streamScreen(Device device, Callbacks callbacks) throws IOException {
Workarounds.prepareMainLooper(); MediaCodec codec = createCodec(videoMimeType, encoderName);
if (Build.BRAND.equalsIgnoreCase("meizu")) { MediaFormat format = createFormat(videoMimeType, bitRate, maxFps, codecOptions);
// <https://github.com/Genymobile/scrcpy/issues/240>
// <https://github.com/Genymobile/scrcpy/issues/2656>
Workarounds.fillAppInfo();
}
internalStreamScreen(device, fd);
}
private void internalStreamScreen(Device device, FileDescriptor fd) throws IOException {
MediaCodec codec = createCodec(encoderName);
MediaFormat format = createFormat(bitRate, maxFps, codecOptions);
IBinder display = createDisplay(); IBinder display = createDisplay();
device.setRotationListener(this); device.setRotationListener(this);
boolean alive; boolean alive;
@@ -105,7 +92,7 @@ public class ScreenEncoder implements Device.RotationListener {
codec.start(); codec.start();
alive = encode(codec, fd); alive = encode(codec, callbacks);
// do not call stop() on exception, it would trigger an IllegalStateException // do not call stop() on exception, it would trigger an IllegalStateException
codec.stop(); codec.stop();
} catch (IllegalStateException | IllegalArgumentException e) { } catch (IllegalStateException | IllegalArgumentException e) {
@@ -174,31 +161,30 @@ public class ScreenEncoder implements Device.RotationListener {
return 0; return 0;
} }
private boolean encode(MediaCodec codec, FileDescriptor fd) throws IOException { private boolean encode(MediaCodec codec, Callbacks callbacks) throws IOException {
boolean eof = false; boolean eof = false;
MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
while (!consumeRotationChange() && !eof) { while (!consumeRotationChange() && !eof) {
int outputBufferId = codec.dequeueOutputBuffer(bufferInfo, -1); int outputBufferId = codec.dequeueOutputBuffer(bufferInfo, -1);
eof = (bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0;
try { try {
if (consumeRotationChange()) { if (consumeRotationChange()) {
// must restart encoding with new size // must restart encoding with new size
break; break;
} }
eof = (bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0;
if (outputBufferId >= 0) { if (outputBufferId >= 0) {
ByteBuffer codecBuffer = codec.getOutputBuffer(outputBufferId); ByteBuffer codecBuffer = codec.getOutputBuffer(outputBufferId);
if (sendFrameMeta) { boolean isConfig = (bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0;
writeFrameMeta(fd, bufferInfo, codecBuffer.remaining()); if (!isConfig) {
}
IO.writeFully(fd, codecBuffer);
if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) == 0) {
// If this is not a config packet, then it contains a frame // If this is not a config packet, then it contains a frame
firstFrameSent = true; firstFrameSent = true;
consecutiveErrors = 0; consecutiveErrors = 0;
} }
callbacks.onPacket(codecBuffer, bufferInfo);
} }
} finally { } finally {
if (outputBufferId >= 0) { if (outputBufferId >= 0) {
@@ -210,47 +196,28 @@ public class ScreenEncoder implements Device.RotationListener {
return !eof; return !eof;
} }
private void writeFrameMeta(FileDescriptor fd, MediaCodec.BufferInfo bufferInfo, int packetSize) throws IOException { private static MediaCodecInfo[] listEncoders(String videoMimeType) {
headerBuffer.clear();
long pts;
if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
pts = PACKET_FLAG_CONFIG; // non-media data packet
} else {
pts = bufferInfo.presentationTimeUs;
if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_KEY_FRAME) != 0) {
pts |= PACKET_FLAG_KEY_FRAME;
}
}
headerBuffer.putLong(pts);
headerBuffer.putInt(packetSize);
headerBuffer.flip();
IO.writeFully(fd, headerBuffer);
}
private static MediaCodecInfo[] listEncoders() {
List<MediaCodecInfo> result = new ArrayList<>(); List<MediaCodecInfo> result = new ArrayList<>();
MediaCodecList list = new MediaCodecList(MediaCodecList.REGULAR_CODECS); MediaCodecList list = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
for (MediaCodecInfo codecInfo : list.getCodecInfos()) { for (MediaCodecInfo codecInfo : list.getCodecInfos()) {
if (codecInfo.isEncoder() && Arrays.asList(codecInfo.getSupportedTypes()).contains(MediaFormat.MIMETYPE_VIDEO_AVC)) { if (codecInfo.isEncoder() && Arrays.asList(codecInfo.getSupportedTypes()).contains(videoMimeType)) {
result.add(codecInfo); result.add(codecInfo);
} }
} }
return result.toArray(new MediaCodecInfo[result.size()]); return result.toArray(new MediaCodecInfo[result.size()]);
} }
private static MediaCodec createCodec(String encoderName) throws IOException { private static MediaCodec createCodec(String videoMimeType, String encoderName) throws IOException {
if (encoderName != null) { if (encoderName != null) {
Ln.d("Creating encoder by name: '" + encoderName + "'"); Ln.d("Creating encoder by name: '" + encoderName + "'");
try { try {
return MediaCodec.createByCodecName(encoderName); return MediaCodec.createByCodecName(encoderName);
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
MediaCodecInfo[] encoders = listEncoders(); MediaCodecInfo[] encoders = listEncoders(videoMimeType);
throw new InvalidEncoderException(encoderName, encoders); throw new InvalidEncoderException(encoderName, encoders);
} }
} }
MediaCodec codec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_VIDEO_AVC); MediaCodec codec = MediaCodec.createEncoderByType(videoMimeType);
Ln.d("Using encoder: '" + codec.getName() + "'"); Ln.d("Using encoder: '" + codec.getName() + "'");
return codec; return codec;
} }
@@ -272,9 +239,9 @@ public class ScreenEncoder implements Device.RotationListener {
Ln.d("Codec option set: " + key + " (" + value.getClass().getSimpleName() + ") = " + value); Ln.d("Codec option set: " + key + " (" + value.getClass().getSimpleName() + ") = " + value);
} }
private static MediaFormat createFormat(int bitRate, int maxFps, List<CodecOption> codecOptions) { private static MediaFormat createFormat(String videoMimeType, int bitRate, int maxFps, List<CodecOption> codecOptions) {
MediaFormat format = new MediaFormat(); MediaFormat format = new MediaFormat();
format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_VIDEO_AVC); format.setString(MediaFormat.KEY_MIME, videoMimeType);
format.setInteger(MediaFormat.KEY_BIT_RATE, bitRate); format.setInteger(MediaFormat.KEY_BIT_RATE, bitRate);
// must be present to configure the encoder, but does not impact the actual frame rate, which is variable // must be present to configure the encoder, but does not impact the actual frame rate, which is variable
format.setInteger(MediaFormat.KEY_FRAME_RATE, 60); format.setInteger(MediaFormat.KEY_FRAME_RATE, 60);

View File

@@ -69,41 +69,71 @@ public final class Server {
int uid = options.getUid(); int uid = options.getUid();
boolean tunnelForward = options.isTunnelForward(); boolean tunnelForward = options.isTunnelForward();
boolean control = options.getControl(); boolean control = options.getControl();
boolean audio = Build.VERSION.SDK_INT >= Build.VERSION_CODES.R; // TODO option
boolean sendDummyByte = options.getSendDummyByte(); boolean sendDummyByte = options.getSendDummyByte();
Workarounds.prepareMainLooper();
// Workarounds must be applied for Meizu phones:
// - <https://github.com/Genymobile/scrcpy/issues/240>
// - <https://github.com/Genymobile/scrcpy/issues/365>
// - <https://github.com/Genymobile/scrcpy/issues/2656>
//
// But only apply when strictly necessary, since workarounds can cause other issues:
// - <https://github.com/Genymobile/scrcpy/issues/940>
// - <https://github.com/Genymobile/scrcpy/issues/994>
boolean mustFillAppInfo = Build.BRAND.equalsIgnoreCase("meizu");
// Before Android 11, audio is not supported.
// Since Android 12, we can properly set a context on the AudioRecord.
// Only on Android 11 we must fill app info for the AudioRecord to work.
mustFillAppInfo |= audio && Build.VERSION.SDK_INT == Build.VERSION_CODES.R;
if (mustFillAppInfo) {
Workarounds.fillAppInfo();
}
try (DesktopConnection connection = DesktopConnection.open(uid, tunnelForward, control, sendDummyByte)) { try (DesktopConnection connection = DesktopConnection.open(uid, tunnelForward, control, sendDummyByte)) {
VideoCodec codec = options.getCodec();
if (options.getSendDeviceMeta()) { if (options.getSendDeviceMeta()) {
Size videoSize = device.getScreenInfo().getVideoSize(); Size videoSize = device.getScreenInfo().getVideoSize();
connection.sendDeviceMeta(Device.getDeviceName(), videoSize.getWidth(), videoSize.getHeight()); connection.sendDeviceMeta(Device.getDeviceName(), videoSize.getWidth(), videoSize.getHeight());
} }
ScreenEncoder screenEncoder = new ScreenEncoder(options.getSendFrameMeta(), options.getBitRate(), options.getMaxFps(), codecOptions, ScreenEncoder screenEncoder = new ScreenEncoder(codec.getMimeType(), options.getBitRate(), options.getMaxFps(), codecOptions,
options.getEncoderName(), options.getDownsizeOnError()); options.getEncoderName(), options.getDownsizeOnError());
Thread controllerThread = null; Controller controller = null;
Thread deviceMessageSenderThread = null;
if (control) { if (control) {
final Controller controller = new Controller(device, connection, options.getClipboardAutosync(), options.getPowerOn()); controller = new Controller(device, connection, options.getClipboardAutosync(), options.getPowerOn());
controller.start();
// asynchronous final Controller controllerRef = controller;
controllerThread = startController(controller); device.setClipboardListener(text -> controllerRef.getSender().pushClipboardText(text));
deviceMessageSenderThread = startDeviceMessageSender(controller.getSender()); }
device.setClipboardListener(text -> controller.getSender().pushClipboardText(text)); AudioEncoder audioEncoder = null;
if (audio) {
audioEncoder = new AudioEncoder();
audioEncoder.start();
} }
try { try {
// synchronous // synchronous
screenEncoder.streamScreen(device, connection.getVideoFd()); VideoStreamer videoStreamer = new VideoStreamer(connection.getVideoFd(), options.getSendFrameMeta());
if (options.getSendCodecId()) {
videoStreamer.writeHeader(codec.getId());
}
screenEncoder.streamScreen(device, videoStreamer);
} catch (IOException e) { } catch (IOException e) {
// this is expected on close // this is expected on close
Ln.d("Screen streaming stopped"); Ln.d("Screen streaming stopped");
} finally { } finally {
initThread.interrupt(); initThread.interrupt();
if (controllerThread != null) { if (controller != null) {
controllerThread.interrupt(); controller.stop();
} }
if (deviceMessageSenderThread != null) { if (audioEncoder != null) {
deviceMessageSenderThread.interrupt(); audioEncoder.stop();
} }
} }
} }
@@ -115,32 +145,6 @@ public final class Server {
return thread; return thread;
} }
private static Thread startController(final Controller controller) {
Thread thread = new Thread(() -> {
try {
controller.control();
} catch (IOException e) {
// this is expected on close
Ln.d("Controller stopped");
}
});
thread.start();
return thread;
}
private static Thread startDeviceMessageSender(final DeviceMessageSender sender) {
Thread thread = new Thread(() -> {
try {
sender.loop();
} catch (IOException | InterruptedException e) {
// this is expected on close
Ln.d("Device message sender stopped");
}
});
thread.start();
return thread;
}
private static Options createOptions(String... args) { private static Options createOptions(String... args) {
if (args.length < 1) { if (args.length < 1) {
throw new IllegalArgumentException("Missing client version"); throw new IllegalArgumentException("Missing client version");
@@ -174,6 +178,13 @@ public final class Server {
Ln.Level level = Ln.Level.valueOf(value.toUpperCase(Locale.ENGLISH)); Ln.Level level = Ln.Level.valueOf(value.toUpperCase(Locale.ENGLISH));
options.setLogLevel(level); options.setLogLevel(level);
break; break;
case "codec":
VideoCodec codec = VideoCodec.findByName(value);
if (codec == null) {
throw new IllegalArgumentException("Video codec " + value + " not supported");
}
options.setCodec(codec);
break;
case "max_size": case "max_size":
int maxSize = Integer.parseInt(value) & ~7; // multiple of 8 int maxSize = Integer.parseInt(value) & ~7; // multiple of 8
options.setMaxSize(maxSize); options.setMaxSize(maxSize);
@@ -255,12 +266,17 @@ public final class Server {
boolean sendDummyByte = Boolean.parseBoolean(value); boolean sendDummyByte = Boolean.parseBoolean(value);
options.setSendDummyByte(sendDummyByte); options.setSendDummyByte(sendDummyByte);
break; break;
case "send_codec_id":
boolean sendCodecId = Boolean.parseBoolean(value);
options.setSendCodecId(sendCodecId);
break;
case "raw_video_stream": case "raw_video_stream":
boolean rawVideoStream = Boolean.parseBoolean(value); boolean rawVideoStream = Boolean.parseBoolean(value);
if (rawVideoStream) { if (rawVideoStream) {
options.setSendDeviceMeta(false); options.setSendDeviceMeta(false);
options.setSendFrameMeta(false); options.setSendFrameMeta(false);
options.setSendDummyByte(false); options.setSendDummyByte(false);
options.setSendCodecId(false);
} }
break; break;
default: default:

View File

@@ -0,0 +1,36 @@
package com.genymobile.scrcpy;
import android.media.MediaFormat;
public enum VideoCodec {
H264(0x68_32_36_34, "h264", MediaFormat.MIMETYPE_VIDEO_AVC),
H265(0x68_32_36_35, "h265", MediaFormat.MIMETYPE_VIDEO_HEVC),
AV1(0x00_61_76_31, "av1", MediaFormat.MIMETYPE_VIDEO_AV1);
private final int id; // 4-byte ASCII representation of the name
private final String name;
private final String mimeType;
VideoCodec(int id, String name, String mimeType) {
this.id = id;
this.name = name;
this.mimeType = mimeType;
}
public int getId() {
return id;
}
public String getMimeType() {
return mimeType;
}
public static VideoCodec findByName(String name) {
for (VideoCodec codec : values()) {
if (codec.name.equals(name)) {
return codec;
}
}
return null;
}
}

View File

@@ -0,0 +1,58 @@
package com.genymobile.scrcpy;
import android.media.MediaCodec;
import java.io.FileDescriptor;
import java.io.IOException;
import java.nio.ByteBuffer;
public final class VideoStreamer implements ScreenEncoder.Callbacks {
private static final long PACKET_FLAG_CONFIG = 1L << 63;
private static final long PACKET_FLAG_KEY_FRAME = 1L << 62;
private final FileDescriptor fd;
private final boolean sendFrameMeta;
private final ByteBuffer headerBuffer = ByteBuffer.allocate(12);
public VideoStreamer(FileDescriptor fd, boolean sendFrameMeta) {
this.fd = fd;
this.sendFrameMeta = sendFrameMeta;
}
public void writeHeader(int codecId) throws IOException {
ByteBuffer buffer = ByteBuffer.allocate(4);
buffer.putInt(codecId);
buffer.flip();
IO.writeFully(fd, buffer);
}
@Override
public void onPacket(ByteBuffer codecBuffer, MediaCodec.BufferInfo bufferInfo) throws IOException {
if (sendFrameMeta) {
writeFrameMeta(fd, bufferInfo, codecBuffer.remaining());
}
IO.writeFully(fd, codecBuffer);
}
private void writeFrameMeta(FileDescriptor fd, MediaCodec.BufferInfo bufferInfo, int packetSize) throws IOException {
headerBuffer.clear();
long ptsAndFlags;
if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
ptsAndFlags = PACKET_FLAG_CONFIG; // non-media data packet
} else {
ptsAndFlags = bufferInfo.presentationTimeUs;
if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_KEY_FRAME) != 0) {
ptsAndFlags |= PACKET_FLAG_KEY_FRAME;
}
}
headerBuffer.putLong(ptsAndFlags);
headerBuffer.putInt(packetSize);
headerBuffer.flip();
IO.writeFully(fd, headerBuffer);
}
}

View File

@@ -3,13 +3,12 @@ package com.genymobile.scrcpy;
import android.annotation.SuppressLint; import android.annotation.SuppressLint;
import android.app.Application; import android.app.Application;
import android.app.Instrumentation; import android.app.Instrumentation;
import android.content.Context; import android.content.ContextWrapper;
import android.content.pm.ApplicationInfo; import android.content.pm.ApplicationInfo;
import android.os.Looper; import android.os.Looper;
import java.lang.reflect.Constructor; import java.lang.reflect.Constructor;
import java.lang.reflect.Field; import java.lang.reflect.Field;
import java.lang.reflect.Method;
public final class Workarounds { public final class Workarounds {
private Workarounds() { private Workarounds() {
@@ -50,7 +49,7 @@ public final class Workarounds {
Object appBindData = appBindDataConstructor.newInstance(); Object appBindData = appBindDataConstructor.newInstance();
ApplicationInfo applicationInfo = new ApplicationInfo(); ApplicationInfo applicationInfo = new ApplicationInfo();
applicationInfo.packageName = "com.genymobile.scrcpy"; applicationInfo.packageName = FakeContext.PACKAGE_NAME;
// appBindData.appInfo = applicationInfo; // appBindData.appInfo = applicationInfo;
Field appInfoField = appBindDataClass.getDeclaredField("appInfo"); Field appInfoField = appBindDataClass.getDeclaredField("appInfo");
@@ -62,11 +61,10 @@ public final class Workarounds {
mBoundApplicationField.setAccessible(true); mBoundApplicationField.setAccessible(true);
mBoundApplicationField.set(activityThread, appBindData); mBoundApplicationField.set(activityThread, appBindData);
// Context ctx = activityThread.getSystemContext(); Application app = Application.class.newInstance();
Method getSystemContextMethod = activityThreadClass.getDeclaredMethod("getSystemContext"); Field baseField = ContextWrapper.class.getDeclaredField("mBase");
Context ctx = (Context) getSystemContextMethod.invoke(activityThread); baseField.setAccessible(true);
baseField.set(app, FakeContext.get());
Application app = Instrumentation.newApplication(Application.class, ctx);
// activityThread.mInitialApplication = app; // activityThread.mInitialApplication = app;
Field mInitialApplicationField = activityThreadClass.getDeclaredField("mInitialApplication"); Field mInitialApplicationField = activityThreadClass.getDeclaredField("mInitialApplication");

View File

@@ -5,6 +5,7 @@ import com.genymobile.scrcpy.Ln;
import android.os.Binder; import android.os.Binder;
import android.os.IBinder; import android.os.IBinder;
import android.os.IInterface; import android.os.IInterface;
import android.os.Process;
import java.lang.reflect.Field; import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException; import java.lang.reflect.InvocationTargetException;
@@ -48,10 +49,10 @@ public class ActivityManager {
Object[] args; Object[] args;
if (getContentProviderExternalMethodNewVersion) { if (getContentProviderExternalMethodNewVersion) {
// new version // new version
args = new Object[]{name, ServiceManager.USER_ID, token, null}; args = new Object[]{name, Process.ROOT_UID, token, null};
} else { } else {
// old version // old version
args = new Object[]{name, ServiceManager.USER_ID, token}; args = new Object[]{name, Process.ROOT_UID, token};
} }
// ContentProviderHolder providerHolder = getContentProviderExternal(...); // ContentProviderHolder providerHolder = getContentProviderExternal(...);
Object providerHolder = method.invoke(manager, args); Object providerHolder = method.invoke(manager, args);

View File

@@ -1,11 +1,13 @@
package com.genymobile.scrcpy.wrappers; package com.genymobile.scrcpy.wrappers;
import com.genymobile.scrcpy.FakeContext;
import com.genymobile.scrcpy.Ln; import com.genymobile.scrcpy.Ln;
import android.content.ClipData; import android.content.ClipData;
import android.content.IOnPrimaryClipChangedListener; import android.content.IOnPrimaryClipChangedListener;
import android.os.Build; import android.os.Build;
import android.os.IInterface; import android.os.IInterface;
import android.os.Process;
import java.lang.reflect.InvocationTargetException; import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method; import java.lang.reflect.Method;
@@ -58,22 +60,22 @@ public class ClipboardManager {
private static ClipData getPrimaryClip(Method method, boolean alternativeMethod, IInterface manager) private static ClipData getPrimaryClip(Method method, boolean alternativeMethod, IInterface manager)
throws InvocationTargetException, IllegalAccessException { throws InvocationTargetException, IllegalAccessException {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) { if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) {
return (ClipData) method.invoke(manager, ServiceManager.PACKAGE_NAME); return (ClipData) method.invoke(manager, FakeContext.PACKAGE_NAME);
} }
if (alternativeMethod) { if (alternativeMethod) {
return (ClipData) method.invoke(manager, ServiceManager.PACKAGE_NAME, null, ServiceManager.USER_ID); return (ClipData) method.invoke(manager, FakeContext.PACKAGE_NAME, null, Process.ROOT_UID);
} }
return (ClipData) method.invoke(manager, ServiceManager.PACKAGE_NAME, ServiceManager.USER_ID); return (ClipData) method.invoke(manager, FakeContext.PACKAGE_NAME, Process.ROOT_UID);
} }
private static void setPrimaryClip(Method method, boolean alternativeMethod, IInterface manager, ClipData clipData) private static void setPrimaryClip(Method method, boolean alternativeMethod, IInterface manager, ClipData clipData)
throws InvocationTargetException, IllegalAccessException { throws InvocationTargetException, IllegalAccessException {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) { if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) {
method.invoke(manager, clipData, ServiceManager.PACKAGE_NAME); method.invoke(manager, clipData, FakeContext.PACKAGE_NAME);
} else if (alternativeMethod) { } else if (alternativeMethod) {
method.invoke(manager, clipData, ServiceManager.PACKAGE_NAME, null, ServiceManager.USER_ID); method.invoke(manager, clipData, FakeContext.PACKAGE_NAME, null, Process.ROOT_UID);
} else { } else {
method.invoke(manager, clipData, ServiceManager.PACKAGE_NAME, ServiceManager.USER_ID); method.invoke(manager, clipData, FakeContext.PACKAGE_NAME, Process.ROOT_UID);
} }
} }
@@ -106,11 +108,11 @@ public class ClipboardManager {
private static void addPrimaryClipChangedListener(Method method, boolean alternativeMethod, IInterface manager, private static void addPrimaryClipChangedListener(Method method, boolean alternativeMethod, IInterface manager,
IOnPrimaryClipChangedListener listener) throws InvocationTargetException, IllegalAccessException { IOnPrimaryClipChangedListener listener) throws InvocationTargetException, IllegalAccessException {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) { if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q) {
method.invoke(manager, listener, ServiceManager.PACKAGE_NAME); method.invoke(manager, listener, FakeContext.PACKAGE_NAME);
} else if (alternativeMethod) { } else if (alternativeMethod) {
method.invoke(manager, listener, ServiceManager.PACKAGE_NAME, null, ServiceManager.USER_ID); method.invoke(manager, listener, FakeContext.PACKAGE_NAME, null, Process.ROOT_UID);
} else { } else {
method.invoke(manager, listener, ServiceManager.PACKAGE_NAME, ServiceManager.USER_ID); method.invoke(manager, listener, FakeContext.PACKAGE_NAME, Process.ROOT_UID);
} }
} }

View File

@@ -1,11 +1,15 @@
package com.genymobile.scrcpy.wrappers; package com.genymobile.scrcpy.wrappers;
import com.genymobile.scrcpy.FakeContext;
import com.genymobile.scrcpy.Ln; import com.genymobile.scrcpy.Ln;
import com.genymobile.scrcpy.SettingsException; import com.genymobile.scrcpy.SettingsException;
import android.annotation.SuppressLint; import android.annotation.SuppressLint;
import android.content.AttributionSource;
import android.os.Build;
import android.os.Bundle; import android.os.Bundle;
import android.os.IBinder; import android.os.IBinder;
import android.os.Process;
import java.io.Closeable; import java.io.Closeable;
import java.lang.reflect.InvocationTargetException; import java.lang.reflect.InvocationTargetException;
@@ -51,11 +55,10 @@ public class ContentProvider implements Closeable {
@SuppressLint("PrivateApi") @SuppressLint("PrivateApi")
private Method getCallMethod() throws NoSuchMethodException { private Method getCallMethod() throws NoSuchMethodException {
if (callMethod == null) { if (callMethod == null) {
try { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
Class<?> attributionSourceClass = Class.forName("android.content.AttributionSource"); callMethod = provider.getClass().getMethod("call", AttributionSource.class, String.class, String.class, String.class, Bundle.class);
callMethod = provider.getClass().getMethod("call", attributionSourceClass, String.class, String.class, String.class, Bundle.class);
callMethodVersion = 0; callMethodVersion = 0;
} catch (NoSuchMethodException | ClassNotFoundException e0) { } else {
// old versions // old versions
try { try {
callMethod = provider.getClass() callMethod = provider.getClass()
@@ -75,40 +78,29 @@ public class ContentProvider implements Closeable {
return callMethod; return callMethod;
} }
@SuppressLint("PrivateApi")
private Object getAttributionSource()
throws ClassNotFoundException, NoSuchMethodException, IllegalAccessException, InvocationTargetException, InstantiationException {
if (attributionSource == null) {
Class<?> cl = Class.forName("android.content.AttributionSource$Builder");
Object builder = cl.getConstructor(int.class).newInstance(ServiceManager.USER_ID);
cl.getDeclaredMethod("setPackageName", String.class).invoke(builder, ServiceManager.PACKAGE_NAME);
attributionSource = cl.getDeclaredMethod("build").invoke(builder);
}
return attributionSource;
}
private Bundle call(String callMethod, String arg, Bundle extras) private Bundle call(String callMethod, String arg, Bundle extras)
throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException { throws NoSuchMethodException, InvocationTargetException, IllegalAccessException {
try { try {
Method method = getCallMethod(); Method method = getCallMethod();
Object[] args; Object[] args;
switch (callMethodVersion) {
case 0: if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S && callMethodVersion == 0) {
args = new Object[]{getAttributionSource(), "settings", callMethod, arg, extras}; args = new Object[]{FakeContext.get().getAttributionSource(), "settings", callMethod, arg, extras};
break; } else {
case 1: switch (callMethodVersion) {
args = new Object[]{ServiceManager.PACKAGE_NAME, null, "settings", callMethod, arg, extras}; case 1:
break; args = new Object[]{FakeContext.PACKAGE_NAME, null, "settings", callMethod, arg, extras};
case 2: break;
args = new Object[]{ServiceManager.PACKAGE_NAME, "settings", callMethod, arg, extras}; case 2:
break; args = new Object[]{FakeContext.PACKAGE_NAME, "settings", callMethod, arg, extras};
default: break;
args = new Object[]{ServiceManager.PACKAGE_NAME, callMethod, arg, extras}; default:
break; args = new Object[]{FakeContext.PACKAGE_NAME, callMethod, arg, extras};
break;
}
} }
return (Bundle) method.invoke(provider, args); return (Bundle) method.invoke(provider, args);
} catch (InvocationTargetException | IllegalAccessException | NoSuchMethodException | ClassNotFoundException | InstantiationException e) { } catch (InvocationTargetException | IllegalAccessException | NoSuchMethodException e) {
Ln.e("Could not invoke method", e); Ln.e("Could not invoke method", e);
throw e; throw e;
} }
@@ -147,7 +139,7 @@ public class ContentProvider implements Closeable {
public String getValue(String table, String key) throws SettingsException { public String getValue(String table, String key) throws SettingsException {
String method = getGetMethod(table); String method = getGetMethod(table);
Bundle arg = new Bundle(); Bundle arg = new Bundle();
arg.putInt(CALL_METHOD_USER_KEY, ServiceManager.USER_ID); arg.putInt(CALL_METHOD_USER_KEY, Process.ROOT_UID);
try { try {
Bundle bundle = call(method, key, arg); Bundle bundle = call(method, key, arg);
if (bundle == null) { if (bundle == null) {
@@ -163,7 +155,7 @@ public class ContentProvider implements Closeable {
public void putValue(String table, String key, String value) throws SettingsException { public void putValue(String table, String key, String value) throws SettingsException {
String method = getPutMethod(table); String method = getPutMethod(table);
Bundle arg = new Bundle(); Bundle arg = new Bundle();
arg.putInt(CALL_METHOD_USER_KEY, ServiceManager.USER_ID); arg.putInt(CALL_METHOD_USER_KEY, Process.ROOT_UID);
arg.putString(NAME_VALUE_TABLE_VALUE, value); arg.putString(NAME_VALUE_TABLE_VALUE, value);
try { try {
call(method, key, arg); call(method, key, arg);

View File

@@ -10,9 +10,6 @@ import java.lang.reflect.Method;
@SuppressLint("PrivateApi,DiscouragedPrivateApi") @SuppressLint("PrivateApi,DiscouragedPrivateApi")
public final class ServiceManager { public final class ServiceManager {
public static final String PACKAGE_NAME = "com.android.shell";
public static final int USER_ID = 0;
private static final Method GET_SERVICE_METHOD; private static final Method GET_SERVICE_METHOD;
static { static {
try { try {