Compare commits

...

49 Commits

Author SHA1 Message Date
Romain Vimont
c1e9d837e0 Warn on ignored audio options
For raw audio codec, some audio options are ignored.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
fd5000a3c2 Add --audio-codec=raw option
Add support for raw (PCM S16 LE) audio codec (a raw decoder is included
in FFmpeg).

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
7292f0f013 Add raw audio recorder
Add an alternative AudioRecorder to stream raw packets without encoding.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
7c6b7cd952 Extract audio recorder interface
In order to support both encoded and raw audio stream, extract a
interface (very minimal, but sufficient to just start and stop).

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
df12314656 Extract audio capture
The audio capture was implemented in AudioEncoder.

In order to reuse it without encoding, extract it to a separate class.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
f71f7e7edd Stop on decoder frame push error
On push, frame sinks report downstream errors to stop upstream
components. Do not ignore the error.
2023-03-06 09:58:02 +01:00
Romain Vimont
2f84a4012b Add --audio-buffer
Expose an option to add a buffering delay (in milliseconds) before
playing audio.

This is similar to the options --display-buffer and --v4l2-buffer for
video frames.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
d93680976b Optionally do not delay the first frame
A delay buffer delayed all the frames except the first one, to open the
scrcpy window immediately and get a picture.

Make this feature optional, so that the delay buffer might also be used
for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
02c3c6148f Accept clock estimation with a single point
If there is only one point, assume the slope is 1.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
a417285c41 Use delay buffer as a frame source/sink
The components needing delayed frames (sc_screen and sc_v4l2_sink)
managed a sc_video_buffer instance, which itself embedded a
sc_frame_buffer instance (to keep only the most recent frame).

In theory, these components should not be aware of delaying: they should
just receive AVFrames later, and only handle a sc_frame_buffer.

Therefore, refactor sc_delay_buffer as a frame source (it consumes)
frames) and a frame sink (it produces frames, after some delay), and
plug an instance in the pipeline only when a delay is requested.

This also removes the need for a specific sc_video_buffer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
9b19de4262 Use frame source trait in decoder 2023-03-06 09:58:02 +01:00
Romain Vimont
3cbe9c0cc2 Introduce frame source trait
There was a frame sink trait, implemented by components able to receive
AVFrames, but each frame source had to manually send to frame sinks.

In order to mutualise sink management, add a frame sink trait.
2023-03-06 09:58:02 +01:00
Romain Vimont
a30e662107 Use packet source trait in demuxer 2023-03-06 09:58:02 +01:00
Romain Vimont
356c247368 Introduce packet source trait
There was a packet sink trait, implemented by components able to
receive AVPackets, but each packet source had to manually send to packet
sinks.

In order to mutualise sink management, add a packet source trait.
2023-03-06 09:58:02 +01:00
Romain Vimont
a70ca0996c Extract sc_delay_buffer
A video buffer had 2 responsibilities:
 - handle the frame delaying mechanism (queuing packets and pushing them
   after the expected delay);
 - keep only the most recent frame (using a sc_frame_buffer).

In order to reuse only the frame delaying mechanism, extract it to a
separate component, sc_delay_buffer.
2023-03-06 09:58:02 +01:00
Romain Vimont
e97cd27af2 Report video buffer downstream errors
Make the video buffer stop if its consumer could not receive a frame.
2023-03-06 09:58:02 +01:00
Romain Vimont
16eece8093 Stop the video buffer on error
If an error occurs from the video buffer thread (typically an
out-of-memory error), then stop.
2023-03-06 09:58:02 +01:00
Romain Vimont
a4f9cca08c Fix possible race condition on video_buffer end
The video_buffer thread clears the queue once it is stopped, but new
frames might still be pushed asynchronously.

To avoid the problem, do not push any frame once the video_buffer is
stopped.
2023-03-06 09:58:02 +01:00
Romain Vimont
c5cca0302d Remove sc_queue
All uses have been replaced by VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
c92a037029 Remove cbuf
All uses have been replaced by VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
a47debe016 Use VecDeque in aoa_hid
Replace cbuf by VecDeque in aoa_hid
2023-03-06 09:58:02 +01:00
Romain Vimont
ba44947066 Use VecDeque in file_pusher
Replace cbuf by VecDeque in file_pusher.

As a side-effect, the new implementation does not limit the queue to an
arbitrary value.
2023-03-06 09:58:02 +01:00
Romain Vimont
b65bd2b87c Use VecDeque in controller
Replace cbuf by VecDeque in controller.
2023-03-06 09:58:02 +01:00
Romain Vimont
9bb7d0f3d6 Use VecDeque in video_buffer
The packets queued for buffering were wrapped in a dynamically allocated
structure with a "next" field.

To avoid this additional layer of allocation and indirection, use a
VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
9389b6fc65 Use VecDeque in recorder
The packets queued for recording were wrapped in a dynamically allocated
structure with a "next" field.

To avoid this additional layer of allocation and indirection, use a
VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
2e041db876 Introduce VecDeque
Introduce a double-ended queue implemented with a growable ring buffer.

Inspired from the Rust VecDeque type:
<https://doc.rust-lang.org/std/collections/struct.VecDeque.html>
2023-03-06 09:58:02 +01:00
Romain Vimont
0b39c613a8 Add sc_allocarray() util
Add a function to allocate an array, which fails safely in the case
where the multiplication would overflow.
2023-03-06 09:58:02 +01:00
Romain Vimont
7e725ae55f Use reallocarray() in sc_vector
This fails safely in case of overflow.
2023-03-06 09:58:02 +01:00
Romain Vimont
cd99f1b643 Add compat for reallocarray()
This function fails safely in the case where the multiplication would
overflow.
2023-03-06 09:58:02 +01:00
Romain Vimont
e4cfad2cd2 Call avcodec_receive_frame() in a loop
Since in scrcpy a video packet passed to avcodec_send_packet() is always
a complete video frame, it is sufficient to call avcodec_receive_frame()
exactly once.

In practice, it also works for audio packets: the decoder produces
exactly 1 frame for 1 input packet.

In theory, it is an implementation detail though, so
avcodec_receive_frame() should be called in a loop.
2023-03-06 09:58:02 +01:00
Romain Vimont
aeea39e589 Add --require-audio
By default, scrcpy mirrors only the video when audio capture fails on
the device. Add a flag to force scrcpy to fail if audio is enabled but
does not work.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
af577f23fe Add compat support for FFmpeg < 5.1
The new chlayout API has been introduced in FFmpeg 5.1. Use the old
channel_layout API on older versions.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Simon Chan
0af893fd73 Add workaround to capture audio on Android 11
On Android 11, it is possible to start the capture only when the running
app is in foreground. But scrcpy is not an app, it's a Java application
started from shell.

As a workaround, start an existing Android shell existing activity just
to start the capture, then close it immediately.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-06 09:58:02 +01:00
Romain Vimont
047b54b9fe Add audio player
Play the decoded audio using SDL.

The audio player frame sink receives the audio frames, resample them
and write them to a byte buffer (introduced by this commit).

On SDL audio callback (from an internal SDL thread), copy samples from
this byte buffer to the SDL audio buffer.

The byte buffer is protected by the SDL_AudioDeviceLock(), but it has
been designed so that the producer and the consumer may write and read
in parallel, provided that they don't access the same slices of the
ring-buffer buffer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-03-06 09:58:02 +01:00
Romain Vimont
a01a257ddb Add two-step write feature to bytebuf
If there is exactly one producer, then it can assume that the remaining
space in the buffer will only increase until it write something.

This assumption may allow the producer to write to the buffer (up to a
known safe size) without any synchronization mechanism, thus allowing
to read and write different parts of the buffer in parallel.

The producer can then commit the write with lock held, and update its
knowledge of the safe empty remaining space.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
8bf9f200cc Introduce bytebuf util
Add a ring-buffer for bytes. It will be useful for buffering audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
134f48534a Pass AVCodecContext to frame sinks
Frame consumers may need details about the frame format.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
37e01e331f Add an audio decoder
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
59fc45b7cc Give a name to decoder instances
This will be useful in logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
84755698fd Rename decoder to video_decoder
This prepares the introduction of audio_decoder.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
cfdb85de1b Log display sizes in display list
This is more convenient than just the display id alone.
2023-03-06 09:58:02 +01:00
Romain Vimont
04b662b2f0 Add --list-displays 2023-03-06 09:58:02 +01:00
Romain Vimont
b00c393cd0 Move log message helpers to LogUtils
This class will also contain other log helpers.
2023-03-06 09:58:02 +01:00
Romain Vimont
94fef53781 Quit on audio configuration failure
When audio capture fails on the device, scrcpy continue mirroring the
video stream. This allows to enable audio by default only when
supported.

However, if an audio configuration occurs (for example the user
explicitly selected an unknown audio encoder), this must be treated as
an error and scrcpy must exit.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
9670645818 Add --list-encoders
Add an option to list the device encoders properly.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
42aa59f2c2 Move await_for_server() logs
Print the logs on the caller side. This will allow to call the function
in another context without printing the logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
ea35c23b6b Add --audio-encoder
Similar to --video-encoder, but for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:01 +01:00
Romain Vimont
aa9860a9f9 Extract unknown encoder error message
This will allow to reuse the same code for audio encoder selection.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:57:42 +01:00
Romain Vimont
400173c280 Add --audio-codec-options
Similar to --video-codec-options, but for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:57:36 +01:00
71 changed files with 3232 additions and 1191 deletions

View File

@@ -15,7 +15,7 @@ First, you need to install the required packages:
sudo apt install ffmpeg libsdl2-2.0-0 adb wget \
gcc git pkg-config meson ninja-build libsdl2-dev \
libavcodec-dev libavdevice-dev libavformat-dev libavutil-dev \
libusb-1.0-0 libusb-1.0-0-dev
libswresample-dev libusb-1.0-0 libusb-1.0-0-dev
```
Then clone the repo and execute the installation script
@@ -94,7 +94,7 @@ sudo apt install ffmpeg libsdl2-2.0-0 adb libusb-1.0-0
# client build dependencies
sudo apt install gcc git pkg-config meson ninja-build libsdl2-dev \
libavcodec-dev libavdevice-dev libavformat-dev libavutil-dev \
libusb-1.0-0-dev
libswresample-dev libusb-1.0-0-dev
# server build dependencies
sudo apt install openjdk-11-jdk

View File

@@ -273,12 +273,10 @@ may cause issues or crash. It is possible to select a different encoder:
scrcpy --video-encoder=OMX.qcom.video.encoder.avc
```
To list the available encoders, you can pass an invalid encoder name; the
error will give the available encoders:
To list the available encoders:
```bash
scrcpy --video-encoder=_ # for the default codec
scrcpy --video-codec=h265 --video-encoder=_ # for a specific codec
scrcpy --list-encoders
```
### Capture
@@ -720,7 +718,7 @@ scrcpy --display=1
The list of display ids can be retrieved by:
```bash
adb shell dumpsys display # search "mDisplayId=" in the output
scrcpy --list-displays
```
The secondary display may only be controlled if the device runs at least Android

View File

@@ -4,6 +4,8 @@ _scrcpy() {
--always-on-top
--audio-bit-rate=
--audio-codec=
--audio-codec-options=
--audio-encoder=
-b --video-bit-rate=
--crop=
-d --select-usb
@@ -17,6 +19,8 @@ _scrcpy() {
-K --hid-keyboard
-h --help
--legacy-paste
--list-displays
--list-encoders
--lock-video-orientation
--lock-video-orientation=
--max-fps=
@@ -41,6 +45,7 @@ _scrcpy() {
-r --record=
--record-format=
--render-driver=
--require-audio
--rotation=
-s --serial=
--shortcut-mod=
@@ -73,7 +78,7 @@ _scrcpy() {
return
;;
--audio-codec)
COMPREPLY=($(compgen -W 'opus aac' -- "$cur"))
COMPREPLY=($(compgen -W 'raw opus aac' -- "$cur"))
return
;;
--lock-video-orientation)

View File

@@ -10,7 +10,9 @@ local arguments
arguments=(
'--always-on-top[Make scrcpy window always on top \(above other windows\)]'
'--audio-bit-rate=[Encode the audio at the given bit-rate]'
'--audio-codec=[Select the audio codec]:codec:(opus aac)'
'--audio-codec=[Select the audio codec]:codec:(raw opus aac)'
'--audio-codec-options=[Set a list of comma-separated key\:type=value options for the device audio encoder]'
'--audio-encoder=[Use a specific MediaCodec audio encoder]'
{-b,--video-bit-rate=}'[Encode the video at the given bit-rate]'
'--crop=[\[width\:height\:x\:y\] Crop the device screen on the server]'
{-d,--select-usb}'[Use USB device]'
@@ -24,6 +26,8 @@ arguments=(
{-K,--hid-keyboard}'[Simulate a physical keyboard by using HID over AOAv2]'
{-h,--help}'[Print the help]'
'--legacy-paste[Inject computer clipboard text as a sequence of key events on Ctrl+v]'
'--list-displays[List displays available on the device]'
'--list-encoders[List video and audio encoders available on the device]'
'--lock-video-orientation=[Lock video orientation]:orientation:(unlocked initial 0 1 2 3)'
'--max-fps=[Limit the frame rate of screen capture]'
{-M,--hid-mouse}'[Simulate a physical mouse by using HID over AOAv2]'
@@ -47,6 +51,7 @@ arguments=(
{-r,--record=}'[Record screen to file]:record file:_files'
'--record-format=[Force recording format]:format:(mp4 mkv)'
'--render-driver=[Request SDL to use the given render driver]:driver name:(direct3d opengl opengles2 opengles metal software)'
'--require-audio=[Make scrcpy fail if audio is enabled but does not work]'
'--rotation=[Set the initial display rotation]:rotation values:(0 1 2 3)'
{-s,--serial=}'[The device serial number \(mandatory for multiple devices only\)]:serial:($("${ADB-adb}" devices | awk '\''$2 == "device" {print $1}'\''))'
'--shortcut-mod=[\[key1,key2+key3,...\] Specify the modifiers to use for scrcpy shortcuts]:shortcut mod:(lctrl rctrl lalt ralt lsuper rsuper)'

View File

@@ -4,12 +4,14 @@ src = [
'src/adb/adb_device.c',
'src/adb/adb_parser.c',
'src/adb/adb_tunnel.c',
'src/audio_player.c',
'src/cli.c',
'src/clock.c',
'src/compat.c',
'src/control_msg.c',
'src/controller.c',
'src/decoder.c',
'src/delay_buffer.c',
'src/demuxer.c',
'src/device_msg.c',
'src/icon.c',
@@ -28,12 +30,16 @@ src = [
'src/screen.c',
'src/server.c',
'src/version.c',
'src/video_buffer.c',
'src/trait/frame_source.c',
'src/trait/packet_source.c',
'src/util/acksync.c',
'src/util/average.c',
'src/util/bytebuf.c',
'src/util/file.c',
'src/util/intmap.c',
'src/util/intr.c',
'src/util/log.c',
'src/util/memory.c',
'src/util/net.c',
'src/util/net_intr.c',
'src/util/process.c',
@@ -99,6 +105,7 @@ if not crossbuild_windows
dependency('libavformat', version: '>= 57.33'),
dependency('libavcodec', version: '>= 57.37'),
dependency('libavutil'),
dependency('libswresample'),
dependency('sdl2', version: '>= 2.0.5'),
]
@@ -134,6 +141,7 @@ else
cc.find_library('avcodec-60', dirs: ffmpeg_bin_dir),
cc.find_library('avformat-60', dirs: ffmpeg_bin_dir),
cc.find_library('avutil-58', dirs: ffmpeg_bin_dir),
cc.find_library('swresample-4', dirs: ffmpeg_bin_dir),
],
include_directories: include_directories(ffmpeg_include_dir)
)
@@ -168,6 +176,7 @@ check_functions = [
'vasprintf',
'nrand48',
'jrand48',
'reallocarray',
]
foreach f : check_functions
@@ -254,8 +263,9 @@ if get_option('buildtype') == 'debug'
['test_binary', [
'tests/test_binary.c',
]],
['test_cbuf', [
'tests/test_cbuf.c',
['test_bytebuf', [
'tests/test_bytebuf.c',
'src/util/bytebuf.c',
]],
['test_cli', [
'tests/test_cli.c',
@@ -281,9 +291,6 @@ if get_option('buildtype') == 'debug'
'tests/test_device_msg_deserialize.c',
'src/device_msg.c',
]],
['test_queue', [
'tests/test_queue.c',
]],
['test_strbuf', [
'tests/test_strbuf.c',
'src/util/strbuf.c',
@@ -293,6 +300,10 @@ if get_option('buildtype') == 'debug'
'src/util/str.c',
'src/util/strbuf.c',
]],
['test_vecdeque', [
'tests/test_vecdeque.c',
'src/util/memory.c',
]],
['test_vector', [
'tests/test_vector.c',
]],

View File

@@ -25,12 +25,34 @@ Encode the audio at the given bit\-rate, expressed in bits/s. Unit suffixes are
Default is 196K (196000).
.TP
.BI "\-\-audio\-buffer ms
Add a buffering delay (in milliseconds) before playing audio. This increases latency to compensate for jitter.
Default is 0 (no buffering).
.TP
.BI "\-\-audio\-codec " name
Select an audio codec (opus or aac).
Select an audio codec (raw, opus or aac).
Default is opus.
.TP
.BI "\-\-audio\-codec\-options " key\fR[:\fItype\fR]=\fIvalue\fR[,...]
Set a list of comma-separated key:type=value options for the device audio encoder.
The possible values for 'type' are 'int' (default), 'long', 'float' and 'string'.
The list of possible codec options is available in the Android documentation
.UR https://d.android.com/reference/android/media/MediaFormat
.UE .
.TP
.BI "\-\-audio\-encoder " name
Use a specific MediaCodec audio encoder (depending on the codec provided by \fB\-\-audio\-codec\fR).
The available encoders can be listed by \-\-list\-encoders.
.TP
.BI "\-b, \-\-video\-bit\-rate " value
Encode the video at the given bit\-rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
@@ -57,10 +79,9 @@ Disable screensaver while scrcpy is running.
.TP
.BI "\-\-display " id
Specify the display id to mirror.
Specify the device display id to mirror.
The list of possible display ids can be listed by "adb shell dumpsys display"
(search "mDisplayId=" in the output).
The available display ids can be listed by \-\-list\-displays.
Default is 0.
@@ -114,6 +135,14 @@ Inject computer clipboard text as a sequence of key events on Ctrl+v (like MOD+S
This is a workaround for some devices not behaving as expected when setting the device clipboard programmatically.
.TP
.B \-\-list\-encoders
List video and audio encoders available on the device.
.TP
.B \-\-list\-displays
List displays available on the device.
.TP
\fB\-\-lock\-video\-orientation\fR[=\fIvalue\fR]
Lock video orientation to \fIvalue\fR. Possible values are "unlocked", "initial" (locked to the initial orientation), 0, 1, 2 and 3. Natural device orientation is 0, and each increment adds a 90 degrees rotation counterclockwise.
@@ -249,6 +278,10 @@ Supported names are currently "direct3d", "opengl", "opengles2", "opengles", "me
.UR https://wiki.libsdl.org/SDL_HINT_RENDER_DRIVER
.UE
.TP
.B \-\-require\-audio
By default, scrcpy mirrors only the video if audio capture fails on the device. This flag makes scrcpy fail if audio is enabled but does not work.
.TP
.BI "\-\-rotation " value
Set the initial display rotation. Possibles values are 0, 1, 2 and 3. Each increment adds a 90 degrees rotation counterclockwise.
@@ -341,6 +374,8 @@ The list of possible codec options is available in the Android documentation
.BI "\-\-video\-encoder " name
Use a specific MediaCodec video encoder (depending on the codec provided by \fB\-\-video\-codec\fR).
The available encoders can be listed by \-\-list\-encoders.
.TP
.B \-w, \-\-stay-awake
Keep the device on while scrcpy is running, when the device is plugged in.

378
app/src/audio_player.c Normal file
View File

@@ -0,0 +1,378 @@
#include "audio_player.h"
#include <libavutil/opt.h>
#include "util/log.h"
//#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
/** Downcast frame_sink to sc_audio_player */
#define DOWNCAST(SINK) container_of(SINK, struct sc_audio_player, frame_sink)
#define SC_AV_SAMPLE_FMT AV_SAMPLE_FMT_FLT
#define SC_SDL_SAMPLE_FMT AUDIO_F32
#define SC_AUDIO_OUTPUT_BUFFER_SAMPLES 480 // 10ms at 48000Hz
// The target number of buffered samples between the producer and the consumer.
// This value is directly use for compensation.
#define SC_TARGET_BUFFERED_SAMPLES (3 * SC_AUDIO_OUTPUT_BUFFER_SAMPLES)
// Use a ring-buffer of 1 second (at 48000Hz) between the producer and the
// consumer. It too big, but it guarantees that the producer and the consumer
// will be able to access it in parallel without locking.
#define SC_BYTEBUF_SIZE_IN_SAMPLES 48000
static inline size_t
bytes_to_samples(struct sc_audio_player *ap, size_t bytes) {
assert(bytes % (ap->nb_channels * ap->out_bytes_per_sample) == 0);
return bytes / (ap->nb_channels * ap->out_bytes_per_sample);
}
static inline size_t
samples_to_bytes(struct sc_audio_player *ap, size_t samples) {
return samples * ap->nb_channels * ap->out_bytes_per_sample;
}
void
sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
struct sc_audio_player *ap = userdata;
// This callback is called with the lock used by SDL_AudioDeviceLock(), so
// the bytebuf is protected
assert(len_int > 0);
size_t len = len_int;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] SDL callback requests %" SC_PRIsizet " samples",
bytes_to_samples(ap, len));
#endif
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
size_t read = MIN(read_avail, len);
if (read) {
sc_bytebuf_read(&ap->buf, stream, read);
}
if (read < len) {
// Insert silence
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Buffer underflow, inserting silence: %" SC_PRIsizet
" samples", bytes_to_samples(ap, len - read));
#endif
memset(stream + read, 0, len - read);
// If the first frame has not been received yet, it's not an underflow
if (ap->received) {
ap->underflow += bytes_to_samples(ap, len - read);
}
}
ap->last_consumed = sc_tick_now();
}
static uint8_t *
sc_audio_player_get_swr_buf(struct sc_audio_player *ap, size_t min_samples) {
size_t min_buf_size = samples_to_bytes(ap, min_samples);
if (min_buf_size > ap->swr_buf_alloc_size) {
size_t new_size = min_buf_size + 4096;
uint8_t *buf = realloc(ap->swr_buf, new_size);
if (!buf) {
LOG_OOM();
// Could not realloc to the requested size
return NULL;
}
ap->swr_buf = buf;
ap->swr_buf_alloc_size = new_size;
}
return ap->swr_buf;
}
static bool
sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
const AVCodecContext *ctx) {
struct sc_audio_player *ap = DOWNCAST(sink);
#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
assert(ctx->ch_layout.nb_channels > 0);
unsigned nb_channels = ctx->ch_layout.nb_channels;
#else
int tmp = av_get_channel_layout_nb_channels(ctx->channel_layout);
assert(tmp > 0);
unsigned nb_channels = tmp;
#endif
SDL_AudioSpec desired = {
.freq = ctx->sample_rate,
.format = SC_SDL_SAMPLE_FMT,
.channels = nb_channels,
.samples = SC_AUDIO_OUTPUT_BUFFER_SAMPLES,
.callback = sc_audio_player_sdl_callback,
.userdata = ap,
};
SDL_AudioSpec obtained;
ap->device = SDL_OpenAudioDevice(NULL, 0, &desired, &obtained, 0);
if (!ap->device) {
LOGE("Could not open audio device: %s", SDL_GetError());
return false;
}
SwrContext *swr_ctx = swr_alloc();
if (!swr_ctx) {
LOG_OOM();
goto error_close_audio_device;
}
ap->swr_ctx = swr_ctx;
assert(ctx->sample_rate > 0);
assert(!av_sample_fmt_is_planar(SC_AV_SAMPLE_FMT));
int out_bytes_per_sample = av_get_bytes_per_sample(SC_AV_SAMPLE_FMT);
assert(out_bytes_per_sample > 0);
#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
av_opt_set_chlayout(swr_ctx, "in_chlayout", &ctx->ch_layout, 0);
av_opt_set_chlayout(swr_ctx, "out_chlayout", &ctx->ch_layout, 0);
#else
av_opt_set_channel_layout(swr_ctx, "in_channel_layout",
ctx->channel_layout, 0);
av_opt_set_channel_layout(swr_ctx, "out_channel_layout",
ctx->channel_layout, 0);
#endif
av_opt_set_int(swr_ctx, "in_sample_rate", ctx->sample_rate, 0);
av_opt_set_int(swr_ctx, "out_sample_rate", ctx->sample_rate, 0);
av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", ctx->sample_fmt, 0);
av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt", SC_AV_SAMPLE_FMT, 0);
int ret = swr_init(swr_ctx);
if (ret) {
LOGE("Failed to initialize the resampling context");
goto error_free_swr_ctx;
}
ap->sample_rate = ctx->sample_rate;
ap->nb_channels = nb_channels;
ap->out_bytes_per_sample = out_bytes_per_sample;
size_t bytebuf_size = samples_to_bytes(ap, SC_BYTEBUF_SIZE_IN_SAMPLES);
bool ok = sc_bytebuf_init(&ap->buf, bytebuf_size);
if (!ok) {
goto error_free_swr_ctx;
}
size_t initial_swr_buf_size = samples_to_bytes(ap, 4096);
ap->swr_buf = malloc(initial_swr_buf_size);
if (!ap->swr_buf) {
LOG_OOM();
goto error_destroy_bytebuf;
}
ap->swr_buf_alloc_size = initial_swr_buf_size;
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
sc_average_init(&ap->avg_buffering, 8);
ap->samples_since_resync = 0;
ap->last_consumed = 0;
ap->underflow = 0;
ap->received = 0;
SDL_PauseAudioDevice(ap->device, 0);
return true;
error_destroy_bytebuf:
sc_bytebuf_destroy(&ap->buf);
error_free_swr_ctx:
swr_free(&ap->swr_ctx);
error_close_audio_device:
SDL_CloseAudioDevice(ap->device);
return false;
}
static void
sc_audio_player_frame_sink_close(struct sc_frame_sink *sink) {
struct sc_audio_player *ap = DOWNCAST(sink);
assert(ap->device);
SDL_PauseAudioDevice(ap->device, 1);
SDL_CloseAudioDevice(ap->device);
free(ap->swr_buf);
sc_bytebuf_destroy(&ap->buf);
swr_free(&ap->swr_ctx);
}
static bool
sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
const AVFrame *frame) {
struct sc_audio_player *ap = DOWNCAST(sink);
SwrContext *swr_ctx = ap->swr_ctx;
int64_t delay = swr_get_delay(swr_ctx, ap->sample_rate);
// No need to av_rescale_rnd(), input and output sample rates are the same
// Add more space (256) for clock compensation
int dst_nb_samples = delay + frame->nb_samples + 256;
uint8_t *swr_buf = sc_audio_player_get_swr_buf(ap, dst_nb_samples);
if (!swr_buf) {
return false;
}
int ret = swr_convert(swr_ctx, &swr_buf, dst_nb_samples,
(const uint8_t **) frame->data, frame->nb_samples);
if (ret < 0) {
LOGE("Resampling failed: %d", ret);
return false;
}
// swr_convert() returns the number of samples which would have been
// written if the buffer was big enough.
size_t samples_written = MIN(ret, dst_nb_samples);
size_t swr_buf_size = samples_to_bytes(ap, samples_written);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] %" SC_PRIsizet " samples written to buffer", samples_written);
#endif
// Since this function is the only writer, the current available space is
// at least the previous available space. In practice, it should almost
// always be possible to write without lock.
bool lockless_write = swr_buf_size <= ap->previous_write_avail;
if (lockless_write) {
sc_bytebuf_prepare_write(&ap->buf, swr_buf, swr_buf_size);
}
SDL_LockAudioDevice(ap->device);
// The consumer requests audio samples blocks (e.g. 480 samples).
// Convert the duration since the last consumption into samples.
size_t extrapolated = 0;
if (ap->last_consumed) {
sc_tick now = sc_tick_now();
assert(now >= ap->last_consumed);
extrapolated = (now - ap->last_consumed) * ap->sample_rate
/ SC_TICK_FREQ;
}
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
// The consumer may not increase underflow value if there are still samples
// available
assert(read_avail == 0 || ap->underflow == 0);
size_t buffered_samples = bytes_to_samples(ap, read_avail);
// Underflow caused silence samples in excess (so it adds buffering).
// Extrapolated samples must be considered consumed for smoothing (so it
// removes buffering).
float buffering = (float) buffered_samples + ap->underflow - extrapolated;
sc_average_push(&ap->avg_buffering, buffering);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] buffered_samples=%" SC_PRIsizet
" underflow=%" SC_PRIsizet
" extrapolated=%" SC_PRIsizet
" buffering=%f avg_buffering=%f",
buffered_samples, ap->underflow, extrapolated, buffering,
sc_average_get(&ap->avg_buffering));
#endif
if (lockless_write) {
sc_bytebuf_commit_write(&ap->buf, swr_buf_size);
} else {
// Take care to keep full samples
size_t align = ap->nb_channels * ap->out_bytes_per_sample;
size_t write_avail =
sc_bytebuf_write_available(&ap->buf) / align * align;
if (swr_buf_size > write_avail) {
// Skip old samples
size_t cap = sc_bytebuf_capacity(&ap->buf) / align * align;
if (swr_buf_size > cap) {
// Ignore the first bytes in swr_buf
swr_buf += swr_buf_size - cap;
swr_buf_size = cap;
}
assert(swr_buf_size > write_avail);
if (swr_buf_size - write_avail > 0) {
sc_bytebuf_skip(&ap->buf, swr_buf_size - write_avail);
}
}
sc_bytebuf_write(&ap->buf, swr_buf, swr_buf_size);
}
// On buffer underflow, typically because a packet is late, silence is
// inserted. In that case, the late samples must be ignored when they
// arrive, otherwise they will delay playback.
//
// As an improvement, instead of naively skipping the silence duration, we
// can absorb it if it helps clock compensation.
if (ap->underflow) {
size_t avg = sc_average_get(&ap->avg_buffering);
if (avg > SC_TARGET_BUFFERED_SAMPLES) {
size_t diff = SC_TARGET_BUFFERED_SAMPLES - avg;
if (ap->underflow > diff) {
// Partially absorb underflow for clock compensation (only keep
// the diff with the target buffering level).
ap->underflow -= diff;
} else {
// Totally absorb underflow for clock compensation
ap->underflow = 0;
}
size_t skip_samples = MIN(ap->underflow, buffered_samples);
if (skip_samples) {
size_t skip_bytes = samples_to_bytes(ap, skip_samples);
sc_bytebuf_skip(&ap->buf, skip_bytes);
read_avail -= skip_bytes;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Skipping %" SC_PRIsizet " samples", skip_samples);
#endif
}
} else {
// Totally absorb underflow for clock compensation
ap->underflow = 0;
}
}
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
ap->received = true;
SDL_UnlockAudioDevice(ap->device);
ap->samples_since_resync += samples_written;
if (ap->samples_since_resync >= ap->sample_rate) {
// Resync every second
ap->samples_since_resync = 0;
float avg = sc_average_get(&ap->avg_buffering);
int diff = SC_TARGET_BUFFERED_SAMPLES - avg;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Average buffering=%f, compensation %d", avg, diff);
#endif
// Compensate the diff over 3 seconds (but will be recomputed after
// 1 second)
int ret = swr_set_compensation(swr_ctx, diff, 3 * ap->sample_rate);
if (ret < 0) {
LOGW("Resampling compensation failed: %d", ret);
// not fatal
}
}
return true;
}
void
sc_audio_player_init(struct sc_audio_player *ap) {
static const struct sc_frame_sink_ops ops = {
.open = sc_audio_player_frame_sink_open,
.close = sc_audio_player_frame_sink_close,
.push = sc_audio_player_frame_sink_push,
};
ap->frame_sink.ops = &ops;
}

61
app/src/audio_player.h Normal file
View File

@@ -0,0 +1,61 @@
#ifndef SC_AUDIO_PLAYER_H
#define SC_AUDIO_PLAYER_H
#include "common.h"
#include <stdbool.h>
#include "trait/frame_sink.h"
#include <util/average.h>
#include <util/bytebuf.h>
#include <util/thread.h>
#include <libavformat/avformat.h>
#include <libswresample/swresample.h>
#include <SDL2/SDL.h>
struct sc_audio_player {
struct sc_frame_sink frame_sink;
SDL_AudioDeviceID device;
// protected by SDL_AudioDeviceLock()
struct sc_bytebuf buf;
size_t previous_write_avail;
struct SwrContext *swr_ctx;
// The sample rate is the same for input and output
unsigned sample_rate;
// The number of channels is the same for input and output
unsigned nb_channels;
// The number of bytes per sample for a single channel
unsigned out_bytes_per_sample;
// Target buffer for resampling
uint8_t *swr_buf;
size_t swr_buf_alloc_size;
// Number of buffered samples (may be negative on underflow)
struct sc_average avg_buffering;
// Count the number of samples to trigger a compensation update regularly
size_t samples_since_resync;
// The last date a sample has been consumed by the audio output
sc_tick last_consumed;
// Number of silence samples inserted to be compensated
size_t underflow;
bool received;
const struct sc_audio_player_callbacks *cbs;
void *cbs_userdata;
};
struct sc_audio_player_callbacks {
void (*on_ended)(struct sc_audio_player *ap, bool success, void *userdata);
};
void
sc_audio_player_init(struct sc_audio_player *ap);
#endif

View File

@@ -65,6 +65,12 @@ enum {
OPT_NO_AUDIO,
OPT_AUDIO_BIT_RATE,
OPT_AUDIO_CODEC,
OPT_AUDIO_CODEC_OPTIONS,
OPT_AUDIO_ENCODER,
OPT_LIST_ENCODERS,
OPT_LIST_DISPLAYS,
OPT_REQUIRE_AUDIO,
OPT_AUDIO_BUFFER,
};
struct sc_option {
@@ -114,13 +120,41 @@ static const struct sc_option options[] = {
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is 196K (196000).",
},
{
.longopt_id = OPT_AUDIO_BUFFER,
.longopt = "audio-buffer",
.argdesc = "ms",
.text = "Add a buffering delay (in milliseconds) before playing audio. "
"This increases latency to compensate for jitter.\n"
"Default is 0 (no buffering).",
},
{
.longopt_id = OPT_AUDIO_CODEC,
.longopt = "audio-codec",
.argdesc = "name",
.text = "Select an audio codec (opus or aac).\n"
.text = "Select an audio codec (raw, opus or aac).\n"
"Default is opus.",
},
{
.longopt_id = OPT_AUDIO_CODEC_OPTIONS,
.longopt = "audio-codec-options",
.argdesc = "key[:type]=value[,...]",
.text = "Set a list of comma-separated key:type=value options for the "
"device audio encoder.\n"
"The possible values for 'type' are 'int' (default), 'long', "
"'float' and 'string'.\n"
"The list of possible codec options is available in the "
"Android documentation: "
"<https://d.android.com/reference/android/media/MediaFormat>",
},
{
.longopt_id = OPT_AUDIO_ENCODER,
.longopt = "audio-encoder",
.argdesc = "name",
.text = "Use a specific MediaCodec audio encoder (depending on the "
"codec provided by --audio-codec).\n"
"The available encoders can be listed by --list-encoders.",
},
{
.shortopt = 'b',
.longopt = "video-bit-rate",
@@ -168,10 +202,9 @@ static const struct sc_option options[] = {
.longopt_id = OPT_DISPLAY_ID,
.longopt = "display",
.argdesc = "id",
.text = "Specify the display id to mirror.\n"
"The list of possible display ids can be listed by:\n"
" adb shell dumpsys display\n"
"(search \"mDisplayId=\" in the output)\n"
.text = "Specify the device display id to mirror.\n"
"The available display ids can be listed by:\n"
" scrcpy --list-displays\n"
"Default is 0.",
},
{
@@ -242,6 +275,16 @@ static const struct sc_option options[] = {
"This is a workaround for some devices not behaving as "
"expected when setting the device clipboard programmatically.",
},
{
.longopt_id = OPT_LIST_DISPLAYS,
.longopt = "list-displays",
.text = "List device displays.",
},
{
.longopt_id = OPT_LIST_ENCODERS,
.longopt = "list-encoders",
.text = "List video and audio encoders available on the device.",
},
{
.longopt_id = OPT_LOCK_VIDEO_ORIENTATION,
.longopt = "lock-video-orientation",
@@ -425,6 +468,13 @@ static const struct sc_option options[] = {
.longopt_id = OPT_RENDER_EXPIRED_FRAMES,
.longopt = "render-expired-frames",
},
{
.longopt_id = OPT_REQUIRE_AUDIO,
.longopt = "require-audio",
.text = "By default, scrcpy mirrors only the video when audio capture "
"fails on the device. This flag makes scrcpy fail if audio is "
"enabled but does not work."
},
{
.longopt_id = OPT_ROTATION,
.longopt = "rotation",
@@ -558,7 +608,8 @@ static const struct sc_option options[] = {
.longopt = "video-encoder",
.argdesc = "name",
.text = "Use a specific MediaCodec video encoder (depending on the "
"codec provided by --video-codec).",
"codec provided by --video-codec).\n"
"The available encoders can be listed by --list-encoders.",
},
{
.shortopt = 'w',
@@ -1455,6 +1506,10 @@ parse_video_codec(const char *optarg, enum sc_codec *codec) {
static bool
parse_audio_codec(const char *optarg, enum sc_codec *codec) {
if (!strcmp(optarg, "raw")) {
*codec = SC_CODEC_RAW;
return true;
}
if (!strcmp(optarg, "opus")) {
*codec = SC_CODEC_OPUS;
return true;
@@ -1463,7 +1518,7 @@ parse_audio_codec(const char *optarg, enum sc_codec *codec) {
*codec = SC_CODEC_AAC;
return true;
}
LOGE("Unsupported audio codec: %s (expected opus or aac)", optarg);
LOGE("Unsupported audio codec: %s (expected raw, opus or aac)", optarg);
return false;
}
@@ -1662,12 +1717,18 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
case OPT_VIDEO_CODEC_OPTIONS:
opts->video_codec_options = optarg;
break;
case OPT_AUDIO_CODEC_OPTIONS:
opts->audio_codec_options = optarg;
break;
case OPT_ENCODER:
LOGW("--encoder is deprecated, use --video-encoder instead.");
// fall through
case OPT_VIDEO_ENCODER:
opts->video_encoder = optarg;
break;
case OPT_AUDIO_ENCODER:
opts->audio_encoder = optarg;
break;
case OPT_FORCE_ADB_FORWARD:
opts->force_adb_forward = true;
break;
@@ -1755,6 +1816,20 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
LOGE("V4L2 (--v4l2-buffer) is only available on Linux.");
return false;
#endif
case OPT_LIST_ENCODERS:
opts->list_encoders = true;
break;
case OPT_LIST_DISPLAYS:
opts->list_displays = true;
break;
case OPT_REQUIRE_AUDIO:
opts->require_audio = true;
break;
case OPT_AUDIO_BUFFER:
if (!parse_buffering_time(optarg, &opts->audio_buffer)) {
return false;
}
break;
default:
// getopt prints the error message on stderr
return false;
@@ -1841,6 +1916,35 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
}
}
if (opts->record_filename && opts->audio_codec == SC_CODEC_RAW) {
LOGW("Recording does not support RAW audio codec");
return false;
}
if (opts->audio_codec == SC_CODEC_RAW) {
if (opts->audio_bit_rate) {
LOGW("--audio-bit-rate is ignored for raw audio codec");
}
if (opts->audio_codec_options) {
LOGW("--audio-codec-options is ignored for raw audio codec");
}
if (opts->audio_encoder) {
LOGW("--audio-encoder is ignored for raw audio codec");
}
}
if (opts->audio_codec == SC_CODEC_RAW) {
if (opts->audio_bit_rate) {
LOGW("--audio-bit-rate is ignored for raw audio codec");
}
if (opts->audio_codec_options) {
LOGW("--audio-codec-options is ignored for raw audio codec");
}
if (opts->audio_encoder) {
LOGW("--audio-encoder is ignored for raw audio codec");
}
}
if (!opts->control) {
if (opts->turn_screen_off) {
LOGE("Could not request to turn screen off if control is disabled");

View File

@@ -18,7 +18,15 @@ sc_clock_init(struct sc_clock *clock) {
static void
sc_clock_estimate(struct sc_clock *clock,
double *out_slope, sc_tick *out_offset) {
assert(clock->count > 1); // two points are necessary
assert(clock->count);
if (clock->count == 1) {
// If there is only 1 point, we can't compute a slope. Assume it is 1.
struct sc_clock_point *single_point = &clock->right_sum;
*out_slope = 1;
*out_offset = single_point->system - single_point->stream;
return;
}
struct sc_clock_point left_avg = {
.system = clock->left_sum.system / (clock->count / 2),
@@ -93,19 +101,17 @@ sc_clock_update(struct sc_clock *clock, sc_tick system, sc_tick stream) {
clock->head = (clock->head + 1) % SC_CLOCK_RANGE;
if (clock->count > 1) {
// Update estimation
sc_clock_estimate(clock, &clock->slope, &clock->offset);
// Update estimation
sc_clock_estimate(clock, &clock->slope, &clock->offset);
#ifndef SC_CLOCK_NDEBUG
LOGD("Clock estimation: %f * pts + %" PRItick,
clock->slope, clock->offset);
LOGD("Clock estimation: %f * pts + %" PRItick,
clock->slope, clock->offset);
#endif
}
}
sc_tick
sc_clock_to_system_time(struct sc_clock *clock, sc_tick stream) {
assert(clock->count > 1); // sc_clock_update() must have been called
assert(clock->count); // sc_clock_update() must have been called
return (sc_tick) (stream * clock->slope) + clock->offset;
}

View File

@@ -3,6 +3,9 @@
#include "config.h"
#include <assert.h>
#ifndef HAVE_REALLOCARRAY
# include <errno.h>
#endif
#include <stdlib.h>
#include <stdio.h>
#include <stdarg.h>
@@ -93,5 +96,15 @@ long jrand48(unsigned short xsubi[3]) {
return v.i;
}
#endif
#endif
#ifndef HAVE_REALLOCARRAY
void *reallocarray(void *ptr, size_t nmemb, size_t size) {
size_t bytes;
if (__builtin_mul_overflow(nmemb, size, &bytes)) {
errno = ENOMEM;
return NULL;
}
return realloc(ptr, bytes);
}
#endif

View File

@@ -37,6 +37,13 @@
# define SCRCPY_LAVF_HAS_AVFORMATCONTEXT_URL
#endif
// Not documented in ffmpeg/doc/APIchanges, but the channel_layout API
// has been replaced by chlayout in FFmpeg commit
// f423497b455da06c1337846902c770028760e094.
#if LIBAVUTIL_VERSION_INT >= AV_VERSION_INT(57, 23, 100)
# define SCRCPY_LAVU_HAS_CHLAYOUT
#endif
#if SDL_VERSION_ATLEAST(2, 0, 6)
// <https://github.com/libsdl-org/SDL/commit/d7a318de563125e5bb465b1000d6bc9576fbc6fc>
# define SCRCPY_SDL_HAS_HINT_TOUCH_MOUSE_EVENTS
@@ -67,4 +74,8 @@ long nrand48(unsigned short xsubi[3]);
long jrand48(unsigned short xsubi[3]);
#endif
#ifndef HAVE_REALLOCARRAY
void *reallocarray(void *ptr, size_t nmemb, size_t size);
#endif
#endif

View File

@@ -4,19 +4,28 @@
#include "util/log.h"
#define SC_CONTROL_MSG_QUEUE_MAX 64
bool
sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
struct sc_acksync *acksync) {
cbuf_init(&controller->queue);
sc_vecdeque_init(&controller->queue);
bool ok = sc_receiver_init(&controller->receiver, control_socket, acksync);
bool ok = sc_vecdeque_reserve(&controller->queue, SC_CONTROL_MSG_QUEUE_MAX);
if (!ok) {
return false;
}
ok = sc_receiver_init(&controller->receiver, control_socket, acksync);
if (!ok) {
sc_vecdeque_destroy(&controller->queue);
return false;
}
ok = sc_mutex_init(&controller->mutex);
if (!ok) {
sc_receiver_destroy(&controller->receiver);
sc_vecdeque_destroy(&controller->queue);
return false;
}
@@ -24,6 +33,7 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
if (!ok) {
sc_receiver_destroy(&controller->receiver);
sc_mutex_destroy(&controller->mutex);
sc_vecdeque_destroy(&controller->queue);
return false;
}
@@ -38,10 +48,12 @@ sc_controller_destroy(struct sc_controller *controller) {
sc_cond_destroy(&controller->msg_cond);
sc_mutex_destroy(&controller->mutex);
struct sc_control_msg msg;
while (cbuf_take(&controller->queue, &msg)) {
sc_control_msg_destroy(&msg);
while (!sc_vecdeque_is_empty(&controller->queue)) {
struct sc_control_msg *msg = sc_vecdeque_popref(&controller->queue);
assert(msg);
sc_control_msg_destroy(msg);
}
sc_vecdeque_destroy(&controller->queue);
sc_receiver_destroy(&controller->receiver);
}
@@ -54,13 +66,19 @@ sc_controller_push_msg(struct sc_controller *controller,
}
sc_mutex_lock(&controller->mutex);
bool was_empty = cbuf_is_empty(&controller->queue);
bool res = cbuf_push(&controller->queue, *msg);
if (was_empty) {
sc_cond_signal(&controller->msg_cond);
bool full = sc_vecdeque_is_full(&controller->queue);
if (!full) {
bool was_empty = sc_vecdeque_is_empty(&controller->queue);
sc_vecdeque_push_noresize(&controller->queue, *msg);
if (was_empty) {
sc_cond_signal(&controller->msg_cond);
}
}
// Otherwise (if the queue is full), the msg is discarded
sc_mutex_unlock(&controller->mutex);
return res;
return !full;
}
static bool
@@ -82,7 +100,8 @@ run_controller(void *data) {
for (;;) {
sc_mutex_lock(&controller->mutex);
while (!controller->stopped && cbuf_is_empty(&controller->queue)) {
while (!controller->stopped
&& sc_vecdeque_is_empty(&controller->queue)) {
sc_cond_wait(&controller->msg_cond, &controller->mutex);
}
if (controller->stopped) {
@@ -90,10 +109,9 @@ run_controller(void *data) {
sc_mutex_unlock(&controller->mutex);
break;
}
struct sc_control_msg msg;
bool non_empty = cbuf_take(&controller->queue, &msg);
assert(non_empty);
(void) non_empty;
assert(!sc_vecdeque_is_empty(&controller->queue));
struct sc_control_msg msg = sc_vecdeque_pop(&controller->queue);
sc_mutex_unlock(&controller->mutex);
bool ok = process_msg(controller, &msg);

View File

@@ -8,11 +8,11 @@
#include "control_msg.h"
#include "receiver.h"
#include "util/acksync.h"
#include "util/cbuf.h"
#include "util/net.h"
#include "util/thread.h"
#include "util/vecdeque.h"
struct sc_control_msg_queue CBUF(struct sc_control_msg, 64);
struct sc_control_msg_queue SC_VECDEQUE(struct sc_control_msg);
struct sc_controller {
sc_socket control_socket;

View File

@@ -2,41 +2,15 @@
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavutil/channel_layout.h>
#include "events.h"
#include "video_buffer.h"
#include "trait/frame_sink.h"
#include "util/log.h"
/** Downcast packet_sink to decoder */
#define DOWNCAST(SINK) container_of(SINK, struct sc_decoder, packet_sink)
static void
sc_decoder_close_first_sinks(struct sc_decoder *decoder, unsigned count) {
while (count) {
struct sc_frame_sink *sink = decoder->sinks[--count];
sink->ops->close(sink);
}
}
static inline void
sc_decoder_close_sinks(struct sc_decoder *decoder) {
sc_decoder_close_first_sinks(decoder, decoder->sink_count);
}
static bool
sc_decoder_open_sinks(struct sc_decoder *decoder) {
for (unsigned i = 0; i < decoder->sink_count; ++i) {
struct sc_frame_sink *sink = decoder->sinks[i];
if (!sink->ops->open(sink)) {
sc_decoder_close_first_sinks(decoder, i);
return false;
}
}
return true;
}
static bool
sc_decoder_open(struct sc_decoder *decoder, const AVCodec *codec) {
decoder->codec_ctx = avcodec_alloc_context3(codec);
@@ -47,8 +21,23 @@ sc_decoder_open(struct sc_decoder *decoder, const AVCodec *codec) {
decoder->codec_ctx->flags |= AV_CODEC_FLAG_LOW_DELAY;
if (codec->type == AVMEDIA_TYPE_VIDEO) {
// Hardcoded video properties
decoder->codec_ctx->pix_fmt = AV_PIX_FMT_YUV420P;
} else {
// Hardcoded audio properties
#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
decoder->codec_ctx->ch_layout =
(AVChannelLayout) AV_CHANNEL_LAYOUT_STEREO;
#else
decoder->codec_ctx->channel_layout = AV_CH_LAYOUT_STEREO;
decoder->codec_ctx->channels = 2;
#endif
decoder->codec_ctx->sample_rate = 48000;
}
if (avcodec_open2(decoder->codec_ctx, codec, NULL) < 0) {
LOGE("Could not open codec");
LOGE("Decoder '%s': could not open codec", decoder->name);
avcodec_free_context(&decoder->codec_ctx);
return false;
}
@@ -61,7 +50,8 @@ sc_decoder_open(struct sc_decoder *decoder, const AVCodec *codec) {
return false;
}
if (!sc_decoder_open_sinks(decoder)) {
if (!sc_frame_source_sinks_open(&decoder->frame_source,
decoder->codec_ctx)) {
av_frame_free(&decoder->frame);
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
@@ -73,24 +63,12 @@ sc_decoder_open(struct sc_decoder *decoder, const AVCodec *codec) {
static void
sc_decoder_close(struct sc_decoder *decoder) {
sc_decoder_close_sinks(decoder);
sc_frame_source_sinks_close(&decoder->frame_source);
av_frame_free(&decoder->frame);
avcodec_close(decoder->codec_ctx);
avcodec_free_context(&decoder->codec_ctx);
}
static bool
push_frame_to_sinks(struct sc_decoder *decoder, const AVFrame *frame) {
for (unsigned i = 0; i < decoder->sink_count; ++i) {
struct sc_frame_sink *sink = decoder->sinks[i];
if (!sink->ops->push(sink, frame)) {
return false;
}
}
return true;
}
static bool
sc_decoder_push(struct sc_decoder *decoder, const AVPacket *packet) {
bool is_config = packet->pts == AV_NOPTS_VALUE;
@@ -101,22 +79,33 @@ sc_decoder_push(struct sc_decoder *decoder, const AVPacket *packet) {
int ret = avcodec_send_packet(decoder->codec_ctx, packet);
if (ret < 0 && ret != AVERROR(EAGAIN)) {
LOGE("Could not send video packet: %d", ret);
LOGE("Decoder '%s': could not send video packet: %d",
decoder->name, ret);
return false;
}
ret = avcodec_receive_frame(decoder->codec_ctx, decoder->frame);
if (!ret) {
// a frame was received
bool ok = push_frame_to_sinks(decoder, decoder->frame);
// A frame lost should not make the whole pipeline fail. The error, if
// any, is already logged.
(void) ok;
for (;;) {
ret = avcodec_receive_frame(decoder->codec_ctx, decoder->frame);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
break;
}
if (ret) {
LOGE("Decoder '%s', could not receive video frame: %d",
decoder->name, ret);
return false;
}
// a frame was received
bool ok = sc_frame_source_sinks_push(&decoder->frame_source,
decoder->frame);
av_frame_unref(decoder->frame);
} else if (ret != AVERROR(EAGAIN)) {
LOGE("Could not receive video frame: %d", ret);
return false;
if (!ok) {
// Error already logged
return false;
}
}
return true;
}
@@ -140,8 +129,9 @@ sc_decoder_packet_sink_push(struct sc_packet_sink *sink,
}
void
sc_decoder_init(struct sc_decoder *decoder) {
decoder->sink_count = 0;
sc_decoder_init(struct sc_decoder *decoder, const char *name) {
decoder->name = name; // statically allocated
sc_frame_source_init(&decoder->frame_source);
static const struct sc_packet_sink_ops ops = {
.open = sc_decoder_packet_sink_open,
@@ -151,11 +141,3 @@ sc_decoder_init(struct sc_decoder *decoder) {
decoder->packet_sink.ops = &ops;
}
void
sc_decoder_add_sink(struct sc_decoder *decoder, struct sc_frame_sink *sink) {
assert(decoder->sink_count < SC_DECODER_MAX_SINKS);
assert(sink);
assert(sink->ops);
decoder->sinks[decoder->sink_count++] = sink;
}

View File

@@ -3,28 +3,25 @@
#include "common.h"
#include "trait/frame_source.h"
#include "trait/packet_sink.h"
#include <stdbool.h>
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#define SC_DECODER_MAX_SINKS 2
struct sc_decoder {
struct sc_packet_sink packet_sink; // packet sink trait
struct sc_frame_source frame_source; // frame source trait
struct sc_frame_sink *sinks[SC_DECODER_MAX_SINKS];
unsigned sink_count;
const char *name; // must be statically allocated (e.g. a string literal)
AVCodecContext *codec_ctx;
AVFrame *frame;
};
// The name must be statically allocated (e.g. a string literal)
void
sc_decoder_init(struct sc_decoder *decoder);
void
sc_decoder_add_sink(struct sc_decoder *decoder, struct sc_frame_sink *sink);
sc_decoder_init(struct sc_decoder *decoder, const char *name);
#endif

244
app/src/delay_buffer.c Normal file
View File

@@ -0,0 +1,244 @@
#include "delay_buffer.h"
#include <assert.h>
#include <stdlib.h>
#include <libavutil/avutil.h>
#include <libavformat/avformat.h>
#include "util/log.h"
#define SC_BUFFERING_NDEBUG // comment to debug
/** Downcast frame_sink to sc_delay_buffer */
#define DOWNCAST(SINK) container_of(SINK, struct sc_delay_buffer, frame_sink)
static bool
sc_delayed_frame_init(struct sc_delayed_frame *dframe, const AVFrame *frame) {
dframe->frame = av_frame_alloc();
if (!dframe->frame) {
LOG_OOM();
return false;
}
if (av_frame_ref(dframe->frame, frame)) {
LOG_OOM();
av_frame_free(&dframe->frame);
return false;
}
return true;
}
static void
sc_delayed_frame_destroy(struct sc_delayed_frame *dframe) {
av_frame_unref(dframe->frame);
av_frame_free(&dframe->frame);
}
static int
run_buffering(void *data) {
struct sc_delay_buffer *db = data;
assert(db->delay > 0);
for (;;) {
sc_mutex_lock(&db->mutex);
while (!db->stopped && sc_vecdeque_is_empty(&db->queue)) {
sc_cond_wait(&db->queue_cond, &db->mutex);
}
if (db->stopped) {
sc_mutex_unlock(&db->mutex);
goto stopped;
}
struct sc_delayed_frame dframe = sc_vecdeque_pop(&db->queue);
sc_tick max_deadline = sc_tick_now() + db->delay;
// PTS (written by the server) are expressed in microseconds
sc_tick pts = SC_TICK_TO_US(dframe.frame->pts);
bool timed_out = false;
while (!db->stopped && !timed_out) {
sc_tick deadline = sc_clock_to_system_time(&db->clock, pts)
+ db->delay;
if (deadline > max_deadline) {
deadline = max_deadline;
}
timed_out =
!sc_cond_timedwait(&db->wait_cond, &db->mutex, deadline);
}
bool stopped = db->stopped;
sc_mutex_unlock(&db->mutex);
if (stopped) {
sc_delayed_frame_destroy(&dframe);
goto stopped;
}
#ifndef SC_BUFFERING_NDEBUG
LOGD("Buffering: %" PRItick ";%" PRItick ";%" PRItick,
pts, dframe.push_date, sc_tick_now());
#endif
bool ok = sc_frame_source_sinks_push(&db->frame_source, dframe.frame);
sc_delayed_frame_destroy(&dframe);
if (!ok) {
LOGE("Delayed frame could not be pushed, stopping");
sc_mutex_lock(&db->mutex);
// Prevent to push any new frame
db->stopped = true;
sc_mutex_unlock(&db->mutex);
goto stopped;
}
}
stopped:
assert(db->stopped);
// Flush queue
while (!sc_vecdeque_is_empty(&db->queue)) {
struct sc_delayed_frame *dframe = sc_vecdeque_popref(&db->queue);
sc_delayed_frame_destroy(dframe);
}
LOGD("Buffering thread ended");
return 0;
}
static bool
sc_delay_buffer_frame_sink_open(struct sc_frame_sink *sink,
const AVCodecContext *ctx) {
struct sc_delay_buffer *db = DOWNCAST(sink);
(void) ctx;
bool ok = sc_mutex_init(&db->mutex);
if (!ok) {
return false;
}
ok = sc_cond_init(&db->queue_cond);
if (!ok) {
goto error_destroy_mutex;
}
ok = sc_cond_init(&db->wait_cond);
if (!ok) {
goto error_destroy_queue_cond;
}
sc_clock_init(&db->clock);
sc_vecdeque_init(&db->queue);
if (!sc_frame_source_sinks_open(&db->frame_source, ctx)) {
goto error_destroy_wait_cond;
}
ok = sc_thread_create(&db->thread, run_buffering, "scrcpy-dbuf", db);
if (!ok) {
LOGE("Could not start buffering thread");
goto error_close_sinks;
}
return true;
error_close_sinks:
sc_frame_source_sinks_close(&db->frame_source);
error_destroy_wait_cond:
sc_cond_destroy(&db->wait_cond);
error_destroy_queue_cond:
sc_cond_destroy(&db->queue_cond);
error_destroy_mutex:
sc_mutex_destroy(&db->mutex);
return false;
}
static void
sc_delay_buffer_frame_sink_close(struct sc_frame_sink *sink) {
struct sc_delay_buffer *db = DOWNCAST(sink);
sc_mutex_lock(&db->mutex);
db->stopped = true;
sc_cond_signal(&db->queue_cond);
sc_cond_signal(&db->wait_cond);
sc_mutex_unlock(&db->mutex);
sc_thread_join(&db->thread, NULL);
sc_frame_source_sinks_close(&db->frame_source);
sc_cond_destroy(&db->wait_cond);
sc_cond_destroy(&db->queue_cond);
sc_mutex_destroy(&db->mutex);
}
static bool
sc_delay_buffer_frame_sink_push(struct sc_frame_sink *sink,
const AVFrame *frame) {
struct sc_delay_buffer *db = DOWNCAST(sink);
sc_mutex_lock(&db->mutex);
if (db->stopped) {
sc_mutex_unlock(&db->mutex);
return false;
}
sc_tick pts = SC_TICK_FROM_US(frame->pts);
sc_clock_update(&db->clock, sc_tick_now(), pts);
sc_cond_signal(&db->wait_cond);
if (db->first_frame_asap && db->clock.count == 1) {
sc_mutex_unlock(&db->mutex);
return sc_frame_source_sinks_push(&db->frame_source, frame);
}
struct sc_delayed_frame dframe;
bool ok = sc_delayed_frame_init(&dframe, frame);
if (!ok) {
sc_mutex_unlock(&db->mutex);
return false;
}
#ifndef SC_BUFFERING_NDEBUG
dframe.push_date = sc_tick_now();
#endif
ok = sc_vecdeque_push(&db->queue, dframe);
if (!ok) {
sc_mutex_unlock(&db->mutex);
LOG_OOM();
return false;
}
sc_cond_signal(&db->queue_cond);
sc_mutex_unlock(&db->mutex);
return true;
}
void
sc_delay_buffer_init(struct sc_delay_buffer *db, sc_tick delay,
bool first_frame_asap) {
assert(delay > 0);
db->delay = delay;
db->first_frame_asap = first_frame_asap;
sc_frame_source_init(&db->frame_source);
static const struct sc_frame_sink_ops ops = {
.open = sc_delay_buffer_frame_sink_open,
.close = sc_delay_buffer_frame_sink_close,
.push = sc_delay_buffer_frame_sink_push,
};
db->frame_sink.ops = &ops;
}

60
app/src/delay_buffer.h Normal file
View File

@@ -0,0 +1,60 @@
#ifndef SC_DELAY_BUFFER_H
#define SC_DELAY_BUFFER_H
#include "common.h"
#include <stdbool.h>
#include "clock.h"
#include "trait/frame_source.h"
#include "trait/frame_sink.h"
#include "util/thread.h"
#include "util/tick.h"
#include "util/vecdeque.h"
// forward declarations
typedef struct AVFrame AVFrame;
struct sc_delayed_frame {
AVFrame *frame;
#ifndef NDEBUG
sc_tick push_date;
#endif
};
struct sc_delayed_frame_queue SC_VECDEQUE(struct sc_delayed_frame);
struct sc_delay_buffer {
struct sc_frame_source frame_source; // frame source trait
struct sc_frame_sink frame_sink; // frame sink trait
sc_tick delay;
bool first_frame_asap;
sc_thread thread;
sc_mutex mutex;
sc_cond queue_cond;
sc_cond wait_cond;
struct sc_clock clock;
struct sc_delayed_frame_queue queue;
bool stopped;
};
struct sc_delay_buffer_callbacks {
bool (*on_new_frame)(struct sc_delay_buffer *db, const AVFrame *frame,
void *userdata);
};
/**
* Initialize a delay buffer.
*
* \param delay a (strictly) positive delay
* \param first_frame_asap if true, do not delay the first frame (useful for
a video stream).
*/
void
sc_delay_buffer_init(struct sc_delay_buffer *db, sc_tick delay,
bool first_frame_asap);
#endif

View File

@@ -23,6 +23,7 @@ sc_demuxer_to_avcodec_id(uint32_t codec_id) {
#define SC_CODEC_ID_H264 UINT32_C(0x68323634) // "h264" in ASCII
#define SC_CODEC_ID_H265 UINT32_C(0x68323635) // "h265" in ASCII
#define SC_CODEC_ID_AV1 UINT32_C(0x00617631) // "av1" in ASCII
#define SC_CODEC_ID_RAW UINT32_C(0x00726177) // "raw" in ASCII
#define SC_CODEC_ID_OPUS UINT32_C(0x6f707573) // "opus" in ASCII
#define SC_CODEC_ID_AAC UINT32_C(0x00616163) // "aac in ASCII"
switch (codec_id) {
@@ -32,6 +33,8 @@ sc_demuxer_to_avcodec_id(uint32_t codec_id) {
return AV_CODEC_ID_HEVC;
case SC_CODEC_ID_AV1:
return AV_CODEC_ID_AV1;
case SC_CODEC_ID_RAW:
return AV_CODEC_ID_PCM_S16LE;
case SC_CODEC_ID_OPUS:
return AV_CODEC_ID_OPUS;
case SC_CODEC_ID_AAC:
@@ -112,86 +115,32 @@ sc_demuxer_recv_packet(struct sc_demuxer *demuxer, AVPacket *packet) {
return true;
}
static bool
push_packet_to_sinks(struct sc_demuxer *demuxer, const AVPacket *packet) {
for (unsigned i = 0; i < demuxer->sink_count; ++i) {
struct sc_packet_sink *sink = demuxer->sinks[i];
if (!sink->ops->push(sink, packet)) {
return false;
}
}
return true;
}
static bool
sc_demuxer_push_packet(struct sc_demuxer *demuxer, AVPacket *packet) {
bool ok = push_packet_to_sinks(demuxer, packet);
if (!ok) {
LOGE("Demuxer '%s': could not process packet", demuxer->name);
return false;
}
return true;
}
static void
sc_demuxer_close_first_sinks(struct sc_demuxer *demuxer, unsigned count) {
while (count) {
struct sc_packet_sink *sink = demuxer->sinks[--count];
sink->ops->close(sink);
}
}
static inline void
sc_demuxer_close_sinks(struct sc_demuxer *demuxer) {
sc_demuxer_close_first_sinks(demuxer, demuxer->sink_count);
}
static bool
sc_demuxer_open_sinks(struct sc_demuxer *demuxer, const AVCodec *codec) {
for (unsigned i = 0; i < demuxer->sink_count; ++i) {
struct sc_packet_sink *sink = demuxer->sinks[i];
if (!sink->ops->open(sink, codec)) {
sc_demuxer_close_first_sinks(demuxer, i);
return false;
}
}
return true;
}
static void
sc_demuxer_disable_sinks(struct sc_demuxer *demuxer) {
for (unsigned i = 0; i < demuxer->sink_count; ++i) {
struct sc_packet_sink *sink = demuxer->sinks[i];
if (sink->ops->disable) {
sink->ops->disable(sink);
}
}
}
static int
run_demuxer(void *data) {
struct sc_demuxer *demuxer = data;
// Flag to report end-of-stream (i.e. device disconnected)
bool eos = false;
enum sc_demuxer_status status = SC_DEMUXER_STATUS_ERROR;
uint32_t raw_codec_id;
bool ok = sc_demuxer_recv_codec_id(demuxer, &raw_codec_id);
if (!ok) {
LOGE("Demuxer '%s': stream disabled due to connection error",
demuxer->name);
eos = true;
goto end;
}
if (raw_codec_id == 0) {
LOGW("Demuxer '%s': stream explicitly disabled by the device",
demuxer->name);
sc_demuxer_disable_sinks(demuxer);
eos = true;
sc_packet_source_sinks_disable(&demuxer->packet_source);
status = SC_DEMUXER_STATUS_DISABLED;
goto end;
}
if (raw_codec_id == 1) {
LOGE("Demuxer '%s': stream configuration error on the device",
demuxer->name);
goto end;
}
@@ -199,7 +148,7 @@ run_demuxer(void *data) {
if (codec_id == AV_CODEC_ID_NONE) {
LOGE("Demuxer '%s': stream disabled due to unsupported codec",
demuxer->name);
sc_demuxer_disable_sinks(demuxer);
sc_packet_source_sinks_disable(&demuxer->packet_source);
goto end;
}
@@ -207,11 +156,11 @@ run_demuxer(void *data) {
if (!codec) {
LOGE("Demuxer '%s': stream disabled due to missing decoder",
demuxer->name);
sc_demuxer_disable_sinks(demuxer);
sc_packet_source_sinks_disable(&demuxer->packet_source);
goto end;
}
if (!sc_demuxer_open_sinks(demuxer, codec)) {
if (!sc_packet_source_sinks_open(&demuxer->packet_source, codec)) {
goto end;
}
@@ -235,7 +184,7 @@ run_demuxer(void *data) {
bool ok = sc_demuxer_recv_packet(demuxer, packet);
if (!ok) {
// end of stream
eos = true;
status = SC_DEMUXER_STATUS_EOS;
break;
}
@@ -248,10 +197,10 @@ run_demuxer(void *data) {
}
}
ok = sc_demuxer_push_packet(demuxer, packet);
ok = sc_packet_source_sinks_push(&demuxer->packet_source, packet);
av_packet_unref(packet);
if (!ok) {
// cannot process packet (error already logged)
LOGE("Demuxer '%s': could not process packet", demuxer->name);
break;
}
}
@@ -264,9 +213,9 @@ run_demuxer(void *data) {
av_packet_free(&packet);
finally_close_sinks:
sc_demuxer_close_sinks(demuxer);
sc_packet_source_sinks_close(&demuxer->packet_source);
end:
demuxer->cbs->on_ended(demuxer, eos, demuxer->cbs_userdata);
demuxer->cbs->on_ended(demuxer, status, demuxer->cbs_userdata);
return 0;
}
@@ -278,7 +227,7 @@ sc_demuxer_init(struct sc_demuxer *demuxer, const char *name, sc_socket socket,
demuxer->name = name; // statically allocated
demuxer->socket = socket;
demuxer->sink_count = 0;
sc_packet_source_init(&demuxer->packet_source);
assert(cbs && cbs->on_ended);
@@ -286,14 +235,6 @@ sc_demuxer_init(struct sc_demuxer *demuxer, const char *name, sc_socket socket,
demuxer->cbs_userdata = cbs_userdata;
}
void
sc_demuxer_add_sink(struct sc_demuxer *demuxer, struct sc_packet_sink *sink) {
assert(demuxer->sink_count < SC_DEMUXER_MAX_SINKS);
assert(sink);
assert(sink->ops);
demuxer->sinks[demuxer->sink_count++] = sink;
}
bool
sc_demuxer_start(struct sc_demuxer *demuxer) {
LOGD("Demuxer '%s': starting thread", demuxer->name);

View File

@@ -8,27 +8,32 @@
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include "trait/packet_source.h"
#include "trait/packet_sink.h"
#include "util/net.h"
#include "util/thread.h"
#define SC_DEMUXER_MAX_SINKS 2
struct sc_demuxer {
struct sc_packet_source packet_source; // packet source trait
const char *name; // must be statically allocated (e.g. a string literal)
sc_socket socket;
sc_thread thread;
struct sc_packet_sink *sinks[SC_DEMUXER_MAX_SINKS];
unsigned sink_count;
const struct sc_demuxer_callbacks *cbs;
void *cbs_userdata;
};
enum sc_demuxer_status {
SC_DEMUXER_STATUS_EOS,
SC_DEMUXER_STATUS_DISABLED,
SC_DEMUXER_STATUS_ERROR,
};
struct sc_demuxer_callbacks {
void (*on_ended)(struct sc_demuxer *demuxer, bool eos, void *userdata);
void (*on_ended)(struct sc_demuxer *demuxer, enum sc_demuxer_status,
void *userdata);
};
// The name must be statically allocated (e.g. a string literal)
@@ -36,9 +41,6 @@ void
sc_demuxer_init(struct sc_demuxer *demuxer, const char *name, sc_socket socket,
const struct sc_demuxer_callbacks *cbs, void *cbs_userdata);
void
sc_demuxer_add_sink(struct sc_demuxer *demuxer, struct sc_packet_sink *sink);
bool
sc_demuxer_start(struct sc_demuxer *demuxer);

View File

@@ -19,7 +19,7 @@ sc_file_pusher_init(struct sc_file_pusher *fp, const char *serial,
const char *push_target) {
assert(serial);
cbuf_init(&fp->queue);
sc_vecdeque_init(&fp->queue);
bool ok = sc_mutex_init(&fp->mutex);
if (!ok) {
@@ -65,9 +65,10 @@ sc_file_pusher_destroy(struct sc_file_pusher *fp) {
sc_intr_destroy(&fp->intr);
free(fp->serial);
struct sc_file_pusher_request req;
while (cbuf_take(&fp->queue, &req)) {
sc_file_pusher_request_destroy(&req);
while (!sc_vecdeque_is_empty(&fp->queue)) {
struct sc_file_pusher_request *req = sc_vecdeque_popref(&fp->queue);
assert(req);
sc_file_pusher_request_destroy(req);
}
}
@@ -91,13 +92,20 @@ sc_file_pusher_request(struct sc_file_pusher *fp,
};
sc_mutex_lock(&fp->mutex);
bool was_empty = cbuf_is_empty(&fp->queue);
bool res = cbuf_push(&fp->queue, req);
bool was_empty = sc_vecdeque_is_empty(&fp->queue);
bool res = sc_vecdeque_push(&fp->queue, req);
if (!res) {
LOG_OOM();
sc_mutex_unlock(&fp->mutex);
return false;
}
if (was_empty) {
sc_cond_signal(&fp->event_cond);
}
sc_mutex_unlock(&fp->mutex);
return res;
return true;
}
static int
@@ -113,7 +121,7 @@ run_file_pusher(void *data) {
for (;;) {
sc_mutex_lock(&fp->mutex);
while (!fp->stopped && cbuf_is_empty(&fp->queue)) {
while (!fp->stopped && sc_vecdeque_is_empty(&fp->queue)) {
sc_cond_wait(&fp->event_cond, &fp->mutex);
}
if (fp->stopped) {
@@ -121,10 +129,9 @@ run_file_pusher(void *data) {
sc_mutex_unlock(&fp->mutex);
break;
}
struct sc_file_pusher_request req;
bool non_empty = cbuf_take(&fp->queue, &req);
assert(non_empty);
(void) non_empty;
assert(!sc_vecdeque_is_empty(&fp->queue));
struct sc_file_pusher_request req = sc_vecdeque_pop(&fp->queue);
sc_mutex_unlock(&fp->mutex);
if (req.action == SC_FILE_PUSHER_ACTION_INSTALL_APK) {

View File

@@ -5,9 +5,9 @@
#include <stdbool.h>
#include "util/cbuf.h"
#include "util/thread.h"
#include "util/intr.h"
#include "util/thread.h"
#include "util/vecdeque.h"
enum sc_file_pusher_action {
SC_FILE_PUSHER_ACTION_INSTALL_APK,
@@ -19,7 +19,7 @@ struct sc_file_pusher_request {
char *file;
};
struct sc_file_pusher_request_queue CBUF(struct sc_file_pusher_request, 16);
struct sc_file_pusher_request_queue SC_VECDEQUE(struct sc_file_pusher_request);
struct sc_file_pusher {
char *serial;

View File

@@ -8,7 +8,9 @@ const struct scrcpy_options scrcpy_options_default = {
.push_target = NULL,
.render_driver = NULL,
.video_codec_options = NULL,
.audio_codec_options = NULL,
.video_encoder = NULL,
.audio_encoder = NULL,
#ifdef HAVE_V4L2
.v4l2_device = NULL,
#endif
@@ -41,6 +43,7 @@ const struct scrcpy_options scrcpy_options_default = {
.display_id = 0,
.display_buffer = 0,
.v4l2_buffer = 0,
.audio_buffer = 0,
#ifdef HAVE_USB
.otg = false,
#endif
@@ -70,4 +73,7 @@ const struct scrcpy_options scrcpy_options_default = {
.start_fps_counter = false,
.power_on = true,
.audio = true,
.require_audio = false,
.list_encoders = false,
.list_displays = false,
};

View File

@@ -27,6 +27,7 @@ enum sc_codec {
SC_CODEC_H264,
SC_CODEC_H265,
SC_CODEC_AV1,
SC_CODEC_RAW,
SC_CODEC_OPUS,
SC_CODEC_AAC,
};
@@ -96,7 +97,9 @@ struct scrcpy_options {
const char *push_target;
const char *render_driver;
const char *video_codec_options;
const char *audio_codec_options;
const char *video_encoder;
const char *audio_encoder;
#ifdef HAVE_V4L2
const char *v4l2_device;
#endif
@@ -123,6 +126,7 @@ struct scrcpy_options {
uint32_t display_id;
sc_tick display_buffer;
sc_tick v4l2_buffer;
sc_tick audio_buffer;
#ifdef HAVE_USB
bool otg;
#endif
@@ -152,6 +156,9 @@ struct scrcpy_options {
bool start_fps_counter;
bool power_on;
bool audio;
bool require_audio;
bool list_encoders;
bool list_displays;
};
extern const struct scrcpy_options scrcpy_options_default;

View File

@@ -33,41 +33,27 @@ find_muxer(const char *name) {
return oformat;
}
static struct sc_record_packet *
sc_record_packet_new(const AVPacket *packet) {
struct sc_record_packet *rec = malloc(sizeof(*rec));
if (!rec) {
static AVPacket *
sc_recorder_packet_ref(const AVPacket *packet) {
AVPacket *p = av_packet_alloc();
if (!p) {
LOG_OOM();
return NULL;
}
rec->packet = av_packet_alloc();
if (!rec->packet) {
LOG_OOM();
free(rec);
if (av_packet_ref(p, packet)) {
av_packet_free(&p);
return NULL;
}
if (av_packet_ref(rec->packet, packet)) {
av_packet_free(&rec->packet);
free(rec);
return NULL;
}
return rec;
}
static void
sc_record_packet_delete(struct sc_record_packet *rec) {
av_packet_free(&rec->packet);
free(rec);
return p;
}
static void
sc_recorder_queue_clear(struct sc_recorder_queue *queue) {
while (!sc_queue_is_empty(queue)) {
struct sc_record_packet *rec;
sc_queue_take(queue, next, &rec);
sc_record_packet_delete(rec);
while (!sc_vecdeque_is_empty(queue)) {
AVPacket *p = sc_vecdeque_pop(queue);
av_packet_free(&p);
}
}
@@ -216,7 +202,12 @@ sc_recorder_wait_audio_stream(struct sc_recorder *recorder) {
stream->codecpar->codec_type = AVMEDIA_TYPE_AUDIO;
stream->codecpar->codec_id = codec->id;
#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
stream->codecpar->ch_layout.nb_channels = 2;
#else
stream->codecpar->channel_layout = AV_CH_LAYOUT_STEREO;
stream->codecpar->channels = 2;
#endif
stream->codecpar->sample_rate = 48000;
recorder->audio_stream_index = stream->index;
@@ -227,12 +218,12 @@ sc_recorder_wait_audio_stream(struct sc_recorder *recorder) {
static inline bool
sc_recorder_has_empty_queues(struct sc_recorder *recorder) {
if (sc_queue_is_empty(&recorder->video_queue)) {
if (sc_vecdeque_is_empty(&recorder->video_queue)) {
// The video queue is empty
return true;
}
if (recorder->audio && sc_queue_is_empty(&recorder->audio_queue)) {
if (recorder->audio && sc_vecdeque_is_empty(&recorder->audio_queue)) {
// The audio queue is empty (when audio is enabled)
return true;
}
@@ -249,27 +240,26 @@ sc_recorder_process_header(struct sc_recorder *recorder) {
sc_cond_wait(&recorder->queue_cond, &recorder->mutex);
}
if (recorder->stopped && sc_queue_is_empty(&recorder->video_queue)) {
if (recorder->stopped && sc_vecdeque_is_empty(&recorder->video_queue)) {
// If the recorder is stopped, don't process anything if there are not
// at least video packets
sc_mutex_unlock(&recorder->mutex);
return false;
}
struct sc_record_packet *video_pkt;
sc_queue_take(&recorder->video_queue, next, &video_pkt);
AVPacket *video_pkt = sc_vecdeque_pop(&recorder->video_queue);
struct sc_record_packet *audio_pkt = NULL;
if (!sc_queue_is_empty(&recorder->audio_queue)) {
AVPacket *audio_pkt = NULL;
if (!sc_vecdeque_is_empty(&recorder->audio_queue)) {
assert(recorder->audio);
sc_queue_take(&recorder->audio_queue, next, &audio_pkt);
audio_pkt = sc_vecdeque_pop(&recorder->audio_queue);
}
sc_mutex_unlock(&recorder->mutex);
int ret = false;
if (video_pkt->packet->pts != AV_NOPTS_VALUE) {
if (video_pkt->pts != AV_NOPTS_VALUE) {
LOGE("The first video packet is not a config packet");
goto end;
}
@@ -277,13 +267,13 @@ sc_recorder_process_header(struct sc_recorder *recorder) {
assert(recorder->video_stream_index >= 0);
AVStream *video_stream =
recorder->ctx->streams[recorder->video_stream_index];
bool ok = sc_recorder_set_extradata(video_stream, video_pkt->packet);
bool ok = sc_recorder_set_extradata(video_stream, video_pkt);
if (!ok) {
goto end;
}
if (audio_pkt) {
if (audio_pkt->packet->pts != AV_NOPTS_VALUE) {
if (audio_pkt->pts != AV_NOPTS_VALUE) {
LOGE("The first audio packet is not a config packet");
goto end;
}
@@ -291,7 +281,7 @@ sc_recorder_process_header(struct sc_recorder *recorder) {
assert(recorder->audio_stream_index >= 0);
AVStream *audio_stream =
recorder->ctx->streams[recorder->audio_stream_index];
ok = sc_recorder_set_extradata(audio_stream, audio_pkt->packet);
ok = sc_recorder_set_extradata(audio_stream, audio_pkt);
if (!ok) {
goto end;
}
@@ -306,9 +296,9 @@ sc_recorder_process_header(struct sc_recorder *recorder) {
ret = true;
end:
sc_record_packet_delete(video_pkt);
av_packet_free(&video_pkt);
if (audio_pkt) {
sc_record_packet_delete(audio_pkt);
av_packet_free(&audio_pkt);
}
return ret;
@@ -323,12 +313,12 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
return false;
}
struct sc_record_packet *video_pkt = NULL;
struct sc_record_packet *audio_pkt = NULL;
AVPacket *video_pkt = NULL;
AVPacket *audio_pkt = NULL;
// We can write a video packet only once we received the next one so that
// we can set its duration (next_pts - current_pts)
struct sc_record_packet *video_pkt_previous = NULL;
AVPacket *video_pkt_previous = NULL;
bool error = false;
@@ -336,12 +326,12 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
sc_mutex_lock(&recorder->mutex);
while (!recorder->stopped) {
if (!video_pkt && !sc_queue_is_empty(&recorder->video_queue)) {
if (!video_pkt && !sc_vecdeque_is_empty(&recorder->video_queue)) {
// A new packet may be assigned to video_pkt and be processed
break;
}
if (recorder->audio && !audio_pkt
&& !sc_queue_is_empty(&recorder->audio_queue)) {
&& !sc_vecdeque_is_empty(&recorder->audio_queue)) {
// A new packet may be assigned to audio_pkt and be processed
break;
}
@@ -353,20 +343,20 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
// If there is no audio, then the audio_queue will remain empty forever
// and audio_pkt will always be NULL.
assert(recorder->audio
|| (!audio_pkt && sc_queue_is_empty(&recorder->audio_queue)));
assert(recorder->audio || (!audio_pkt
&& sc_vecdeque_is_empty(&recorder->audio_queue)));
if (!video_pkt && !sc_queue_is_empty(&recorder->video_queue)) {
sc_queue_take(&recorder->video_queue, next, &video_pkt);
if (!video_pkt && !sc_vecdeque_is_empty(&recorder->video_queue)) {
video_pkt = sc_vecdeque_pop(&recorder->video_queue);
}
if (!audio_pkt && !sc_queue_is_empty(&recorder->audio_queue)) {
sc_queue_take(&recorder->audio_queue, next, &audio_pkt);
if (!audio_pkt && !sc_vecdeque_is_empty(&recorder->audio_queue)) {
audio_pkt = sc_vecdeque_pop(&recorder->audio_queue);
}
if (recorder->stopped && !video_pkt && !audio_pkt) {
assert(sc_queue_is_empty(&recorder->video_queue));
assert(sc_queue_is_empty(&recorder->audio_queue));
assert(sc_vecdeque_is_empty(&recorder->video_queue));
assert(sc_vecdeque_is_empty(&recorder->audio_queue));
sc_mutex_unlock(&recorder->mutex);
break;
}
@@ -378,28 +368,27 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
// Ignore further config packets (e.g. on device orientation
// change). The next non-config packet will have the config packet
// data prepended.
if (video_pkt && video_pkt->packet->pts == AV_NOPTS_VALUE) {
sc_record_packet_delete(video_pkt);
if (video_pkt && video_pkt->pts == AV_NOPTS_VALUE) {
av_packet_free(&video_pkt);
video_pkt = NULL;
}
if (audio_pkt && audio_pkt->packet->pts == AV_NOPTS_VALUE) {
sc_record_packet_delete(audio_pkt);
audio_pkt= NULL;
if (audio_pkt && audio_pkt->pts == AV_NOPTS_VALUE) {
av_packet_free(&audio_pkt);
audio_pkt = NULL;
}
if (pts_origin == AV_NOPTS_VALUE) {
if (!recorder->audio) {
assert(video_pkt);
pts_origin = video_pkt->packet->pts;
pts_origin = video_pkt->pts;
} else if (video_pkt && audio_pkt) {
pts_origin =
MIN(video_pkt->packet->pts, audio_pkt->packet->pts);
pts_origin = MIN(video_pkt->pts, audio_pkt->pts);
} else if (recorder->stopped) {
if (video_pkt) {
// The recorder is stopped without audio, record the video
// packets
pts_origin = video_pkt->packet->pts;
pts_origin = video_pkt->pts;
} else {
// Fail if there is no video
error = true;
@@ -408,8 +397,7 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
// If the recorder is stopped while one of the streams has no
// packets, then we must avoid a live-loop and correctly record
// the stream having packets.
pts_origin = video_pkt ? video_pkt->packet->pts
: audio_pkt->packet->pts;
pts_origin = video_pkt ? video_pkt->pts : audio_pkt->pts;
} else {
// We need both video and audio packets to initialize pts_origin
continue;
@@ -419,17 +407,16 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
assert(pts_origin != AV_NOPTS_VALUE);
if (video_pkt) {
video_pkt->packet->pts -= pts_origin;
video_pkt->packet->dts = video_pkt->packet->pts;
video_pkt->pts -= pts_origin;
video_pkt->dts = video_pkt->pts;
if (video_pkt_previous) {
// we now know the duration of the previous packet
video_pkt_previous->packet->duration =
video_pkt->packet->pts - video_pkt_previous->packet->pts;
video_pkt_previous->duration = video_pkt->pts
- video_pkt_previous->pts;
bool ok = sc_recorder_write_video(recorder,
video_pkt_previous->packet);
sc_record_packet_delete(video_pkt_previous);
bool ok = sc_recorder_write_video(recorder, video_pkt_previous);
av_packet_free(&video_pkt_previous);
if (!ok) {
LOGE("Could not record video packet");
error = true;
@@ -442,34 +429,34 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
}
if (audio_pkt) {
audio_pkt->packet->pts -= pts_origin;
audio_pkt->packet->dts = audio_pkt->packet->pts;
audio_pkt->pts -= pts_origin;
audio_pkt->dts = audio_pkt->pts;
bool ok = sc_recorder_write_audio(recorder, audio_pkt->packet);
bool ok = sc_recorder_write_audio(recorder, audio_pkt);
if (!ok) {
LOGE("Could not record audio packet");
error = true;
goto end;
}
sc_record_packet_delete(audio_pkt);
av_packet_free(&audio_pkt);
audio_pkt = NULL;
}
}
// Write the last video packet
struct sc_record_packet *last = video_pkt_previous;
AVPacket *last = video_pkt_previous;
if (last) {
// assign an arbitrary duration to the last packet
last->packet->duration = 100000;
bool ok = sc_recorder_write_video(recorder, last->packet);
last->duration = 100000;
bool ok = sc_recorder_write_video(recorder, last);
if (!ok) {
// failing to write the last frame is not very serious, no
// future frame may depend on it, so the resulting file
// will still be valid
LOGW("Could not record last packet");
}
sc_record_packet_delete(last);
av_packet_free(&last);
}
int ret = av_write_trailer(recorder->ctx);
@@ -480,10 +467,10 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
end:
if (video_pkt) {
sc_record_packet_delete(video_pkt);
av_packet_free(&video_pkt);
}
if (audio_pkt) {
sc_record_packet_delete(audio_pkt);
av_packet_free(&audio_pkt);
}
return !error;
@@ -589,16 +576,22 @@ sc_recorder_video_packet_sink_push(struct sc_packet_sink *sink,
return false;
}
struct sc_record_packet *rec = sc_record_packet_new(packet);
AVPacket *rec = sc_recorder_packet_ref(packet);
if (!rec) {
LOG_OOM();
sc_mutex_unlock(&recorder->mutex);
return false;
}
rec->packet->stream_index = 0;
rec->stream_index = 0;
bool ok = sc_vecdeque_push(&recorder->video_queue, rec);
if (!ok) {
LOG_OOM();
sc_mutex_unlock(&recorder->mutex);
return false;
}
sc_queue_push(&recorder->video_queue, next, rec);
sc_cond_signal(&recorder->queue_cond);
sc_mutex_unlock(&recorder->mutex);
@@ -652,16 +645,22 @@ sc_recorder_audio_packet_sink_push(struct sc_packet_sink *sink,
return false;
}
struct sc_record_packet *rec = sc_record_packet_new(packet);
AVPacket *rec = sc_recorder_packet_ref(packet);
if (!rec) {
LOG_OOM();
sc_mutex_unlock(&recorder->mutex);
return false;
}
rec->packet->stream_index = 1;
rec->stream_index = 1;
bool ok = sc_vecdeque_push(&recorder->audio_queue, rec);
if (!ok) {
LOG_OOM();
sc_mutex_unlock(&recorder->mutex);
return false;
}
sc_queue_push(&recorder->audio_queue, next, rec);
sc_cond_signal(&recorder->queue_cond);
sc_mutex_unlock(&recorder->mutex);
@@ -712,8 +711,8 @@ sc_recorder_init(struct sc_recorder *recorder, const char *filename,
recorder->audio = audio;
sc_queue_init(&recorder->video_queue);
sc_queue_init(&recorder->audio_queue);
sc_vecdeque_init(&recorder->video_queue);
sc_vecdeque_init(&recorder->audio_queue);
recorder->stopped = false;
recorder->video_codec = NULL;

View File

@@ -9,15 +9,10 @@
#include "coords.h"
#include "options.h"
#include "trait/packet_sink.h"
#include "util/queue.h"
#include "util/thread.h"
#include "util/vecdeque.h"
struct sc_record_packet {
AVPacket *packet;
struct sc_record_packet *next;
};
struct sc_recorder_queue SC_QUEUE(struct sc_record_packet);
struct sc_recorder_queue SC_VECDEQUE(AVPacket *);
struct sc_recorder {
struct sc_packet_sink video_packet_sink;

View File

@@ -13,8 +13,10 @@
# include <windows.h>
#endif
#include "audio_player.h"
#include "controller.h"
#include "decoder.h"
#include "delay_buffer.h"
#include "demuxer.h"
#include "events.h"
#include "file_pusher.h"
@@ -40,12 +42,17 @@
struct scrcpy {
struct sc_server server;
struct sc_screen screen;
struct sc_audio_player audio_player;
struct sc_delay_buffer audio_buffer;
struct sc_demuxer video_demuxer;
struct sc_demuxer audio_demuxer;
struct sc_decoder decoder;
struct sc_decoder video_decoder;
struct sc_decoder audio_decoder;
struct sc_recorder recorder;
struct sc_delay_buffer display_buffer;
#ifdef HAVE_V4L2
struct sc_v4l2_sink v4l2_sink;
struct sc_delay_buffer v4l2_buffer;
#endif
struct sc_controller controller;
struct sc_file_pusher file_pusher;
@@ -183,15 +190,16 @@ await_for_server(bool *connected) {
while (SDL_WaitEvent(&event)) {
switch (event.type) {
case SDL_QUIT:
LOGD("User requested to quit");
*connected = false;
if (connected) {
*connected = false;
}
return true;
case SC_EVENT_SERVER_CONNECTION_FAILED:
LOGE("Server connection failed");
return false;
case SC_EVENT_SERVER_CONNECTED:
LOGD("Server connected");
*connected = true;
if (connected) {
*connected = true;
}
return true;
default:
break;
@@ -214,12 +222,15 @@ sc_recorder_on_ended(struct sc_recorder *recorder, bool success,
}
static void
sc_video_demuxer_on_ended(struct sc_demuxer *demuxer, bool eos,
void *userdata) {
sc_video_demuxer_on_ended(struct sc_demuxer *demuxer,
enum sc_demuxer_status status, void *userdata) {
(void) demuxer;
(void) userdata;
if (eos) {
// The device may not decide to disable the video
assert(status != SC_DEMUXER_STATUS_DISABLED);
if (status == SC_DEMUXER_STATUS_EOS) {
PUSH_EVENT(SC_EVENT_DEVICE_DISCONNECTED);
} else {
PUSH_EVENT(SC_EVENT_DEMUXER_ERROR);
@@ -227,13 +238,26 @@ sc_video_demuxer_on_ended(struct sc_demuxer *demuxer, bool eos,
}
static void
sc_audio_demuxer_on_ended(struct sc_demuxer *demuxer, bool eos,
void *userdata) {
sc_audio_demuxer_on_ended(struct sc_demuxer *demuxer,
enum sc_demuxer_status status, void *userdata) {
(void) demuxer;
(void) eos;
(void) userdata;
const struct scrcpy_options *options = userdata;
// Contrary to the video demuxer, keep mirroring if only the audio fails
// (unless --require-audio is set).
// 'eos' is true on end-of-stream, including when audio capture is not
// possible on the device (so that scrcpy continue to mirror video without
// failing).
// However, if an audio configuration failure occurs (for example the user
// explicitly selected an unknown audio encoder), 'eos' is false and scrcpy
// must exit.
if (status == SC_DEMUXER_STATUS_ERROR
|| (status == SC_DEMUXER_STATUS_DISABLED
&& options->require_audio)) {
PUSH_EVENT(SC_EVENT_DEMUXER_ERROR);
}
}
static void
@@ -331,7 +355,9 @@ scrcpy(struct scrcpy_options *options) {
.show_touches = options->show_touches,
.stay_awake = options->stay_awake,
.video_codec_options = options->video_codec_options,
.audio_codec_options = options->audio_codec_options,
.video_encoder = options->video_encoder,
.audio_encoder = options->audio_encoder,
.force_adb_forward = options->force_adb_forward,
.power_off_on_close = options->power_off_on_close,
.clipboard_autosync = options->clipboard_autosync,
@@ -340,6 +366,8 @@ scrcpy(struct scrcpy_options *options) {
.tcpip_dst = options->tcpip_dst,
.cleanup = options->cleanup,
.power_on = options->power_on,
.list_encoders = options->list_encoders,
.list_displays = options->list_displays,
};
static const struct sc_server_callbacks cbs = {
@@ -357,14 +385,27 @@ scrcpy(struct scrcpy_options *options) {
server_started = true;
if (options->list_encoders || options->list_displays) {
bool ok = await_for_server(NULL);
ret = ok ? SCRCPY_EXIT_SUCCESS : SCRCPY_EXIT_FAILURE;
goto end;
}
if (options->display) {
sdl_set_hints(options->render_driver);
}
// Initialize SDL video in addition if display is enabled
if (options->display && SDL_Init(SDL_INIT_VIDEO)) {
LOGE("Could not initialize SDL: %s", SDL_GetError());
goto end;
if (options->display) {
if (SDL_Init(SDL_INIT_VIDEO)) {
LOGE("Could not initialize SDL video: %s", SDL_GetError());
goto end;
}
if (options->audio && SDL_Init(SDL_INIT_AUDIO)) {
LOGE("Could not initialize SDL audio: %s", SDL_GetError());
goto end;
}
}
sdl_configure(options->display, options->disable_screensaver);
@@ -372,15 +413,19 @@ scrcpy(struct scrcpy_options *options) {
// Await for server without blocking Ctrl+C handling
bool connected;
if (!await_for_server(&connected)) {
LOGE("Server connection failed");
goto end;
}
if (!connected) {
// This is not an error, user requested to quit
LOGD("User requested to quit");
ret = SCRCPY_EXIT_SUCCESS;
goto end;
}
LOGD("Server connected");
// It is necessarily initialized here, since the device is connected
struct sc_server_info *info = &s->server.info;
@@ -409,16 +454,23 @@ scrcpy(struct scrcpy_options *options) {
.on_ended = sc_audio_demuxer_on_ended,
};
sc_demuxer_init(&s->audio_demuxer, "audio", s->server.audio_socket,
&audio_demuxer_cbs, NULL);
&audio_demuxer_cbs, options);
}
bool needs_decoder = options->display;
bool needs_video_decoder = options->display;
bool needs_audio_decoder = options->audio && options->display;
#ifdef HAVE_V4L2
needs_decoder |= !!options->v4l2_device;
needs_video_decoder |= !!options->v4l2_device;
#endif
if (needs_decoder) {
sc_decoder_init(&s->decoder);
sc_demuxer_add_sink(&s->video_demuxer, &s->decoder.packet_sink);
if (needs_video_decoder) {
sc_decoder_init(&s->video_decoder, "video");
sc_packet_source_add_sink(&s->video_demuxer.packet_source,
&s->video_decoder.packet_sink);
}
if (needs_audio_decoder) {
sc_decoder_init(&s->audio_decoder, "audio");
sc_packet_source_add_sink(&s->audio_demuxer.packet_source,
&s->audio_decoder.packet_sink);
}
if (options->record_filename) {
@@ -437,10 +489,11 @@ scrcpy(struct scrcpy_options *options) {
}
recorder_started = true;
sc_demuxer_add_sink(&s->video_demuxer, &s->recorder.video_packet_sink);
sc_packet_source_add_sink(&s->video_demuxer.packet_source,
&s->recorder.video_packet_sink);
if (options->audio) {
sc_demuxer_add_sink(&s->audio_demuxer,
&s->recorder.audio_packet_sink);
sc_packet_source_add_sink(&s->audio_demuxer.packet_source,
&s->recorder.audio_packet_sink);
}
}
@@ -624,7 +677,6 @@ aoa_hid_end:
.mipmaps = options->mipmaps,
.fullscreen = options->fullscreen,
.start_fps_counter = options->start_fps_counter,
.buffering_time = options->display_buffer,
};
if (!sc_screen_init(&s->screen, &screen_params)) {
@@ -632,17 +684,45 @@ aoa_hid_end:
}
screen_initialized = true;
sc_decoder_add_sink(&s->decoder, &s->screen.frame_sink);
struct sc_frame_source *src = &s->video_decoder.frame_source;
if (options->display_buffer) {
sc_delay_buffer_init(&s->display_buffer, options->display_buffer,
true);
sc_frame_source_add_sink(src, &s->display_buffer.frame_sink);
src = &s->display_buffer.frame_source;
}
sc_frame_source_add_sink(src, &s->screen.frame_sink);
if (options->audio) {
struct sc_frame_source *src = &s->audio_decoder.frame_source;
if (options->audio_buffer) {
sc_delay_buffer_init(&s->audio_buffer, options->audio_buffer,
false);
sc_frame_source_add_sink(src, &s->audio_buffer.frame_sink);
src = &s->audio_buffer.frame_source;
}
sc_audio_player_init(&s->audio_player);
sc_frame_source_add_sink(src, &s->audio_player.frame_sink);
}
}
#ifdef HAVE_V4L2
if (options->v4l2_device) {
if (!sc_v4l2_sink_init(&s->v4l2_sink, options->v4l2_device,
info->frame_size, options->v4l2_buffer)) {
info->frame_size)) {
goto end;
}
sc_decoder_add_sink(&s->decoder, &s->v4l2_sink.frame_sink);
struct sc_frame_source *src = &s->video_decoder.frame_source;
if (options->v4l2_buffer) {
sc_delay_buffer_init(&s->v4l2_buffer, options->v4l2_buffer, true);
sc_frame_source_add_sink(src, &s->v4l2_buffer.frame_sink);
src = &s->v4l2_buffer.frame_source;
}
sc_frame_source_add_sink(src, &s->v4l2_sink.frame_sink);
v4l2_sink_initialized = true;
}

View File

@@ -7,7 +7,6 @@
#include "events.h"
#include "icon.h"
#include "options.h"
#include "video_buffer.h"
#include "util/log.h"
#define DISPLAY_MARGINS 96
@@ -330,7 +329,11 @@ event_watcher(void *data, SDL_Event *event) {
#endif
static bool
sc_screen_frame_sink_open(struct sc_frame_sink *sink) {
sc_screen_frame_sink_open(struct sc_frame_sink *sink,
const AVCodecContext *ctx) {
assert(ctx->pix_fmt == AV_PIX_FMT_YUV420P);
(void) ctx;
struct sc_screen *screen = DOWNCAST(sink);
(void) screen;
#ifndef NDEBUG
@@ -355,30 +358,18 @@ sc_screen_frame_sink_close(struct sc_frame_sink *sink) {
static bool
sc_screen_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
struct sc_screen *screen = DOWNCAST(sink);
return sc_video_buffer_push(&screen->vb, frame);
}
static void
sc_video_buffer_on_new_frame(struct sc_video_buffer *vb, bool previous_skipped,
void *userdata) {
(void) vb;
struct sc_screen *screen = userdata;
bool previous_skipped;
bool ok = sc_frame_buffer_push(&screen->fb, frame, &previous_skipped);
if (!ok) {
return false;
}
// event_failed implies previous_skipped (the previous frame may not have
// been consumed if the event was not sent)
assert(!screen->event_failed || previous_skipped);
bool need_new_event;
if (previous_skipped) {
sc_fps_counter_add_skipped_frame(&screen->fps_counter);
// The SC_EVENT_NEW_FRAME triggered for the previous frame will consume
// this new frame instead, unless the previous event failed
need_new_event = screen->event_failed;
// this new frame instead
} else {
need_new_event = true;
}
if (need_new_event) {
static SDL_Event new_frame_event = {
.type = SC_EVENT_NEW_FRAME,
};
@@ -387,11 +378,11 @@ sc_video_buffer_on_new_frame(struct sc_video_buffer *vb, bool previous_skipped,
int ret = SDL_PushEvent(&new_frame_event);
if (ret < 0) {
LOGW("Could not post new frame event: %s", SDL_GetError());
screen->event_failed = true;
} else {
screen->event_failed = false;
return false;
}
}
return true;
}
bool
@@ -401,7 +392,6 @@ sc_screen_init(struct sc_screen *screen,
screen->has_frame = false;
screen->fullscreen = false;
screen->maximized = false;
screen->event_failed = false;
screen->mouse_capture_key_pressed = 0;
screen->req.x = params->window_x;
@@ -411,23 +401,13 @@ sc_screen_init(struct sc_screen *screen,
screen->req.fullscreen = params->fullscreen;
screen->req.start_fps_counter = params->start_fps_counter;
static const struct sc_video_buffer_callbacks cbs = {
.on_new_frame = sc_video_buffer_on_new_frame,
};
bool ok = sc_video_buffer_init(&screen->vb, params->buffering_time, &cbs,
screen);
bool ok = sc_frame_buffer_init(&screen->fb);
if (!ok) {
return false;
}
ok = sc_video_buffer_start(&screen->vb);
if (!ok) {
goto error_destroy_video_buffer;
}
if (!sc_fps_counter_init(&screen->fps_counter)) {
goto error_stop_and_join_video_buffer;
goto error_destroy_frame_buffer;
}
screen->frame_size = params->frame_size;
@@ -559,11 +539,8 @@ error_destroy_window:
SDL_DestroyWindow(screen->window);
error_destroy_fps_counter:
sc_fps_counter_destroy(&screen->fps_counter);
error_stop_and_join_video_buffer:
sc_video_buffer_stop(&screen->vb);
sc_video_buffer_join(&screen->vb);
error_destroy_video_buffer:
sc_video_buffer_destroy(&screen->vb);
error_destroy_frame_buffer:
sc_frame_buffer_destroy(&screen->fb);
return false;
}
@@ -600,13 +577,11 @@ sc_screen_hide_window(struct sc_screen *screen) {
void
sc_screen_interrupt(struct sc_screen *screen) {
sc_video_buffer_stop(&screen->vb);
sc_fps_counter_interrupt(&screen->fps_counter);
}
void
sc_screen_join(struct sc_screen *screen) {
sc_video_buffer_join(&screen->vb);
sc_fps_counter_join(&screen->fps_counter);
}
@@ -620,7 +595,7 @@ sc_screen_destroy(struct sc_screen *screen) {
SDL_DestroyRenderer(screen->renderer);
SDL_DestroyWindow(screen->window);
sc_fps_counter_destroy(&screen->fps_counter);
sc_video_buffer_destroy(&screen->vb);
sc_frame_buffer_destroy(&screen->fb);
}
static void
@@ -726,7 +701,7 @@ update_texture(struct sc_screen *screen, const AVFrame *frame) {
static bool
sc_screen_update_frame(struct sc_screen *screen) {
av_frame_unref(screen->frame);
sc_video_buffer_consume(&screen->vb, screen->frame);
sc_frame_buffer_consume(&screen->fb, screen->frame);
AVFrame *frame = screen->frame;
sc_fps_counter_add_rendered_frame(&screen->fps_counter);

View File

@@ -10,12 +10,12 @@
#include "controller.h"
#include "coords.h"
#include "fps_counter.h"
#include "frame_buffer.h"
#include "input_manager.h"
#include "opengl.h"
#include "trait/key_processor.h"
#include "trait/frame_sink.h"
#include "trait/mouse_processor.h"
#include "video_buffer.h"
struct sc_screen {
struct sc_frame_sink frame_sink; // frame sink trait
@@ -25,7 +25,7 @@ struct sc_screen {
#endif
struct sc_input_manager im;
struct sc_video_buffer vb;
struct sc_frame_buffer fb;
struct sc_fps_counter fps_counter;
// The initial requested window properties
@@ -59,8 +59,6 @@ struct sc_screen {
bool maximized;
bool mipmaps;
bool event_failed; // in case SDL_PushEvent() returned an error
// To enable/disable mouse capture, a mouse capture key (LALT, LGUI or
// RGUI) must be pressed. This variable tracks the pressed capture key.
SDL_Keycode mouse_capture_key_pressed;
@@ -95,8 +93,6 @@ struct sc_screen_params {
bool fullscreen;
bool start_fps_counter;
sc_tick buffering_time;
};
// initialize screen, create window, renderer and texture (window is hidden)

View File

@@ -72,7 +72,9 @@ sc_server_params_destroy(struct sc_server_params *params) {
free((char *) params->req_serial);
free((char *) params->crop);
free((char *) params->video_codec_options);
free((char *) params->audio_codec_options);
free((char *) params->video_encoder);
free((char *) params->audio_encoder);
free((char *) params->tcpip_dst);
}
@@ -96,7 +98,9 @@ sc_server_params_copy(struct sc_server_params *dst,
COPY(req_serial);
COPY(crop);
COPY(video_codec_options);
COPY(audio_codec_options);
COPY(video_encoder);
COPY(audio_encoder);
COPY(tcpip_dst);
#undef COPY
@@ -165,6 +169,8 @@ sc_server_get_codec_name(enum sc_codec codec) {
return "h265";
case SC_CODEC_AV1:
return "av1";
case SC_CODEC_RAW:
return "raw";
case SC_CODEC_OPUS:
return "opus";
case SC_CODEC_AAC:
@@ -268,9 +274,15 @@ execute_server(struct sc_server *server,
if (params->video_codec_options) {
ADD_PARAM("video_codec_options=%s", params->video_codec_options);
}
if (params->audio_codec_options) {
ADD_PARAM("audio_codec_options=%s", params->audio_codec_options);
}
if (params->video_encoder) {
ADD_PARAM("video_encoder=%s", params->video_encoder);
}
if (params->audio_encoder) {
ADD_PARAM("audio_encoder=%s", params->audio_encoder);
}
if (params->power_off_on_close) {
ADD_PARAM("power_off_on_close=true");
}
@@ -290,6 +302,12 @@ execute_server(struct sc_server *server,
// By default, power_on is true
ADD_PARAM("power_on=false");
}
if (params->list_encoders) {
ADD_PARAM("list_encoders=true");
}
if (params->list_displays) {
ADD_PARAM("list_displays=true");
}
#undef ADD_PARAM
@@ -838,6 +856,25 @@ run_server(void *data) {
assert(serial);
LOGD("Device serial: %s", serial);
ok = push_server(&server->intr, serial);
if (!ok) {
goto error_connection_failed;
}
// If --list-* is passed, then the server just prints the requested data
// then exits.
if (params->list_encoders || params->list_displays) {
sc_pid pid = execute_server(server, params);
if (pid == SC_PROCESS_NONE) {
goto error_connection_failed;
}
sc_process_wait(pid, NULL); // ignore exit code
sc_process_close(pid);
// Wake up await_for_server()
server->cbs->on_connected(server, server->cbs_userdata);
return 0;
}
int r = asprintf(&server->device_socket_name, SC_SOCKET_NAME_PREFIX "%08x",
params->scid);
if (r == -1) {
@@ -847,11 +884,6 @@ run_server(void *data) {
assert(r == sizeof(SC_SOCKET_NAME_PREFIX) - 1 + 8);
assert(server->device_socket_name);
ok = push_server(&server->intr, serial);
if (!ok) {
goto error_connection_failed;
}
ok = sc_adb_tunnel_open(&server->tunnel, &server->intr, serial,
server->device_socket_name, params->port_range,
params->force_adb_forward);

View File

@@ -29,7 +29,9 @@ struct sc_server_params {
enum sc_codec audio_codec;
const char *crop;
const char *video_codec_options;
const char *audio_codec_options;
const char *video_encoder;
const char *audio_encoder;
struct sc_port_range port_range;
uint32_t tunnel_host;
uint16_t tunnel_port;
@@ -53,6 +55,8 @@ struct sc_server_params {
bool select_tcpip;
bool cleanup;
bool power_on;
bool list_encoders;
bool list_displays;
};
struct sc_server {

View File

@@ -5,6 +5,7 @@
#include <assert.h>
#include <stdbool.h>
#include <libavcodec/avcodec.h>
typedef struct AVFrame AVFrame;
@@ -18,7 +19,7 @@ struct sc_frame_sink {
};
struct sc_frame_sink_ops {
bool (*open)(struct sc_frame_sink *sink);
bool (*open)(struct sc_frame_sink *sink, const AVCodecContext *ctx);
void (*close)(struct sc_frame_sink *sink);
bool (*push)(struct sc_frame_sink *sink, const AVFrame *frame);
};

View File

@@ -0,0 +1,59 @@
#include "frame_source.h"
void
sc_frame_source_init(struct sc_frame_source *source) {
source->sink_count = 0;
}
void
sc_frame_source_add_sink(struct sc_frame_source *source,
struct sc_frame_sink *sink) {
assert(source->sink_count < SC_FRAME_SOURCE_MAX_SINKS);
assert(sink);
assert(sink->ops);
source->sinks[source->sink_count++] = sink;
}
static void
sc_frame_source_sinks_close_firsts(struct sc_frame_source *source,
unsigned count) {
while (count) {
struct sc_frame_sink *sink = source->sinks[--count];
sink->ops->close(sink);
}
}
bool
sc_frame_source_sinks_open(struct sc_frame_source *source,
const AVCodecContext *ctx) {
assert(source->sink_count);
for (unsigned i = 0; i < source->sink_count; ++i) {
struct sc_frame_sink *sink = source->sinks[i];
if (!sink->ops->open(sink, ctx)) {
sc_frame_source_sinks_close_firsts(source, i);
return false;
}
}
return true;
}
void
sc_frame_source_sinks_close(struct sc_frame_source *source) {
assert(source->sink_count);
sc_frame_source_sinks_close_firsts(source, source->sink_count);
}
bool
sc_frame_source_sinks_push(struct sc_frame_source *source,
const AVFrame *frame) {
assert(source->sink_count);
for (unsigned i = 0; i < source->sink_count; ++i) {
struct sc_frame_sink *sink = source->sinks[i];
if (!sink->ops->push(sink, frame)) {
return false;
}
}
return true;
}

View File

@@ -0,0 +1,38 @@
#ifndef SC_FRAME_SOURCE_H
#define SC_FRAME_SOURCE_H
#include "common.h"
#include "frame_sink.h"
#define SC_FRAME_SOURCE_MAX_SINKS 2
/**
* Frame source trait
*
* Component able to send AVFrames should implement this trait.
*/
struct sc_frame_source {
struct sc_frame_sink *sinks[SC_FRAME_SOURCE_MAX_SINKS];
unsigned sink_count;
};
void
sc_frame_source_init(struct sc_frame_source *source);
void
sc_frame_source_add_sink(struct sc_frame_source *source,
struct sc_frame_sink *sink);
bool
sc_frame_source_sinks_open(struct sc_frame_source *source,
const AVCodecContext *ctx);
void
sc_frame_source_sinks_close(struct sc_frame_source *source);
bool
sc_frame_source_sinks_push(struct sc_frame_source *source,
const AVFrame *frame);
#endif

View File

@@ -0,0 +1,70 @@
#include "packet_source.h"
void
sc_packet_source_init(struct sc_packet_source *source) {
source->sink_count = 0;
}
void
sc_packet_source_add_sink(struct sc_packet_source *source,
struct sc_packet_sink *sink) {
assert(source->sink_count < SC_PACKET_SOURCE_MAX_SINKS);
assert(sink);
assert(sink->ops);
source->sinks[source->sink_count++] = sink;
}
static void
sc_packet_source_sinks_close_firsts(struct sc_packet_source *source,
unsigned count) {
while (count) {
struct sc_packet_sink *sink = source->sinks[--count];
sink->ops->close(sink);
}
}
bool
sc_packet_source_sinks_open(struct sc_packet_source *source,
const AVCodec *codec) {
assert(source->sink_count);
for (unsigned i = 0; i < source->sink_count; ++i) {
struct sc_packet_sink *sink = source->sinks[i];
if (!sink->ops->open(sink, codec)) {
sc_packet_source_sinks_close_firsts(source, i);
return false;
}
}
return true;
}
void
sc_packet_source_sinks_close(struct sc_packet_source *source) {
assert(source->sink_count);
sc_packet_source_sinks_close_firsts(source, source->sink_count);
}
bool
sc_packet_source_sinks_push(struct sc_packet_source *source,
const AVPacket *packet) {
assert(source->sink_count);
for (unsigned i = 0; i < source->sink_count; ++i) {
struct sc_packet_sink *sink = source->sinks[i];
if (!sink->ops->push(sink, packet)) {
return false;
}
}
return true;
}
void
sc_packet_source_sinks_disable(struct sc_packet_source *source) {
assert(source->sink_count);
for (unsigned i = 0; i < source->sink_count; ++i) {
struct sc_packet_sink *sink = source->sinks[i];
if (sink->ops->disable) {
sink->ops->disable(sink);
}
}
}

View File

@@ -0,0 +1,41 @@
#ifndef SC_PACKET_SOURCE_H
#define SC_PACKET_SOURCE_H
#include "common.h"
#include "packet_sink.h"
#define SC_PACKET_SOURCE_MAX_SINKS 2
/**
* Packet source trait
*
* Component able to send AVPackets should implement this trait.
*/
struct sc_packet_source {
struct sc_packet_sink *sinks[SC_PACKET_SOURCE_MAX_SINKS];
unsigned sink_count;
};
void
sc_packet_source_init(struct sc_packet_source *source);
void
sc_packet_source_add_sink(struct sc_packet_source *source,
struct sc_packet_sink *sink);
bool
sc_packet_source_sinks_open(struct sc_packet_source *source,
const AVCodec *codec);
void
sc_packet_source_sinks_close(struct sc_packet_source *source);
bool
sc_packet_source_sinks_push(struct sc_packet_source *source,
const AVPacket *packet);
void
sc_packet_source_sinks_disable(struct sc_packet_source *source);
#endif

View File

@@ -14,6 +14,8 @@
#define DEFAULT_TIMEOUT 1000
#define SC_HID_EVENT_QUEUE_MAX 64
static void
sc_hid_event_log(const struct sc_hid_event *event) {
// HID Event: [00] FF FF FF FF...
@@ -48,14 +50,20 @@ sc_hid_event_destroy(struct sc_hid_event *hid_event) {
bool
sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
struct sc_acksync *acksync) {
cbuf_init(&aoa->queue);
sc_vecdeque_init(&aoa->queue);
if (!sc_vecdeque_reserve(&aoa->queue, SC_HID_EVENT_QUEUE_MAX)) {
return false;
}
if (!sc_mutex_init(&aoa->mutex)) {
sc_vecdeque_destroy(&aoa->queue);
return false;
}
if (!sc_cond_init(&aoa->event_cond)) {
sc_mutex_destroy(&aoa->mutex);
sc_vecdeque_destroy(&aoa->queue);
return false;
}
@@ -69,9 +77,10 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
void
sc_aoa_destroy(struct sc_aoa *aoa) {
// Destroy remaining events
struct sc_hid_event event;
while (cbuf_take(&aoa->queue, &event)) {
sc_hid_event_destroy(&event);
while (!sc_vecdeque_is_empty(&aoa->queue)) {
struct sc_hid_event *event = sc_vecdeque_popref(&aoa->queue);
assert(event);
sc_hid_event_destroy(event);
}
sc_cond_destroy(&aoa->event_cond);
@@ -212,13 +221,19 @@ sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
}
sc_mutex_lock(&aoa->mutex);
bool was_empty = cbuf_is_empty(&aoa->queue);
bool res = cbuf_push(&aoa->queue, *event);
if (was_empty) {
sc_cond_signal(&aoa->event_cond);
bool full = sc_vecdeque_is_full(&aoa->queue);
if (!full) {
bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
sc_vecdeque_push_noresize(&aoa->queue, *event);
if (was_empty) {
sc_cond_signal(&aoa->event_cond);
}
}
// Otherwise (if the queue is full), the event is discarded
sc_mutex_unlock(&aoa->mutex);
return res;
return !full;
}
static int
@@ -227,7 +242,7 @@ run_aoa_thread(void *data) {
for (;;) {
sc_mutex_lock(&aoa->mutex);
while (!aoa->stopped && cbuf_is_empty(&aoa->queue)) {
while (!aoa->stopped && sc_vecdeque_is_empty(&aoa->queue)) {
sc_cond_wait(&aoa->event_cond, &aoa->mutex);
}
if (aoa->stopped) {
@@ -235,11 +250,9 @@ run_aoa_thread(void *data) {
sc_mutex_unlock(&aoa->mutex);
break;
}
struct sc_hid_event event;
bool non_empty = cbuf_take(&aoa->queue, &event);
assert(non_empty);
(void) non_empty;
assert(!sc_vecdeque_is_empty(&aoa->queue));
struct sc_hid_event event = sc_vecdeque_pop(&aoa->queue);
uint64_t ack_to_wait = event.ack_to_wait;
sc_mutex_unlock(&aoa->mutex);

View File

@@ -8,9 +8,9 @@
#include "usb.h"
#include "util/acksync.h"
#include "util/cbuf.h"
#include "util/thread.h"
#include "util/tick.h"
#include "util/vecdeque.h"
struct sc_hid_event {
uint16_t accessory_id;
@@ -27,7 +27,7 @@ sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
void
sc_hid_event_destroy(struct sc_hid_event *hid_event);
struct sc_hid_event_queue CBUF(struct sc_hid_event, 64);
struct sc_hid_event_queue SC_VECDEQUE(struct sc_hid_event);
struct sc_aoa {
struct sc_usb *usb;

26
app/src/util/average.c Normal file
View File

@@ -0,0 +1,26 @@
#include "average.h"
#include <assert.h>
void
sc_average_init(struct sc_average *avg, unsigned range) {
avg->range = range;
avg->avg = 0;
avg->count = 0;
}
void
sc_average_push(struct sc_average *avg, float value) {
if (avg->count < avg->range) {
++avg->count;
}
assert(avg->count);
avg->avg = ((avg->count - 1) * avg->avg + value) / avg->count;
}
float
sc_average_get(struct sc_average *avg) {
assert(avg->count);
return avg->avg;
}

40
app/src/util/average.h Normal file
View File

@@ -0,0 +1,40 @@
#ifndef SC_AVERAGE
#define SC_AVERAGE
#include "common.h"
#include <stdbool.h>
#include <stdint.h>
struct sc_average {
// Current average value
float avg;
// Target range, to update the average as follow:
// avg = ((range - 1) * avg + new_value) / range
unsigned range;
// Number of values pushed when less than range (count <= range).
// The purpose is to handle the first (range - 1) values properly.
unsigned count;
};
void
sc_average_init(struct sc_average *avg, unsigned range);
/**
* Push a new value to update the "rolling" average
*/
void
sc_average_push(struct sc_average *avg, float value);
/**
* Get the current average value
*
* It is an error to call this function if sc_average_push() has not been
* called at least once.
*/
float
sc_average_get(struct sc_average *avg);
#endif

105
app/src/util/bytebuf.c Normal file
View File

@@ -0,0 +1,105 @@
#include "bytebuf.h"
#include <assert.h>
#include <stdlib.h>
#include <string.h>
#include "util/log.h"
bool
sc_bytebuf_init(struct sc_bytebuf *buf, size_t alloc_size) {
assert(alloc_size);
// sufficient, but use more for alignment.
buf->data = malloc(alloc_size);
if (!buf->data) {
LOG_OOM();
return false;
}
buf->alloc_size = alloc_size;
buf->head = 0;
buf->tail = 0;
return true;
}
void
sc_bytebuf_destroy(struct sc_bytebuf *buf) {
free(buf->data);
}
void
sc_bytebuf_read(struct sc_bytebuf *buf, uint8_t *to, size_t len) {
assert(len);
assert(len <= sc_bytebuf_read_available(buf));
assert(buf->tail != buf->head); // the buffer could not be empty
size_t right_limit = buf->tail < buf->head ? buf->head : buf->alloc_size;
size_t right_len = right_limit - buf->tail;
if (len < right_len) {
right_len = len;
}
memcpy(to, buf->data + buf->tail, right_len);
if (len > right_len) {
memcpy(to + right_len, buf->data, len - right_len);
}
buf->tail = (buf->tail + len) % buf->alloc_size;
}
void
sc_bytebuf_skip(struct sc_bytebuf *buf, size_t len) {
assert(len);
assert(len <= sc_bytebuf_read_available(buf));
assert(buf->tail != buf->head); // the buffer could not be empty
buf->tail = (buf->tail + len) % buf->alloc_size;
}
static inline void
sc_bytebuf_write_step0(struct sc_bytebuf *buf, const uint8_t *from,
size_t len) {
size_t right_len = buf->alloc_size - buf->head;
if (len < right_len) {
right_len = len;
}
memcpy(buf->data + buf->head, from, right_len);
if (len > right_len) {
memcpy(buf->data, from + right_len, len - right_len);
}
}
static inline void
sc_bytebuf_write_step1(struct sc_bytebuf *buf, size_t len) {
buf->head = (buf->head + len) % buf->alloc_size;
}
void
sc_bytebuf_write(struct sc_bytebuf *buf, const uint8_t *from, size_t len) {
assert(len);
assert(len <= sc_bytebuf_write_available(buf));
sc_bytebuf_write_step0(buf, from, len);
sc_bytebuf_write_step1(buf, len);
}
void
sc_bytebuf_prepare_write(struct sc_bytebuf *buf, const uint8_t *from,
size_t len) {
// *This function MUST NOT access buf->tail (even in assert()).*
// The purpose of this function is to allow a reader and a writer to access
// different parts of the buffer in parallel simultaneously. It is intended
// to be called without lock (only sc_bytebuf_commit_write() is intended to
// be called with lock held).
assert(len < buf->alloc_size - 1);
sc_bytebuf_write_step0(buf, from, len);
}
void
sc_bytebuf_commit_write(struct sc_bytebuf *buf, size_t len) {
assert(len <= sc_bytebuf_write_available(buf));
sc_bytebuf_write_step1(buf, len);
}

116
app/src/util/bytebuf.h Normal file
View File

@@ -0,0 +1,116 @@
#ifndef SC_BYTEBUF_H
#define SC_BYTEBUF_H
#include "common.h"
#include <stdbool.h>
#include <stdint.h>
struct sc_bytebuf {
uint8_t *data;
// The actual capacity is (allocated - 1) so that head == tail is
// non-ambiguous
size_t alloc_size;
size_t head; // writter cursor
size_t tail; // reader cursor
// empty: tail == head
// full: (tail + 1) % allocated == head
};
bool
sc_bytebuf_init(struct sc_bytebuf *buf, size_t alloc_size);
/**
* Copy from the bytebuf to a user-provided array
*
* The caller must check that len <= sc_bytebuf_read_available() (it is an
* error to attempt to read more bytes than available).
*
* This function is guaranteed not to write to buf->head.
*/
void
sc_bytebuf_read(struct sc_bytebuf *buf, uint8_t *to, size_t len);
/**
* Drop len bytes from the buffer
*
* The caller must check that len <= sc_bytebuf_read_available() (it is an
* error to attempt to skip more bytes than available).
*
* This function is guaranteed not to change the head.
*
* This function is guaranteed to not change the head.
*
* It is equivalent to call sc_bytebuf_read() to some array and discard the
* array (but more efficient since there is no copy).
*/
void
sc_bytebuf_skip(struct sc_bytebuf *buf, size_t len);
/**
* Copy the user-provided array to the bytebuf
*
* The caller must check that len <= sc_bytebuf_write_available() (it is an
* error to write more bytes than the remaining available space).
*
* This function is guaranteed not to write to buf->tail.
*/
void
sc_bytebuf_write(struct sc_bytebuf *buf, const uint8_t *from, size_t len);
/**
* Copy the user-provided array to the bytebuf, but do not advance the cursor
*
* The caller must check that len <= sc_bytebuf_write_available() (it is an
* error to write more bytes than the remaining available space).
*
* After this function is called, the write must be committed with
* sc_bytebuf_commit_write().
*
* The purpose of this mechanism is to acquire a lock only to commit the write,
* but not to perform the actual copy.
*
* This function is guaranteed not to access buf->tail.
*/
void
sc_bytebuf_prepare_write(struct sc_bytebuf *buf, const uint8_t *from,
size_t len);
/**
* Commit a prepared write
*/
void
sc_bytebuf_commit_write(struct sc_bytebuf *buf, size_t len);
/**
* Return the number of bytes which can be read
*
* It is an error to read more bytes than available.
*/
static inline size_t
sc_bytebuf_read_available(struct sc_bytebuf *buf) {
return (buf->alloc_size + buf->head - buf->tail) % buf->alloc_size;
}
/**
* Return the number of bytes which can be written
*
* It is an error to write more bytes than available.
*/
static inline size_t
sc_bytebuf_write_available(struct sc_bytebuf *buf) {
return (buf->alloc_size + buf->tail - buf->head - 1) % buf->alloc_size;
}
/**
* Return the actual capacity of the buffer (read available + write available)
*/
static inline size_t
sc_bytebuf_capacity(struct sc_bytebuf *buf) {
return buf->alloc_size - 1;
}
void
sc_bytebuf_destroy(struct sc_bytebuf *buf);
#endif

View File

@@ -1,52 +0,0 @@
// generic circular buffer (bounded queue) implementation
#ifndef SC_CBUF_H
#define SC_CBUF_H
#include "common.h"
#include <stdbool.h>
#include <unistd.h>
// To define a circular buffer type of 20 ints:
// struct cbuf_int CBUF(int, 20);
//
// data has length CAP + 1 to distinguish empty vs full.
#define CBUF(TYPE, CAP) { \
TYPE data[(CAP) + 1]; \
size_t head; \
size_t tail; \
}
#define cbuf_size_(PCBUF) \
(sizeof((PCBUF)->data) / sizeof(*(PCBUF)->data))
#define cbuf_is_empty(PCBUF) \
((PCBUF)->head == (PCBUF)->tail)
#define cbuf_is_full(PCBUF) \
(((PCBUF)->head + 1) % cbuf_size_(PCBUF) == (PCBUF)->tail)
#define cbuf_init(PCBUF) \
(void) ((PCBUF)->head = (PCBUF)->tail = 0)
#define cbuf_push(PCBUF, ITEM) \
({ \
bool ok = !cbuf_is_full(PCBUF); \
if (ok) { \
(PCBUF)->data[(PCBUF)->head] = (ITEM); \
(PCBUF)->head = ((PCBUF)->head + 1) % cbuf_size_(PCBUF); \
} \
ok; \
})
#define cbuf_take(PCBUF, PITEM) \
({ \
bool ok = !cbuf_is_empty(PCBUF); \
if (ok) { \
*(PITEM) = (PCBUF)->data[(PCBUF)->tail]; \
(PCBUF)->tail = ((PCBUF)->tail + 1) % cbuf_size_(PCBUF); \
} \
ok; \
})
#endif

14
app/src/util/memory.c Normal file
View File

@@ -0,0 +1,14 @@
#include "memory.h"
#include <stdlib.h>
#include <errno.h>
void *
sc_allocarray(size_t nmemb, size_t size) {
size_t bytes;
if (__builtin_mul_overflow(nmemb, size, &bytes)) {
errno = ENOMEM;
return NULL;
}
return malloc(bytes);
}

12
app/src/util/memory.h Normal file
View File

@@ -0,0 +1,12 @@
#ifndef SC_MEMORY_H
#define SC_MEMORY_H
#include <stddef.h>
/* Like calloc(), but without initialization.
* Like reallocarray(), but without reallocation.
*/
void *
sc_allocarray(size_t nmemb, size_t size);
#endif

View File

@@ -1,77 +0,0 @@
// generic intrusive FIFO queue
#ifndef SC_QUEUE_H
#define SC_QUEUE_H
#include "common.h"
#include <assert.h>
#include <stdbool.h>
#include <stddef.h>
// To define a queue type of "struct foo":
// struct queue_foo QUEUE(struct foo);
#define SC_QUEUE(TYPE) { \
TYPE *first; \
TYPE *last; \
}
#define sc_queue_init(PQ) \
(void) ((PQ)->first = (PQ)->last = NULL)
#define sc_queue_is_empty(PQ) \
!(PQ)->first
// NEXTFIELD is the field in the ITEM type used for intrusive linked-list
//
// For example:
// struct foo {
// int value;
// struct foo *next;
// };
//
// // define the type "struct my_queue"
// struct my_queue SC_QUEUE(struct foo);
//
// struct my_queue queue;
// sc_queue_init(&queue);
//
// struct foo v1 = { .value = 42 };
// struct foo v2 = { .value = 27 };
//
// sc_queue_push(&queue, next, v1);
// sc_queue_push(&queue, next, v2);
//
// struct foo *foo;
// sc_queue_take(&queue, next, &foo);
// assert(foo->value == 42);
// sc_queue_take(&queue, next, &foo);
// assert(foo->value == 27);
// assert(sc_queue_is_empty(&queue));
//
// push a new item into the queue
#define sc_queue_push(PQ, NEXTFIELD, ITEM) \
(void) ({ \
(ITEM)->NEXTFIELD = NULL; \
if (sc_queue_is_empty(PQ)) { \
(PQ)->first = (PQ)->last = (ITEM); \
} else { \
(PQ)->last->NEXTFIELD = (ITEM); \
(PQ)->last = (ITEM); \
} \
})
// take the next item and remove it from the queue (the queue must not be empty)
// the result is stored in *(PITEM)
// (without typeof(), we could not store a local variable having the correct
// type so that we can "return" it)
#define sc_queue_take(PQ, NEXTFIELD, PITEM) \
(void) ({ \
assert(!sc_queue_is_empty(PQ)); \
*(PITEM) = (PQ)->first; \
(PQ)->first = (PQ)->first->NEXTFIELD; \
})
// no need to update (PQ)->last if the queue is left empty:
// (PQ)->last is undefined if !(PQ)->first anyway
#endif

379
app/src/util/vecdeque.h Normal file
View File

@@ -0,0 +1,379 @@
#ifndef SC_VECDEQUE_H
#define SC_VECDEQUE_H
#include "common.h"
#include <assert.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdlib.h>
#include <string.h>
#include "util/memory.h"
/**
* A double-ended queue implemented with a growable ring buffer.
*
* Inspired from the Rust VecDeque type:
* <https://doc.rust-lang.org/std/collections/struct.VecDeque.html>
*/
/**
* VecDeque struct body
*
* A VecDeque is a dynamic ring-buffer, managed by the sc_vecdeque_* helpers.
*
* It is generic over the type of its items, so it is implemented via macros.
*
* To use a VecDeque, a new type must be defined:
*
* struct vecdeque_int SC_VECDEQUE(int);
*
* The struct may be anonymous:
*
* struct SC_VECDEQUE(const char *) names;
*
* Functions and macros having name ending with '_' are private.
*/
#define SC_VECDEQUE(type) { \
size_t cap; \
size_t origin; \
size_t size; \
type *data; \
}
/**
* Static initializer for a VecDeque
*/
#define SC_VECDEQUE_INITIALIZER { 0, 0, 0, NULL }
/**
* Initialize an empty VecDeque
*/
#define sc_vecdeque_init(pv) \
({ \
(pv)->data = NULL; \
(pv)->cap = 0; \
(pv)->origin = 0; \
(pv)->size = 0; \
})
/**
* Destroy a VecDeque
*/
#define sc_vecdeque_destroy(pv) \
free((pv)->data)
/**
* Clear a VecDeque
*
* Remove all items.
*/
#define sc_vecdeque_clear(pv) \
(void) ({ \
sc_vecdeque_destroy(pv); \
sc_vecdeque_init(pv); \
})
/**
* Returns the content size
*/
#define sc_vecdeque_size(pv) \
(pv)->size
/**
* Return whether the VecDeque is empty (i.e. its size is 0)
*/
#define sc_vecdeque_is_empty(pv) \
((pv)->size == 0)
/**
* Return whether the VecDeque is full
*
* A VecDeque is full when its size equals its current capacity. However, it
* does not prevent to push a new item (with sc_vecdeque_push()), since this
* will increase its capacity.
*/
#define sc_vecdeque_is_full(pv) \
((pv)->size == (pv)->cap)
/**
* The minimal allocation size, in number of items
*
* Private.
*/
#define SC_VECDEQUE_MINCAP_ ((size_t) 10)
/**
* The maximal allocation size, in number of items
*
* Use SIZE_MAX/2 to fit in ssize_t, and so that cap*1.5 does not overflow.
*
* Private.
*/
#define sc_vecdeque_max_cap_(pv) (SIZE_MAX / 2 / sizeof(*(pv)->data))
/**
* Realloc the internal array to a specific capacity
*
* On reallocation success, update the VecDeque capacity (`*pcap`) and origin
* (`*porigin`), and return the reallocated data.
*
* On reallocation failure, return NULL without any change.
*
* Private.
*
* \param ptr the current `data` field of the SC_VECDEQUE to realloc
* \param newcap the requested capacity, in number of items
* \param item_size the size of one item (the generic type is unknown from this
* function)
* \param pcap a pointer to the `cap` field of the SC_VECDEQUE [IN/OUT]
* \param porigin a pointer to pv->origin (will be read and updated)
* \param size the `size` field of the SC_VECDEQUE
* \return the new array to assign to the `data` field of the SC_VECDEQUE (if
* not NULL)
*/
static inline void *
sc_vecdeque_reallocdata_(void *ptr, size_t newcap, size_t item_size,
size_t *pcap, size_t *porigin, size_t size) {
size_t oldcap = *pcap;
size_t oldorigin = *porigin;
assert(newcap > oldcap); // Could only grow
if (oldorigin + size <= oldcap) {
// The current content will stay in place, just realloc
//
// As an example, here is the content of a ring-buffer (oldcap=10)
// before the realloc:
//
// _ _ 2 3 4 5 6 7 _ _
// ^
// origin
//
// It is resized (newcap=15), e.g. with sc_vecdeque_reserve():
//
// _ _ 2 3 4 5 6 7 _ _ _ _ _ _ _
// ^
// origin
void *newptr = reallocarray(ptr, newcap, item_size);
if (!newptr) {
return NULL;
}
*pcap = newcap;
return newptr;
}
// Copy the current content to the new array
//
// As an example, here is the content of a ring-buffer (oldcap=10) before
// the realloc:
//
// 5 6 7 _ _ 0 1 2 3 4
// ^
// origin
//
// It is resized (newcap=15), e.g. with sc_vecdeque_reserve():
//
// 0 1 2 3 4 5 6 7 _ _ _ _ _ _ _
// ^
// origin
assert(size);
void *newptr = sc_allocarray(newcap, item_size);
if (!newptr) {
return NULL;
}
size_t right_len = MIN(size, oldcap - oldorigin);
assert(right_len);
memcpy(newptr, ptr + (oldorigin * item_size), right_len * item_size);
if (size > right_len) {
memcpy(newptr + (right_len * item_size), ptr,
(size - right_len) * item_size);
}
free(ptr);
*pcap = newcap;
*porigin = 0;
return newptr;
}
/**
* Macro to realloc the internal data to a new capacity
*
* Private.
*
* \retval true on success
* \retval false on allocation failure (the VecDeque is left untouched)
*/
#define sc_vecdeque_realloc_(pv, newcap) \
({ \
void *p = sc_vecdeque_reallocdata_((pv)->data, newcap, \
sizeof(*(pv)->data), &(pv)->cap, \
&(pv)->origin, (pv)->size); \
if (p) { \
(pv)->data = p; \
} \
(bool) p; \
});
static inline size_t
sc_vecdeque_growsize_(size_t value)
{
/* integer multiplication by 1.5 */
return value + (value >> 1);
}
/**
* Increase the capacity of the VecDeque to at least `mincap`
*
* \param pv a pointer to the VecDeque
* \param mincap (`size_t`) the requested capacity
* \retval true on success
* \retval false on allocation failure (the VecDeque is left untouched)
*/
#define sc_vecdeque_reserve(pv, mincap) \
({ \
assert(mincap <= sc_vecdeque_max_cap_(pv)); \
bool ok; \
/* avoid to allocate tiny arrays (< SC_VECDEQUE_MINCAP_) */ \
size_t mincap_ = MAX(mincap, SC_VECDEQUE_MINCAP_); \
if (mincap_ <= (pv)->cap) { \
/* nothing to do */ \
ok = true; \
} else if (mincap_ <= sc_vecdeque_max_cap_(pv)) { \
/* not too big */ \
size_t newsize = sc_vecdeque_growsize_((pv)->cap); \
newsize = CLAMP(newsize, mincap_, sc_vecdeque_max_cap_(pv)); \
ok = sc_vecdeque_realloc_(pv, newsize); \
} else { \
ok = false; \
} \
ok; \
})
/**
* Automatically grow the VecDeque capacity
*
* Private.
*
* \retval true on success
* \retval false on allocation failure (the VecDeque is left untouched)
*/
#define sc_vecdeque_grow_(pv) \
({ \
bool ok; \
if ((pv)->cap < sc_vecdeque_max_cap_(pv)) { \
size_t newsize = sc_vecdeque_growsize_((pv)->cap); \
newsize = CLAMP(newsize, SC_VECDEQUE_MINCAP_, \
sc_vecdeque_max_cap_(pv)); \
ok = sc_vecdeque_realloc_(pv, newsize); \
} else { \
ok = false; \
} \
ok; \
})
/**
* Grow the VecDeque capacity if it is full
*
* Private.
*
* \retval true on success
* \retval false on allocation failure (the VecDeque is left untouched)
*/
#define sc_vecdeque_grow_if_needed_(pv) \
(!sc_vecdeque_is_full(pv) || sc_vecdeque_grow_(pv))
/**
* Push an uninitialized item, and return a pointer to it
*
* It does not attempt to resize the VecDeque. It is an error to this function
* if the VecDeque is full.
*
* This function may not fail. It returns a valid non-NULL pointer to the
* uninitialized item just pushed.
*/
#define sc_vecdeque_push_hole_noresize(pv) \
({ \
assert(!sc_vecdeque_is_full(pv)); \
++(pv)->size; \
&(pv)->data[((pv)->origin + (pv)->size - 1) % (pv)->cap]; \
})
/**
* Push an uninitialized item, and return a pointer to it
*
* If the VecDeque is full, it is resized.
*
* This function returns either a valid non-nULL pointer to the uninitialized
* item just pushed, or NULL on reallocation failure.
*/
#define sc_vecdeque_push_hole(pv) \
(sc_vecdeque_grow_if_needed_(pv) ? \
sc_vecdeque_push_hole_noresize(pv) : NULL)
/**
* Push an item
*
* It does not attempt to resize the VecDeque. It is an error to this function
* if the VecDeque is full.
*
* This function may not fail.
*/
#define sc_vecdeque_push_noresize(pv, item) \
(void) ({ \
assert(!sc_vecdeque_is_full(pv)); \
++(pv)->size; \
(pv)->data[((pv)->origin + (pv)->size - 1) % (pv)->cap] = item; \
})
/**
* Push an item
*
* If the VecDeque is full, it is resized.
*
* \retval true on success
* \retval false on allocation failure (the VecDeque is left untouched)
*/
#define sc_vecdeque_push(pv, item) \
({ \
bool ok = sc_vecdeque_grow_if_needed_(pv); \
if (ok) { \
sc_vecdeque_push_noresize(pv, item); \
} \
ok; \
})
/**
* Pop an item and return a pointer to it (still in the VecDeque)
*
* Returning a pointer allows the caller to destroy it in place without copy
* (especially if the item type is big).
*
* It is an error to call this function if the VecDeque is empty.
*/
#define sc_vecdeque_popref(pv) \
({ \
assert(!sc_vecdeque_is_empty(pv)); \
size_t pos = (pv)->origin; \
(pv)->origin = ((pv)->origin + 1) % (pv)->cap; \
--(pv)->size; \
&(pv)->data[pos]; \
})
/**
* Pop an item and returns it
*
* It is an error to call this function if the VecDeque is empty.
*/
#define sc_vecdeque_pop(pv) \
(*sc_vecdeque_popref(pv))
#endif

View File

@@ -118,7 +118,7 @@ static inline void *
sc_vector_reallocdata_(void *ptr, size_t count, size_t size,
size_t *restrict pcap, size_t *restrict psize)
{
void *p = realloc(ptr, count * size);
void *p = reallocarray(ptr, count, size);
if (!p) {
return NULL;
}

View File

@@ -126,7 +126,7 @@ run_v4l2_sink(void *data) {
vs->has_frame = false;
sc_mutex_unlock(&vs->mutex);
sc_video_buffer_consume(&vs->vb, vs->frame);
sc_frame_buffer_consume(&vs->fb, vs->frame);
bool ok = encode_and_write_frame(vs, vs->frame);
av_frame_unref(vs->frame);
@@ -141,39 +141,19 @@ run_v4l2_sink(void *data) {
return 0;
}
static void
sc_video_buffer_on_new_frame(struct sc_video_buffer *vb, bool previous_skipped,
void *userdata) {
(void) vb;
struct sc_v4l2_sink *vs = userdata;
if (!previous_skipped) {
sc_mutex_lock(&vs->mutex);
vs->has_frame = true;
sc_cond_signal(&vs->cond);
sc_mutex_unlock(&vs->mutex);
}
}
static bool
sc_v4l2_sink_open(struct sc_v4l2_sink *vs) {
static const struct sc_video_buffer_callbacks cbs = {
.on_new_frame = sc_video_buffer_on_new_frame,
};
sc_v4l2_sink_open(struct sc_v4l2_sink *vs, const AVCodecContext *ctx) {
assert(ctx->pix_fmt == AV_PIX_FMT_YUV420P);
(void) ctx;
bool ok = sc_video_buffer_init(&vs->vb, vs->buffering_time, &cbs, vs);
bool ok = sc_frame_buffer_init(&vs->fb);
if (!ok) {
return false;
}
ok = sc_video_buffer_start(&vs->vb);
if (!ok) {
goto error_video_buffer_destroy;
}
ok = sc_mutex_init(&vs->mutex);
if (!ok) {
goto error_video_buffer_stop_and_join;
goto error_frame_buffer_destroy;
}
ok = sc_cond_init(&vs->cond);
@@ -298,11 +278,8 @@ error_cond_destroy:
sc_cond_destroy(&vs->cond);
error_mutex_destroy:
sc_mutex_destroy(&vs->mutex);
error_video_buffer_stop_and_join:
sc_video_buffer_stop(&vs->vb);
sc_video_buffer_join(&vs->vb);
error_video_buffer_destroy:
sc_video_buffer_destroy(&vs->vb);
error_frame_buffer_destroy:
sc_frame_buffer_destroy(&vs->fb);
return false;
}
@@ -314,10 +291,7 @@ sc_v4l2_sink_close(struct sc_v4l2_sink *vs) {
sc_cond_signal(&vs->cond);
sc_mutex_unlock(&vs->mutex);
sc_video_buffer_stop(&vs->vb);
sc_thread_join(&vs->thread, NULL);
sc_video_buffer_join(&vs->vb);
av_packet_free(&vs->packet);
av_frame_free(&vs->frame);
@@ -327,18 +301,31 @@ sc_v4l2_sink_close(struct sc_v4l2_sink *vs) {
avformat_free_context(vs->format_ctx);
sc_cond_destroy(&vs->cond);
sc_mutex_destroy(&vs->mutex);
sc_video_buffer_destroy(&vs->vb);
sc_frame_buffer_destroy(&vs->fb);
}
static bool
sc_v4l2_sink_push(struct sc_v4l2_sink *vs, const AVFrame *frame) {
return sc_video_buffer_push(&vs->vb, frame);
bool previous_skipped;
bool ok = sc_frame_buffer_push(&vs->fb, frame, &previous_skipped);
if (!ok) {
return false;
}
if (!previous_skipped) {
sc_mutex_lock(&vs->mutex);
vs->has_frame = true;
sc_cond_signal(&vs->cond);
sc_mutex_unlock(&vs->mutex);
}
return true;
}
static bool
sc_v4l2_frame_sink_open(struct sc_frame_sink *sink) {
sc_v4l2_frame_sink_open(struct sc_frame_sink *sink, const AVCodecContext *ctx) {
struct sc_v4l2_sink *vs = DOWNCAST(sink);
return sc_v4l2_sink_open(vs);
return sc_v4l2_sink_open(vs, ctx);
}
static void
@@ -355,7 +342,7 @@ sc_v4l2_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
bool
sc_v4l2_sink_init(struct sc_v4l2_sink *vs, const char *device_name,
struct sc_size frame_size, sc_tick buffering_time) {
struct sc_size frame_size) {
vs->device_name = strdup(device_name);
if (!vs->device_name) {
LOGE("Could not strdup v4l2 device name");
@@ -363,7 +350,6 @@ sc_v4l2_sink_init(struct sc_v4l2_sink *vs, const char *device_name,
}
vs->frame_size = frame_size;
vs->buffering_time = buffering_time;
static const struct sc_frame_sink_ops ops = {
.open = sc_v4l2_frame_sink_open,

View File

@@ -8,19 +8,18 @@
#include "coords.h"
#include "trait/frame_sink.h"
#include "video_buffer.h"
#include "frame_buffer.h"
#include "util/tick.h"
struct sc_v4l2_sink {
struct sc_frame_sink frame_sink; // frame sink trait
struct sc_video_buffer vb;
struct sc_frame_buffer fb;
AVFormatContext *format_ctx;
AVCodecContext *encoder_ctx;
char *device_name;
struct sc_size frame_size;
sc_tick buffering_time;
sc_thread thread;
sc_mutex mutex;
@@ -35,7 +34,7 @@ struct sc_v4l2_sink {
bool
sc_v4l2_sink_init(struct sc_v4l2_sink *vs, const char *device_name,
struct sc_size frame_size, sc_tick buffering_time);
struct sc_size frame_size);
void
sc_v4l2_sink_destroy(struct sc_v4l2_sink *vs);

View File

@@ -1,254 +0,0 @@
#include "video_buffer.h"
#include <assert.h>
#include <stdlib.h>
#include <libavutil/avutil.h>
#include <libavformat/avformat.h>
#include "util/log.h"
#define SC_BUFFERING_NDEBUG // comment to debug
static struct sc_video_buffer_frame *
sc_video_buffer_frame_new(const AVFrame *frame) {
struct sc_video_buffer_frame *vb_frame = malloc(sizeof(*vb_frame));
if (!vb_frame) {
LOG_OOM();
return NULL;
}
vb_frame->frame = av_frame_alloc();
if (!vb_frame->frame) {
LOG_OOM();
free(vb_frame);
return NULL;
}
if (av_frame_ref(vb_frame->frame, frame)) {
av_frame_free(&vb_frame->frame);
free(vb_frame);
return NULL;
}
return vb_frame;
}
static void
sc_video_buffer_frame_delete(struct sc_video_buffer_frame *vb_frame) {
av_frame_unref(vb_frame->frame);
av_frame_free(&vb_frame->frame);
free(vb_frame);
}
static bool
sc_video_buffer_offer(struct sc_video_buffer *vb, const AVFrame *frame) {
bool previous_skipped;
bool ok = sc_frame_buffer_push(&vb->fb, frame, &previous_skipped);
if (!ok) {
return false;
}
vb->cbs->on_new_frame(vb, previous_skipped, vb->cbs_userdata);
return true;
}
static int
run_buffering(void *data) {
struct sc_video_buffer *vb = data;
assert(vb->buffering_time > 0);
for (;;) {
sc_mutex_lock(&vb->b.mutex);
while (!vb->b.stopped && sc_queue_is_empty(&vb->b.queue)) {
sc_cond_wait(&vb->b.queue_cond, &vb->b.mutex);
}
if (vb->b.stopped) {
sc_mutex_unlock(&vb->b.mutex);
goto stopped;
}
struct sc_video_buffer_frame *vb_frame;
sc_queue_take(&vb->b.queue, next, &vb_frame);
sc_tick max_deadline = sc_tick_now() + vb->buffering_time;
// PTS (written by the server) are expressed in microseconds
sc_tick pts = SC_TICK_TO_US(vb_frame->frame->pts);
bool timed_out = false;
while (!vb->b.stopped && !timed_out) {
sc_tick deadline = sc_clock_to_system_time(&vb->b.clock, pts)
+ vb->buffering_time;
if (deadline > max_deadline) {
deadline = max_deadline;
}
timed_out =
!sc_cond_timedwait(&vb->b.wait_cond, &vb->b.mutex, deadline);
}
if (vb->b.stopped) {
sc_video_buffer_frame_delete(vb_frame);
sc_mutex_unlock(&vb->b.mutex);
goto stopped;
}
sc_mutex_unlock(&vb->b.mutex);
#ifndef SC_BUFFERING_NDEBUG
LOGD("Buffering: %" PRItick ";%" PRItick ";%" PRItick,
pts, vb_frame->push_date, sc_tick_now());
#endif
sc_video_buffer_offer(vb, vb_frame->frame);
sc_video_buffer_frame_delete(vb_frame);
}
stopped:
// Flush queue
while (!sc_queue_is_empty(&vb->b.queue)) {
struct sc_video_buffer_frame *vb_frame;
sc_queue_take(&vb->b.queue, next, &vb_frame);
sc_video_buffer_frame_delete(vb_frame);
}
LOGD("Buffering thread ended");
return 0;
}
bool
sc_video_buffer_init(struct sc_video_buffer *vb, sc_tick buffering_time,
const struct sc_video_buffer_callbacks *cbs,
void *cbs_userdata) {
bool ok = sc_frame_buffer_init(&vb->fb);
if (!ok) {
return false;
}
assert(buffering_time >= 0);
if (buffering_time) {
ok = sc_mutex_init(&vb->b.mutex);
if (!ok) {
sc_frame_buffer_destroy(&vb->fb);
return false;
}
ok = sc_cond_init(&vb->b.queue_cond);
if (!ok) {
sc_mutex_destroy(&vb->b.mutex);
sc_frame_buffer_destroy(&vb->fb);
return false;
}
ok = sc_cond_init(&vb->b.wait_cond);
if (!ok) {
sc_cond_destroy(&vb->b.queue_cond);
sc_mutex_destroy(&vb->b.mutex);
sc_frame_buffer_destroy(&vb->fb);
return false;
}
sc_clock_init(&vb->b.clock);
sc_queue_init(&vb->b.queue);
}
assert(cbs);
assert(cbs->on_new_frame);
vb->buffering_time = buffering_time;
vb->cbs = cbs;
vb->cbs_userdata = cbs_userdata;
return true;
}
bool
sc_video_buffer_start(struct sc_video_buffer *vb) {
if (vb->buffering_time) {
bool ok =
sc_thread_create(&vb->b.thread, run_buffering, "scrcpy-vbuf", vb);
if (!ok) {
LOGE("Could not start buffering thread");
return false;
}
}
return true;
}
void
sc_video_buffer_stop(struct sc_video_buffer *vb) {
if (vb->buffering_time) {
sc_mutex_lock(&vb->b.mutex);
vb->b.stopped = true;
sc_cond_signal(&vb->b.queue_cond);
sc_cond_signal(&vb->b.wait_cond);
sc_mutex_unlock(&vb->b.mutex);
}
}
void
sc_video_buffer_join(struct sc_video_buffer *vb) {
if (vb->buffering_time) {
sc_thread_join(&vb->b.thread, NULL);
}
}
void
sc_video_buffer_destroy(struct sc_video_buffer *vb) {
sc_frame_buffer_destroy(&vb->fb);
if (vb->buffering_time) {
sc_cond_destroy(&vb->b.wait_cond);
sc_cond_destroy(&vb->b.queue_cond);
sc_mutex_destroy(&vb->b.mutex);
}
}
bool
sc_video_buffer_push(struct sc_video_buffer *vb, const AVFrame *frame) {
if (!vb->buffering_time) {
// No buffering
return sc_video_buffer_offer(vb, frame);
}
sc_mutex_lock(&vb->b.mutex);
sc_tick pts = SC_TICK_FROM_US(frame->pts);
sc_clock_update(&vb->b.clock, sc_tick_now(), pts);
sc_cond_signal(&vb->b.wait_cond);
if (vb->b.clock.count == 1) {
sc_mutex_unlock(&vb->b.mutex);
// First frame, offer it immediately, for two reasons:
// - not to delay the opening of the scrcpy window
// - the buffering estimation needs at least two clock points, so it
// could not handle the first frame
return sc_video_buffer_offer(vb, frame);
}
struct sc_video_buffer_frame *vb_frame = sc_video_buffer_frame_new(frame);
if (!vb_frame) {
sc_mutex_unlock(&vb->b.mutex);
LOG_OOM();
return false;
}
#ifndef SC_BUFFERING_NDEBUG
vb_frame->push_date = sc_tick_now();
#endif
sc_queue_push(&vb->b.queue, next, vb_frame);
sc_cond_signal(&vb->b.queue_cond);
sc_mutex_unlock(&vb->b.mutex);
return true;
}
void
sc_video_buffer_consume(struct sc_video_buffer *vb, AVFrame *dst) {
sc_frame_buffer_consume(&vb->fb, dst);
}

View File

@@ -1,76 +0,0 @@
#ifndef SC_VIDEO_BUFFER_H
#define SC_VIDEO_BUFFER_H
#include "common.h"
#include <stdbool.h>
#include "clock.h"
#include "frame_buffer.h"
#include "util/queue.h"
#include "util/thread.h"
#include "util/tick.h"
// forward declarations
typedef struct AVFrame AVFrame;
struct sc_video_buffer_frame {
AVFrame *frame;
struct sc_video_buffer_frame *next;
#ifndef NDEBUG
sc_tick push_date;
#endif
};
struct sc_video_buffer_frame_queue SC_QUEUE(struct sc_video_buffer_frame);
struct sc_video_buffer {
struct sc_frame_buffer fb;
sc_tick buffering_time;
// only if buffering_time > 0
struct {
sc_thread thread;
sc_mutex mutex;
sc_cond queue_cond;
sc_cond wait_cond;
struct sc_clock clock;
struct sc_video_buffer_frame_queue queue;
bool stopped;
} b; // buffering
const struct sc_video_buffer_callbacks *cbs;
void *cbs_userdata;
};
struct sc_video_buffer_callbacks {
void (*on_new_frame)(struct sc_video_buffer *vb, bool previous_skipped,
void *userdata);
};
bool
sc_video_buffer_init(struct sc_video_buffer *vb, sc_tick buffering_time,
const struct sc_video_buffer_callbacks *cbs,
void *cbs_userdata);
bool
sc_video_buffer_start(struct sc_video_buffer *vb);
void
sc_video_buffer_stop(struct sc_video_buffer *vb);
void
sc_video_buffer_join(struct sc_video_buffer *vb);
void
sc_video_buffer_destroy(struct sc_video_buffer *vb);
bool
sc_video_buffer_push(struct sc_video_buffer *vb, const AVFrame *frame);
void
sc_video_buffer_consume(struct sc_video_buffer *vb, AVFrame *dst);
#endif

126
app/tests/test_bytebuf.c Normal file
View File

@@ -0,0 +1,126 @@
#include "common.h"
#include <assert.h>
#include <string.h>
#include "util/bytebuf.h"
void test_bytebuf_simple(void) {
struct sc_bytebuf buf;
uint8_t data[20];
bool ok = sc_bytebuf_init(&buf, 20);
assert(ok);
sc_bytebuf_write(&buf, (uint8_t *) "hello", sizeof("hello") - 1);
assert(sc_bytebuf_read_available(&buf) == 5);
sc_bytebuf_read(&buf, data, 4);
assert(!strncmp((char *) data, "hell", 4));
sc_bytebuf_write(&buf, (uint8_t *) " world", sizeof(" world") - 1);
assert(sc_bytebuf_read_available(&buf) == 7);
sc_bytebuf_write(&buf, (uint8_t *) "!", 1);
assert(sc_bytebuf_read_available(&buf) == 8);
sc_bytebuf_read(&buf, &data[4], 8);
assert(sc_bytebuf_read_available(&buf) == 0);
data[12] = '\0';
assert(!strcmp((char *) data, "hello world!"));
assert(sc_bytebuf_read_available(&buf) == 0);
sc_bytebuf_destroy(&buf);
}
void test_bytebuf_boundaries(void) {
struct sc_bytebuf buf;
uint8_t data[20];
bool ok = sc_bytebuf_init(&buf, 20);
assert(ok);
sc_bytebuf_write(&buf, (uint8_t *) "hello ", sizeof("hello ") - 1);
assert(sc_bytebuf_read_available(&buf) == 6);
sc_bytebuf_write(&buf, (uint8_t *) "hello ", sizeof("hello ") - 1);
assert(sc_bytebuf_read_available(&buf) == 12);
sc_bytebuf_write(&buf, (uint8_t *) "hello ", sizeof("hello ") - 1);
assert(sc_bytebuf_read_available(&buf) == 18);
sc_bytebuf_read(&buf, data, 9);
assert(!strncmp((char *) data, "hello hel", 9));
assert(sc_bytebuf_read_available(&buf) == 9);
sc_bytebuf_write(&buf, (uint8_t *) "world", sizeof("world") - 1);
assert(sc_bytebuf_read_available(&buf) == 14);
sc_bytebuf_write(&buf, (uint8_t *) "!", 1);
assert(sc_bytebuf_read_available(&buf) == 15);
sc_bytebuf_skip(&buf, 3);
assert(sc_bytebuf_read_available(&buf) == 12);
sc_bytebuf_read(&buf, data, 12);
data[12] = '\0';
assert(!strcmp((char *) data, "hello world!"));
assert(sc_bytebuf_read_available(&buf) == 0);
sc_bytebuf_destroy(&buf);
}
void test_bytebuf_two_steps_write(void) {
struct sc_bytebuf buf;
uint8_t data[20];
bool ok = sc_bytebuf_init(&buf, 20);
assert(ok);
sc_bytebuf_write(&buf, (uint8_t *) "hello ", sizeof("hello ") - 1);
assert(sc_bytebuf_read_available(&buf) == 6);
sc_bytebuf_write(&buf, (uint8_t *) "hello ", sizeof("hello ") - 1);
assert(sc_bytebuf_read_available(&buf) == 12);
sc_bytebuf_prepare_write(&buf, (uint8_t *) "hello ", sizeof("hello ") - 1);
assert(sc_bytebuf_read_available(&buf) == 12); // write not committed yet
sc_bytebuf_read(&buf, data, 9);
assert(!strncmp((char *) data, "hello hel", 3));
assert(sc_bytebuf_read_available(&buf) == 3);
sc_bytebuf_commit_write(&buf, sizeof("hello ") - 1);
assert(sc_bytebuf_read_available(&buf) == 9);
sc_bytebuf_prepare_write(&buf, (uint8_t *) "world", sizeof("world") - 1);
assert(sc_bytebuf_read_available(&buf) == 9); // write not committed yet
sc_bytebuf_commit_write(&buf, sizeof("world") - 1);
assert(sc_bytebuf_read_available(&buf) == 14);
sc_bytebuf_write(&buf, (uint8_t *) "!", 1);
assert(sc_bytebuf_read_available(&buf) == 15);
sc_bytebuf_skip(&buf, 3);
assert(sc_bytebuf_read_available(&buf) == 12);
sc_bytebuf_read(&buf, data, 12);
data[12] = '\0';
assert(!strcmp((char *) data, "hello world!"));
assert(sc_bytebuf_read_available(&buf) == 0);
sc_bytebuf_destroy(&buf);
}
int main(int argc, char *argv[]) {
(void) argc;
(void) argv;
test_bytebuf_simple();
test_bytebuf_boundaries();
test_bytebuf_two_steps_write();
return 0;
}

View File

@@ -1,78 +0,0 @@
#include "common.h"
#include <assert.h>
#include <string.h>
#include "util/cbuf.h"
struct int_queue CBUF(int, 32);
static void test_cbuf_empty(void) {
struct int_queue queue;
cbuf_init(&queue);
assert(cbuf_is_empty(&queue));
bool push_ok = cbuf_push(&queue, 42);
assert(push_ok);
assert(!cbuf_is_empty(&queue));
int item;
bool take_ok = cbuf_take(&queue, &item);
assert(take_ok);
assert(cbuf_is_empty(&queue));
bool take_empty_ok = cbuf_take(&queue, &item);
assert(!take_empty_ok); // the queue is empty
}
static void test_cbuf_full(void) {
struct int_queue queue;
cbuf_init(&queue);
assert(!cbuf_is_full(&queue));
// fill the queue
for (int i = 0; i < 32; ++i) {
bool ok = cbuf_push(&queue, i);
assert(ok);
}
bool ok = cbuf_push(&queue, 42);
assert(!ok); // the queue if full
int item;
bool take_ok = cbuf_take(&queue, &item);
assert(take_ok);
assert(!cbuf_is_full(&queue));
}
static void test_cbuf_push_take(void) {
struct int_queue queue;
cbuf_init(&queue);
bool push1_ok = cbuf_push(&queue, 42);
assert(push1_ok);
bool push2_ok = cbuf_push(&queue, 35);
assert(push2_ok);
int item;
bool take1_ok = cbuf_take(&queue, &item);
assert(take1_ok);
assert(item == 42);
bool take2_ok = cbuf_take(&queue, &item);
assert(take2_ok);
assert(item == 35);
}
int main(int argc, char *argv[]) {
(void) argc;
(void) argv;
test_cbuf_empty();
test_cbuf_full();
test_cbuf_push_take();
return 0;
}

View File

@@ -1,43 +0,0 @@
#include "common.h"
#include <assert.h>
#include "util/queue.h"
struct foo {
int value;
struct foo *next;
};
static void test_queue(void) {
struct my_queue SC_QUEUE(struct foo) queue;
sc_queue_init(&queue);
assert(sc_queue_is_empty(&queue));
struct foo v1 = { .value = 42 };
struct foo v2 = { .value = 27 };
sc_queue_push(&queue, next, &v1);
sc_queue_push(&queue, next, &v2);
struct foo *foo;
assert(!sc_queue_is_empty(&queue));
sc_queue_take(&queue, next, &foo);
assert(foo->value == 42);
assert(!sc_queue_is_empty(&queue));
sc_queue_take(&queue, next, &foo);
assert(foo->value == 27);
assert(sc_queue_is_empty(&queue));
}
int main(int argc, char *argv[]) {
(void) argc;
(void) argv;
test_queue();
return 0;
}

197
app/tests/test_vecdeque.c Normal file
View File

@@ -0,0 +1,197 @@
#include "common.h"
#include <assert.h>
#include "util/vecdeque.h"
#define pr(pv) \
({ \
fprintf(stderr, "cap=%lu origin=%lu size=%lu\n", (pv)->cap, (pv)->origin, (pv)->size); \
for (size_t i = 0; i < (pv)->cap; ++i) \
fprintf(stderr, "%d ", (pv)->data[i]); \
fprintf(stderr, "\n"); \
})
static void test_vecdeque_push_pop(void) {
struct SC_VECDEQUE(int) vdq = SC_VECDEQUE_INITIALIZER;
assert(sc_vecdeque_is_empty(&vdq));
assert(sc_vecdeque_size(&vdq) == 0);
bool ok = sc_vecdeque_push(&vdq, 5);
assert(ok);
assert(sc_vecdeque_size(&vdq) == 1);
ok = sc_vecdeque_push(&vdq, 12);
assert(ok);
assert(sc_vecdeque_size(&vdq) == 2);
int v = sc_vecdeque_pop(&vdq);
assert(v == 5);
assert(sc_vecdeque_size(&vdq) == 1);
ok = sc_vecdeque_push(&vdq, 7);
assert(ok);
assert(sc_vecdeque_size(&vdq) == 2);
int *p = sc_vecdeque_popref(&vdq);
assert(p);
assert(*p == 12);
assert(sc_vecdeque_size(&vdq) == 1);
v = sc_vecdeque_pop(&vdq);
assert(v == 7);
assert(sc_vecdeque_size(&vdq) == 0);
assert(sc_vecdeque_is_empty(&vdq));
sc_vecdeque_destroy(&vdq);
}
static void test_vecdeque_reserve(void) {
struct SC_VECDEQUE(int) vdq = SC_VECDEQUE_INITIALIZER;
bool ok = sc_vecdeque_reserve(&vdq, 20);
assert(ok);
assert(vdq.cap == 20);
assert(sc_vecdeque_size(&vdq) == 0);
for (size_t i = 0; i < 20; ++i) {
ok = sc_vecdeque_push(&vdq, i);
assert(ok);
}
assert(sc_vecdeque_size(&vdq) == 20);
// It is now full
for (int i = 0; i < 5; ++i) {
int v = sc_vecdeque_pop(&vdq);
assert(v == i);
}
assert(sc_vecdeque_size(&vdq) == 15);
for (int i = 20; i < 25; ++i) {
ok = sc_vecdeque_push(&vdq, i);
assert(ok);
}
assert(sc_vecdeque_size(&vdq) == 20);
assert(vdq.cap == 20);
// Now, the content wraps around the ring buffer:
// 20 21 22 23 24 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
// ^
// origin
// It is now full, let's reserve some space
ok = sc_vecdeque_reserve(&vdq, 30);
assert(ok);
assert(vdq.cap == 30);
assert(sc_vecdeque_size(&vdq) == 20);
for (int i = 0; i < 20; ++i) {
// We should retrieve the items we inserted in order
int v = sc_vecdeque_pop(&vdq);
assert(v == i + 5);
}
assert(sc_vecdeque_size(&vdq) == 0);
sc_vecdeque_destroy(&vdq);
}
static void test_vecdeque_grow() {
struct SC_VECDEQUE(int) vdq = SC_VECDEQUE_INITIALIZER;
bool ok = sc_vecdeque_reserve(&vdq, 20);
assert(ok);
assert(vdq.cap == 20);
assert(sc_vecdeque_size(&vdq) == 0);
for (int i = 0; i < 500; ++i) {
ok = sc_vecdeque_push(&vdq, i);
assert(ok);
}
assert(sc_vecdeque_size(&vdq) == 500);
for (int i = 0; i < 100; ++i) {
int v = sc_vecdeque_pop(&vdq);
assert(v == i);
}
assert(sc_vecdeque_size(&vdq) == 400);
for (int i = 500; i < 1000; ++i) {
ok = sc_vecdeque_push(&vdq, i);
assert(ok);
}
assert(sc_vecdeque_size(&vdq) == 900);
for (int i = 100; i < 1000; ++i) {
int v = sc_vecdeque_pop(&vdq);
assert(v == i);
}
assert(sc_vecdeque_size(&vdq) == 0);
sc_vecdeque_destroy(&vdq);
}
static void test_vecdeque_push_hole() {
struct SC_VECDEQUE(int) vdq = SC_VECDEQUE_INITIALIZER;
bool ok = sc_vecdeque_reserve(&vdq, 20);
assert(ok);
assert(vdq.cap == 20);
assert(sc_vecdeque_size(&vdq) == 0);
for (int i = 0; i < 20; ++i) {
int *p = sc_vecdeque_push_hole(&vdq);
assert(p);
*p = i * 10;
}
assert(sc_vecdeque_size(&vdq) == 20);
for (int i = 0; i < 10; ++i) {
int v = sc_vecdeque_pop(&vdq);
assert(v == i * 10);
}
assert(sc_vecdeque_size(&vdq) == 10);
for (int i = 20; i < 30; ++i) {
int *p = sc_vecdeque_push_hole(&vdq);
assert(p);
*p = i * 10;
}
assert(sc_vecdeque_size(&vdq) == 20);
for (int i = 10; i < 30; ++i) {
int v = sc_vecdeque_pop(&vdq);
assert(v == i * 10);
}
assert(sc_vecdeque_size(&vdq) == 0);
sc_vecdeque_destroy(&vdq);
}
int main(int argc, char *argv[]) {
(void) argc;
(void) argv;
test_vecdeque_push_pop();
test_vecdeque_reserve();
test_vecdeque_grow();
test_vecdeque_push_hole();
return 0;
}

View File

@@ -0,0 +1,148 @@
package com.genymobile.scrcpy;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import android.annotation.SuppressLint;
import android.annotation.TargetApi;
import android.content.ComponentName;
import android.content.Intent;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.AudioTimestamp;
import android.media.MediaCodec;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.SystemClock;
import java.io.IOException;
import java.nio.ByteBuffer;
public final class AudioCapture {
public static final int SAMPLE_RATE = 48000;
public static final int CHANNEL_CONFIG = AudioFormat.CHANNEL_IN_STEREO;
public static final int CHANNELS = 2;
public static final int FORMAT = AudioFormat.ENCODING_PCM_16BIT;
public static final int BYTES_PER_SAMPLE = 2;
private AudioRecord recorder;
private final AudioTimestamp timestamp = new AudioTimestamp();
private long previousPts = 0;
private long nextPts = 0;
public static int millisToBytes(int millis) {
return SAMPLE_RATE * CHANNELS * BYTES_PER_SAMPLE * millis / 1000;
}
private static AudioFormat createAudioFormat() {
AudioFormat.Builder builder = new AudioFormat.Builder();
builder.setEncoding(FORMAT);
builder.setSampleRate(SAMPLE_RATE);
builder.setChannelMask(CHANNEL_CONFIG);
return builder.build();
}
@TargetApi(Build.VERSION_CODES.M)
@SuppressLint({"WrongConstant", "MissingPermission"})
private static AudioRecord createAudioRecord() {
AudioRecord.Builder builder = new AudioRecord.Builder();
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
// On older APIs, Workarounds.fillAppInfo() must be called beforehand
builder.setContext(FakeContext.get());
}
builder.setAudioSource(MediaRecorder.AudioSource.REMOTE_SUBMIX);
builder.setAudioFormat(createAudioFormat());
int minBufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, CHANNEL_CONFIG, FORMAT);
// This buffer size does not impact latency
builder.setBufferSizeInBytes(8 * minBufferSize);
return builder.build();
}
private static void startWorkaroundAndroid11() {
if (Build.VERSION.SDK_INT == Build.VERSION_CODES.R) {
// Android 11 requires Apps to be at foreground to record audio.
// Normally, each App has its own user ID, so Android checks whether the requesting App has the user ID that's at the foreground.
// But scrcpy server is NOT an App, it's a Java application started from Android shell, so it has the same user ID (2000) with Android
// shell ("com.android.shell").
// If there is an Activity from Android shell running at foreground, then the permission system will believe scrcpy is also in the
// foreground.
if (Build.VERSION.SDK_INT == Build.VERSION_CODES.R) {
Intent intent = new Intent(Intent.ACTION_MAIN);
intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
intent.addCategory(Intent.CATEGORY_LAUNCHER);
intent.setComponent(new ComponentName(FakeContext.PACKAGE_NAME, "com.android.shell.HeapDumpActivity"));
ServiceManager.getActivityManager().startActivityAsUserWithFeature(intent);
// Wait for activity to start
SystemClock.sleep(150);
}
}
}
private static void stopWorkaroundAndroid11() {
if (Build.VERSION.SDK_INT == Build.VERSION_CODES.R) {
ServiceManager.getActivityManager().forceStopPackage(FakeContext.PACKAGE_NAME);
}
}
public void start() throws AudioCaptureForegroundException {
startWorkaroundAndroid11();
try {
recorder = createAudioRecord();
recorder.startRecording();
} catch (UnsupportedOperationException e) {
if (Build.VERSION.SDK_INT == Build.VERSION_CODES.R) {
Ln.e("Failed to start audio capture");
Ln.e("On Android 11, it is only possible to capture in foreground, make sure that the device is unlocked when starting scrcpy.");
throw new AudioCaptureForegroundException();
}
throw e;
} finally {
stopWorkaroundAndroid11();
}
}
public void stop() {
if (recorder != null) {
// Will call .stop() if necessary, without throwing an IllegalStateException
recorder.release();
}
}
@TargetApi(Build.VERSION_CODES.N)
public int read(ByteBuffer directBuffer, int size, MediaCodec.BufferInfo outBufferInfo) throws IOException {
int r = recorder.read(directBuffer, size);
if (r < 0) {
return r;
}
long pts;
int ret = recorder.getTimestamp(timestamp, AudioTimestamp.TIMEBASE_MONOTONIC);
if (ret == AudioRecord.SUCCESS) {
pts = timestamp.nanoTime / 1000;
} else {
if (nextPts == 0) {
Ln.w("Could not get any audio timestamp");
}
// compute from previous timestamp and packet size
pts = nextPts;
}
long durationMs = r * 1000 / CHANNELS / SAMPLE_RATE;
nextPts = pts + durationMs;
if (previousPts != 0 && pts < previousPts) {
// Audio PTS may come from two sources:
// - recorder.getTimestamp() if the call works;
// - an estimation from the previous PTS and the packet size as a fallback.
//
// Therefore, the property that PTS are monotonically increasing is no guaranteed in corner cases, so enforce it.
pts = previousPts + 1;
}
previousPts = pts;
outBufferInfo.set(0, r, pts, 0);
return r;
}
}

View File

@@ -0,0 +1,7 @@
package com.genymobile.scrcpy;
/**
* Exception thrown if audio capture failed on Android 11 specifically because the running App (shell) was not in foreground.
*/
public class AudioCaptureForegroundException extends Exception {
}

View File

@@ -3,6 +3,7 @@ package com.genymobile.scrcpy;
import android.media.MediaFormat;
public enum AudioCodec implements Codec {
RAW(0x00_72_61_77, "raw", MediaFormat.MIMETYPE_AUDIO_RAW),
OPUS(0x6f_70_75_73, "opus", MediaFormat.MIMETYPE_AUDIO_OPUS),
AAC(0x00_61_61_63, "aac", MediaFormat.MIMETYPE_AUDIO_AAC);

View File

@@ -1,13 +1,8 @@
package com.genymobile.scrcpy;
import android.annotation.SuppressLint;
import android.annotation.TargetApi;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.AudioTimestamp;
import android.media.MediaCodec;
import android.media.MediaFormat;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.Handler;
import android.os.HandlerThread;
@@ -15,10 +10,11 @@ import android.os.Looper;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
public final class AudioEncoder {
public final class AudioEncoder implements AudioRecorder {
private static class InputTask {
private final int index;
@@ -38,17 +34,16 @@ public final class AudioEncoder {
}
}
private static final int SAMPLE_RATE = 48000;
private static final int CHANNEL_CONFIG = AudioFormat.CHANNEL_IN_STEREO;
private static final int CHANNELS = 2;
private static final int FORMAT = AudioFormat.ENCODING_PCM_16BIT;
private static final int BYTES_PER_SAMPLE = 2;
private static final int SAMPLE_RATE = AudioCapture.SAMPLE_RATE;
private static final int CHANNELS = AudioCapture.CHANNELS;
private static final int READ_MS = 5; // milliseconds
private static final int READ_SIZE = SAMPLE_RATE * CHANNELS * BYTES_PER_SAMPLE * READ_MS / 1000;
private static final int READ_SIZE = AudioCapture.millisToBytes(READ_MS);
private final Streamer streamer;
private final int bitRate;
private final List<CodecOption> codecOptions;
private final String encoderName;
// Capacity of 64 is in practice "infinite" (it is limited by the number of available MediaCodec buffers, typically 4).
// So many pending tasks would lead to an unacceptable delay anyway.
@@ -63,86 +58,45 @@ public final class AudioEncoder {
private boolean ended;
public AudioEncoder(Streamer streamer, int bitRate) {
public AudioEncoder(Streamer streamer, int bitRate, List<CodecOption> codecOptions, String encoderName) {
this.streamer = streamer;
this.bitRate = bitRate;
this.codecOptions = codecOptions;
this.encoderName = encoderName;
}
private static AudioFormat createAudioFormat() {
AudioFormat.Builder builder = new AudioFormat.Builder();
builder.setEncoding(FORMAT);
builder.setSampleRate(SAMPLE_RATE);
builder.setChannelMask(CHANNEL_CONFIG);
return builder.build();
}
@TargetApi(Build.VERSION_CODES.M)
@SuppressLint({"WrongConstant", "MissingPermission"})
private static AudioRecord createAudioRecord() {
AudioRecord.Builder builder = new AudioRecord.Builder();
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
// On older APIs, Workarounds.fillAppInfo() must be called beforehand
builder.setContext(FakeContext.get());
}
builder.setAudioSource(MediaRecorder.AudioSource.REMOTE_SUBMIX);
builder.setAudioFormat(createAudioFormat());
int minBufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, CHANNEL_CONFIG, FORMAT);
// This buffer size does not impact latency
builder.setBufferSizeInBytes(8 * minBufferSize);
return builder.build();
}
private static MediaFormat createFormat(String mimeType, int bitRate) {
private static MediaFormat createFormat(String mimeType, int bitRate, List<CodecOption> codecOptions) {
MediaFormat format = new MediaFormat();
format.setString(MediaFormat.KEY_MIME, mimeType);
format.setInteger(MediaFormat.KEY_BIT_RATE, bitRate);
format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, CHANNELS);
format.setInteger(MediaFormat.KEY_SAMPLE_RATE, SAMPLE_RATE);
if (codecOptions != null) {
for (CodecOption option : codecOptions) {
String key = option.getKey();
Object value = option.getValue();
CodecUtils.setCodecOption(format, key, value);
Ln.d("Audio codec option set: " + key + " (" + value.getClass().getSimpleName() + ") = " + value);
}
}
return format;
}
@TargetApi(Build.VERSION_CODES.N)
private void inputThread(MediaCodec mediaCodec, AudioRecord recorder) throws IOException, InterruptedException {
final AudioTimestamp timestamp = new AudioTimestamp();
long previousPts = 0;
long nextPts = 0;
private void inputThread(MediaCodec mediaCodec, AudioCapture capture) throws IOException, InterruptedException {
final MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
while (!Thread.currentThread().isInterrupted()) {
InputTask task = inputTasks.take();
ByteBuffer buffer = mediaCodec.getInputBuffer(task.index);
int r = recorder.read(buffer, READ_SIZE);
int r = capture.read(buffer, READ_SIZE, bufferInfo);
if (r < 0) {
throw new IOException("Could not read audio: " + r);
}
long pts;
int ret = recorder.getTimestamp(timestamp, AudioTimestamp.TIMEBASE_MONOTONIC);
if (ret == AudioRecord.SUCCESS) {
pts = timestamp.nanoTime / 1000;
} else {
if (nextPts == 0) {
Ln.w("Could not get any audio timestamp");
}
// compute from previous timestamp and packet size
pts = nextPts;
}
long durationMs = r * 1000 / CHANNELS / SAMPLE_RATE;
nextPts = pts + durationMs;
if (previousPts != 0 && pts < previousPts) {
// Audio PTS may come from two sources:
// - recorder.getTimestamp() if the call works;
// - an estimation from the previous PTS and the packet size as a fallback.
//
// Therefore, the property that PTS are monotonically increasing is no guaranteed in corner cases, so enforce it.
pts = previousPts + 1;
}
previousPts = pts;
mediaCodec.queueInputBuffer(task.index, 0, r, pts, 0);
mediaCodec.queueInputBuffer(task.index, bufferInfo.offset, bufferInfo.size, bufferInfo.presentationTimeUs, bufferInfo.flags);
}
}
@@ -164,6 +118,8 @@ public final class AudioEncoder {
thread = new Thread(() -> {
try {
encode();
} catch (ConfigurationException | AudioCaptureForegroundException e) {
// Do not print stack trace, a user-friendly error-message has already been logged
} catch (IOException e) {
Ln.e("Audio encoding error", e);
} finally {
@@ -202,38 +158,35 @@ public final class AudioEncoder {
}
@TargetApi(Build.VERSION_CODES.M)
public void encode() throws IOException {
public void encode() throws IOException, ConfigurationException, AudioCaptureForegroundException {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.R) {
Ln.w("Audio disabled: it is not supported before Android 11");
streamer.writeDisableStream();
streamer.writeDisableStream(false);
return;
}
MediaCodec mediaCodec = null;
AudioRecord recorder = null;
AudioCapture capture = new AudioCapture();
boolean mediaCodecStarted = false;
boolean recorderStarted = false;
try {
String mimeType = streamer.getCodec().getMimeType();
mediaCodec = MediaCodec.createEncoderByType(mimeType); // may throw IOException
Codec codec = streamer.getCodec();
mediaCodec = createMediaCodec(codec, encoderName);
mediaCodecThread = new HandlerThread("AudioEncoder");
mediaCodecThread.start();
MediaFormat format = createFormat(mimeType, bitRate);
MediaFormat format = createFormat(codec.getMimeType(), bitRate, codecOptions);
mediaCodec.setCallback(new EncoderCallback(), new Handler(mediaCodecThread.getLooper()));
mediaCodec.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
recorder = createAudioRecord();
recorder.startRecording();
recorderStarted = true;
capture.start();
final MediaCodec mediaCodecRef = mediaCodec;
final AudioRecord recorderRef = recorder;
final AudioCapture captureRef = capture;
inputThread = new Thread(() -> {
try {
inputThread(mediaCodecRef, recorderRef);
inputThread(mediaCodecRef, captureRef);
} catch (IOException | InterruptedException e) {
Ln.e("Audio capture error", e);
} finally {
@@ -262,9 +215,13 @@ public final class AudioEncoder {
outputThread.start();
waitEnded();
} catch (ConfigurationException e) {
// Notify the error to make scrcpy exit
streamer.writeDisableStream(true);
throw e;
} catch (Throwable e) {
// Notify the client that the audio could not be captured
streamer.writeDisableStream();
streamer.writeDisableStream(false);
throw e;
} finally {
// Cleanup everything (either at the end or on error at any step of the initialization)
@@ -302,15 +259,27 @@ public final class AudioEncoder {
}
mediaCodec.release();
}
if (recorder != null) {
if (recorderStarted) {
recorder.stop();
}
recorder.release();
if (capture != null) {
capture.stop();
}
}
}
private static MediaCodec createMediaCodec(Codec codec, String encoderName) throws IOException, ConfigurationException {
if (encoderName != null) {
Ln.d("Creating audio encoder by name: '" + encoderName + "'");
try {
return MediaCodec.createByCodecName(encoderName);
} catch (IllegalArgumentException e) {
Ln.e("Encoder '" + encoderName + "' for " + codec.getName() + " not found\n" + LogUtils.buildAudioEncoderListMessage());
throw new ConfigurationException("Unknown encoder: " + encoderName);
}
}
MediaCodec mediaCodec = MediaCodec.createEncoderByType(codec.getMimeType());
Ln.d("Using audio encoder: '" + mediaCodec.getName() + "'");
return mediaCodec;
}
private class EncoderCallback extends MediaCodec.Callback {
@TargetApi(Build.VERSION_CODES.N)
@Override

View File

@@ -0,0 +1,75 @@
package com.genymobile.scrcpy;
import android.media.MediaCodec;
import java.io.IOException;
import java.nio.ByteBuffer;
public final class AudioRawRecorder implements AudioRecorder {
private final Streamer streamer;
private Thread thread;
private static final int READ_MS = 5; // milliseconds
private static final int READ_SIZE = AudioCapture.millisToBytes(READ_MS);
public AudioRawRecorder(Streamer streamer) {
this.streamer = streamer;
}
private void record() throws IOException, AudioCaptureForegroundException {
final ByteBuffer buffer = ByteBuffer.allocateDirect(READ_SIZE);
final MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
AudioCapture capture = new AudioCapture();
try {
capture.start();
streamer.writeHeader();
while (!Thread.currentThread().isInterrupted()) {
buffer.position(0);
int r = capture.read(buffer, READ_SIZE, bufferInfo);
if (r < 0) {
throw new IOException("Could not read audio: " + r);
}
buffer.limit(r);
streamer.writePacket(buffer, bufferInfo);
}
} catch (Throwable e) {
// Notify the client that the audio could not be captured
streamer.writeDisableStream(false);
throw e;
} finally {
capture.stop();
}
}
public void start() {
thread = new Thread(() -> {
try {
record();
} catch (AudioCaptureForegroundException e) {
// Do not print stack trace, a user-friendly error-message has already been logged
} catch (IOException e) {
Ln.e("Audio recording error", e);
} finally {
Ln.d("Audio recorder stopped");
}
});
thread.start();
}
public void stop() {
if (thread != null) {
thread.interrupt();
}
}
public void join() throws InterruptedException {
if (thread != null) {
thread.join();
}
}
}

View File

@@ -0,0 +1,12 @@
package com.genymobile.scrcpy;
/**
* A component able to record audio asynchronously
*
* The implementation is responsible to send packets.
*/
public interface AudioRecorder {
void start();
void stop();
void join() throws InterruptedException;
}

View File

@@ -139,7 +139,7 @@ public final class CleanUp {
builder.start();
}
private static void unlinkSelf() {
public static void unlinkSelf() {
try {
new File(SERVER_PATH).delete();
} catch (Exception e) {

View File

@@ -10,6 +10,24 @@ import java.util.List;
public final class CodecUtils {
public static final class DeviceEncoder {
private final Codec codec;
private final MediaCodecInfo info;
DeviceEncoder(Codec codec, MediaCodecInfo info) {
this.codec = codec;
this.info = info;
}
public Codec getCodec() {
return codec;
}
public MediaCodecInfo getInfo() {
return info;
}
}
private CodecUtils() {
// not instantiable
}
@@ -25,4 +43,36 @@ public final class CodecUtils {
format.setString(key, (String) value);
}
}
private static MediaCodecInfo[] getEncoders(MediaCodecList codecs, String mimeType) {
List<MediaCodecInfo> result = new ArrayList<>();
for (MediaCodecInfo codecInfo : codecs.getCodecInfos()) {
if (codecInfo.isEncoder() && Arrays.asList(codecInfo.getSupportedTypes()).contains(mimeType)) {
result.add(codecInfo);
}
}
return result.toArray(new MediaCodecInfo[result.size()]);
}
public static List<DeviceEncoder> listVideoEncoders() {
List<DeviceEncoder> encoders = new ArrayList<>();
MediaCodecList codecs = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
for (VideoCodec codec : VideoCodec.values()) {
for (MediaCodecInfo info : getEncoders(codecs, codec.getMimeType())) {
encoders.add(new DeviceEncoder(codec, info));
}
}
return encoders;
}
public static List<DeviceEncoder> listAudioEncoders() {
List<DeviceEncoder> encoders = new ArrayList<>();
MediaCodecList codecs = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
for (AudioCodec codec : AudioCodec.values()) {
for (MediaCodecInfo info : getEncoders(codecs, codec.getMimeType())) {
encoders.add(new DeviceEncoder(codec, info));
}
}
return encoders;
}
}

View File

@@ -65,7 +65,7 @@ public final class Device {
displayId = options.getDisplayId();
DisplayInfo displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(displayId);
if (displayInfo == null) {
Ln.e(buildUnknownDisplayIdMessage(displayId));
Ln.e("Display " + displayId + " not found\n" + LogUtils.buildDisplayListMessage());
throw new ConfigurationException("Unknown display id: " + displayId);
}
@@ -130,18 +130,6 @@ public final class Device {
}
}
private static String buildUnknownDisplayIdMessage(int displayId) {
StringBuilder msg = new StringBuilder("Display ").append(displayId).append(" not found");
int[] displayIds = ServiceManager.getDisplayManager().getDisplayIds();
if (displayIds != null && displayIds.length > 0) {
msg.append("\nTry to use one of the available display ids:");
for (int id : displayIds) {
msg.append("\n scrcpy --display=").append(id);
}
}
return msg.toString();
}
public synchronized void setMaxSize(int newMaxSize) {
maxSize = newMaxSize;
screenInfo = ScreenInfo.computeScreenInfo(screenInfo.getReverseVideoRotation(), deviceSize, crop, newMaxSize, lockVideoOrientation);

View File

@@ -0,0 +1,63 @@
package com.genymobile.scrcpy;
import com.genymobile.scrcpy.wrappers.DisplayManager;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import java.util.List;
public final class LogUtils {
private LogUtils() {
// not instantiable
}
public static String buildVideoEncoderListMessage() {
StringBuilder builder = new StringBuilder("List of video encoders:");
List<CodecUtils.DeviceEncoder> videoEncoders = CodecUtils.listVideoEncoders();
if (videoEncoders.isEmpty()) {
builder.append("\n (none)");
} else {
for (CodecUtils.DeviceEncoder encoder : videoEncoders) {
builder.append("\n --video-codec=").append(encoder.getCodec().getName());
builder.append(" --video-encoder='").append(encoder.getInfo().getName()).append("'");
}
}
return builder.toString();
}
public static String buildAudioEncoderListMessage() {
StringBuilder builder = new StringBuilder("List of audio encoders:");
List<CodecUtils.DeviceEncoder> audioEncoders = CodecUtils.listAudioEncoders();
if (audioEncoders.isEmpty()) {
builder.append("\n (none)");
} else {
for (CodecUtils.DeviceEncoder encoder : audioEncoders) {
builder.append("\n --audio-codec=").append(encoder.getCodec().getName());
builder.append(" --audio-encoder='").append(encoder.getInfo().getName()).append("'");
}
}
return builder.toString();
}
public static String buildDisplayListMessage() {
StringBuilder builder = new StringBuilder("List of displays:");
DisplayManager displayManager = ServiceManager.getDisplayManager();
int[] displayIds = displayManager.getDisplayIds();
if (displayIds == null || displayIds.length == 0) {
builder.append("\n (none)");
} else {
for (int id : displayIds) {
builder.append("\n --display=").append(id).append(" (");
DisplayInfo displayInfo = displayManager.getDisplayInfo(id);
if (displayInfo != null) {
Size size = displayInfo.getSize();
builder.append(size.getWidth()).append("x").append(size.getHeight());
} else {
builder.append("size unknown");
}
builder.append(")");
}
}
return builder.toString();
}
}

View File

@@ -23,13 +23,19 @@ public class Options {
private boolean showTouches;
private boolean stayAwake;
private List<CodecOption> videoCodecOptions;
private List<CodecOption> audioCodecOptions;
private String videoEncoder;
private String audioEncoder;
private boolean powerOffScreenOnClose;
private boolean clipboardAutosync = true;
private boolean downsizeOnError = true;
private boolean cleanup = true;
private boolean powerOn = true;
private boolean listEncoders;
private boolean listDisplays;
// Options not used by the scrcpy client, but useful to use scrcpy-server directly
private boolean sendDeviceMeta = true; // send device name and size
private boolean sendFrameMeta = true; // send PTS so that the client may record properly
@@ -172,6 +178,14 @@ public class Options {
this.videoCodecOptions = videoCodecOptions;
}
public List<CodecOption> getAudioCodecOptions() {
return audioCodecOptions;
}
public void setAudioCodecOptions(List<CodecOption> audioCodecOptions) {
this.audioCodecOptions = audioCodecOptions;
}
public String getVideoEncoder() {
return videoEncoder;
}
@@ -180,6 +194,14 @@ public class Options {
this.videoEncoder = videoEncoder;
}
public String getAudioEncoder() {
return audioEncoder;
}
public void setAudioEncoder(String audioEncoder) {
this.audioEncoder = audioEncoder;
}
public void setPowerOffScreenOnClose(boolean powerOffScreenOnClose) {
this.powerOffScreenOnClose = powerOffScreenOnClose;
}
@@ -220,6 +242,22 @@ public class Options {
this.powerOn = powerOn;
}
public boolean getListEncoders() {
return listEncoders;
}
public void setListEncoders(boolean listEncoders) {
this.listEncoders = listEncoders;
}
public boolean getListDisplays() {
return listDisplays;
}
public void setListDisplays(boolean listDisplays) {
this.listDisplays = listDisplays;
}
public boolean getSendDeviceMeta() {
return sendDeviceMeta;
}

View File

@@ -5,7 +5,6 @@ import com.genymobile.scrcpy.wrappers.SurfaceControl;
import android.graphics.Rect;
import android.media.MediaCodec;
import android.media.MediaCodecInfo;
import android.media.MediaCodecList;
import android.media.MediaFormat;
import android.os.Build;
import android.os.IBinder;
@@ -14,8 +13,6 @@ import android.view.Surface;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.atomic.AtomicBoolean;
@@ -199,24 +196,13 @@ public class ScreenEncoder implements Device.RotationListener {
return !eof;
}
private static MediaCodecInfo[] listEncoders(String videoMimeType) {
List<MediaCodecInfo> result = new ArrayList<>();
MediaCodecList list = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
for (MediaCodecInfo codecInfo : list.getCodecInfos()) {
if (codecInfo.isEncoder() && Arrays.asList(codecInfo.getSupportedTypes()).contains(videoMimeType)) {
result.add(codecInfo);
}
}
return result.toArray(new MediaCodecInfo[result.size()]);
}
private static MediaCodec createMediaCodec(Codec codec, String encoderName) throws IOException, ConfigurationException {
if (encoderName != null) {
Ln.d("Creating encoder by name: '" + encoderName + "'");
try {
return MediaCodec.createByCodecName(encoderName);
} catch (IllegalArgumentException e) {
Ln.e(buildUnknownEncoderMessage(codec, encoderName));
Ln.e("Encoder '" + encoderName + "' for " + codec.getName() + " not found\n" + LogUtils.buildVideoEncoderListMessage());
throw new ConfigurationException("Unknown encoder: " + encoderName);
}
}
@@ -225,18 +211,6 @@ public class ScreenEncoder implements Device.RotationListener {
return mediaCodec;
}
private static String buildUnknownEncoderMessage(Codec codec, String encoderName) {
StringBuilder msg = new StringBuilder("Encoder '").append(encoderName).append("' for ").append(codec.getName()).append(" not found");
MediaCodecInfo[] encoders = listEncoders(codec.getMimeType());
if (encoders != null && encoders.length > 0) {
msg.append("\nTry to use one of the available encoders:");
for (MediaCodecInfo encoder : encoders) {
msg.append("\n scrcpy --video-codec=").append(codec.getName()).append(" --encoder='").append(encoder.getName()).append("'");
}
}
return msg.toString();
}
private static MediaFormat createFormat(String videoMimeType, int bitRate, int maxFps, List<CodecOption> codecOptions) {
MediaFormat format = new MediaFormat();
format.setString(MediaFormat.KEY_MIME, videoMimeType);

View File

@@ -92,7 +92,7 @@ public final class Server {
}
Controller controller = null;
AudioEncoder audioEncoder = null;
AudioRecorder audioRecorder = null;
try (DesktopConnection connection = DesktopConnection.open(scid, tunnelForward, audio, control, sendDummyByte)) {
if (options.getSendDeviceMeta()) {
@@ -109,10 +109,15 @@ public final class Server {
}
if (audio) {
Streamer audioStreamer = new Streamer(connection.getAudioFd(), options.getAudioCodec(), options.getSendCodecId(),
options.getSendFrameMeta());
audioEncoder = new AudioEncoder(audioStreamer, options.getAudioBitRate());
audioEncoder.start();
AudioCodec audioCodec = options.getAudioCodec();
Streamer audioStreamer = new Streamer(connection.getAudioFd(), audioCodec, options.getSendCodecId(), options.getSendFrameMeta());
if (audioCodec == AudioCodec.RAW) {
audioRecorder = new AudioRawRecorder(audioStreamer);
} else {
audioRecorder = new AudioEncoder(audioStreamer, options.getAudioBitRate(), options.getAudioCodecOptions(),
options.getAudioEncoder());
}
audioRecorder.start();
}
Streamer videoStreamer = new Streamer(connection.getVideoFd(), options.getVideoCodec(), options.getSendCodecId(),
@@ -131,8 +136,8 @@ public final class Server {
} finally {
Ln.d("Screen streaming stopped");
initThread.interrupt();
if (audioEncoder != null) {
audioEncoder.stop();
if (audioRecorder != null) {
audioRecorder.stop();
}
if (controller != null) {
controller.stop();
@@ -140,8 +145,8 @@ public final class Server {
try {
initThread.join();
if (audioEncoder != null) {
audioEncoder.join();
if (audioRecorder != null) {
audioRecorder.join();
}
if (controller != null) {
controller.join();
@@ -258,11 +263,19 @@ public final class Server {
List<CodecOption> videoCodecOptions = CodecOption.parse(value);
options.setVideoCodecOptions(videoCodecOptions);
break;
case "audio_codec_options":
List<CodecOption> audioCodecOptions = CodecOption.parse(value);
options.setAudioCodecOptions(audioCodecOptions);
break;
case "video_encoder":
if (!value.isEmpty()) {
options.setVideoEncoder(value);
}
break;
case "audio_encoder":
if (!value.isEmpty()) {
options.setAudioEncoder(value);
}
case "power_off_on_close":
boolean powerOffScreenOnClose = Boolean.parseBoolean(value);
options.setPowerOffScreenOnClose(powerOffScreenOnClose);
@@ -283,6 +296,14 @@ public final class Server {
boolean powerOn = Boolean.parseBoolean(value);
options.setPowerOn(powerOn);
break;
case "list_encoders":
boolean listEncoders = Boolean.parseBoolean(value);
options.setListEncoders(listEncoders);
break;
case "list_displays":
boolean listDisplays = Boolean.parseBoolean(value);
options.setListDisplays(listDisplays);
break;
case "send_device_meta":
boolean sendDeviceMeta = Boolean.parseBoolean(value);
options.setSendDeviceMeta(sendDeviceMeta);
@@ -342,6 +363,22 @@ public final class Server {
Ln.initLogLevel(options.getLogLevel());
if (options.getListEncoders() || options.getListDisplays()) {
if (options.getCleanup()) {
CleanUp.unlinkSelf();
}
if (options.getListEncoders()) {
Ln.i(LogUtils.buildVideoEncoderListMessage());
Ln.i(LogUtils.buildAudioEncoderListMessage());
}
if (options.getListDisplays()) {
Ln.i(LogUtils.buildDisplayListMessage());
}
// Just print the requested data, do not mirror
return;
}
try {
scrcpy(options);
} catch (ConfigurationException e) {

View File

@@ -40,10 +40,15 @@ public final class Streamer {
}
}
public void writeDisableStream() throws IOException {
// Writing 0 (32-bit) as codec-id means that the device disables the stream (because it could not capture)
byte[] zeros = new byte[4];
IO.writeFully(fd, zeros, 0, zeros.length);
public void writeDisableStream(boolean error) throws IOException {
// Writing a specific code as codec-id means that the device disables the stream
// code 0: it explicitly disables the stream (because it could not capture audio), scrcpy should continue mirroring video only
// code 1: a configuration error occurred, scrcpy must be stopped
byte[] code = new byte[4];
if (error) {
code[3] = 1;
}
IO.writeFully(fd, code, 0, code.length);
}
public void writePacket(ByteBuffer buffer, long pts, boolean config, boolean keyFrame) throws IOException {

View File

@@ -3,7 +3,12 @@ package com.genymobile.scrcpy.wrappers;
import com.genymobile.scrcpy.FakeContext;
import com.genymobile.scrcpy.Ln;
import android.annotation.SuppressLint;
import android.annotation.TargetApi;
import android.content.Intent;
import android.os.Binder;
import android.os.Build;
import android.os.Bundle;
import android.os.IBinder;
import android.os.IInterface;
@@ -11,12 +16,15 @@ import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
@SuppressLint("PrivateApi,DiscouragedPrivateApi")
public class ActivityManager {
private final IInterface manager;
private Method getContentProviderExternalMethod;
private boolean getContentProviderExternalMethodNewVersion = true;
private Method removeContentProviderExternalMethod;
private Method startActivityAsUserWithFeatureMethod;
private Method forceStopPackageMethod;
public ActivityManager(IInterface manager) {
this.manager = manager;
@@ -43,6 +51,7 @@ public class ActivityManager {
return removeContentProviderExternalMethod;
}
@TargetApi(Build.VERSION_CODES.Q)
private ContentProvider getContentProviderExternal(String name, IBinder token) {
try {
Method method = getGetContentProviderExternalMethod();
@@ -85,4 +94,55 @@ public class ActivityManager {
public ContentProvider createSettingsProvider() {
return getContentProviderExternal("settings", new Binder());
}
private Method getStartActivityAsUserWithFeatureMethod() throws NoSuchMethodException, ClassNotFoundException {
if (startActivityAsUserWithFeatureMethod == null) {
Class<?> iApplicationThreadClass = Class.forName("android.app.IApplicationThread");
Class<?> profilerInfo = Class.forName("android.app.ProfilerInfo");
startActivityAsUserWithFeatureMethod = manager.getClass()
.getMethod("startActivityAsUserWithFeature", iApplicationThreadClass, String.class, String.class, Intent.class, String.class,
IBinder.class, String.class, int.class, int.class, profilerInfo, Bundle.class, int.class);
}
return startActivityAsUserWithFeatureMethod;
}
@SuppressWarnings("ConstantConditions")
public int startActivityAsUserWithFeature(Intent intent) {
try {
Method method = getStartActivityAsUserWithFeatureMethod();
return (int) method.invoke(
/* this */ manager,
/* caller */ null,
/* callingPackage */ FakeContext.PACKAGE_NAME,
/* callingFeatureId */ null,
/* intent */ intent,
/* resolvedType */ null,
/* resultTo */ null,
/* resultWho */ null,
/* requestCode */ 0,
/* startFlags */ 0,
/* profilerInfo */ null,
/* bOptions */ null,
/* userId */ /* UserHandle.USER_CURRENT */ -2);
} catch (Throwable e) {
Ln.e("Could not invoke method", e);
return 0;
}
}
private Method getForceStopPackageMethod() throws NoSuchMethodException {
if (forceStopPackageMethod == null) {
forceStopPackageMethod = manager.getClass().getMethod("forceStopPackage", String.class, int.class);
}
return forceStopPackageMethod;
}
public void forceStopPackage(String packageName) {
try {
Method method = getForceStopPackageMethod();
method.invoke(manager, packageName, /* userId */ /* UserHandle.USER_CURRENT */ -2);
} catch (Throwable e) {
Ln.e("Could not invoke method", e);
}
}
}