Compare commits

..

118 Commits

Author SHA1 Message Date
Romain Vimont
408f458636 Decrease recorder thread priority
Recording is background task, writing the packets to a file is not
urgent.
2023-03-10 22:22:16 +01:00
Romain Vimont
aa450ffc3f Increase audio thread priority
The audio demuxer thread is the one filling the audio buffer read by the
SDL audio thread. It is time critical to avoid buffer underflow.
2023-03-10 22:22:16 +01:00
Romain Vimont
5ee59e0f13 Add thread priority API
Expose an API to change the priority of the current thread.
2023-03-10 22:22:16 +01:00
Romain Vimont
4a25f3e53b Print info logs to stdout
All server logs were printed to stdout, while all client logs were
printed to stderr.

Instead, use stderr for warnings and errors, stdout for the others:
 - stdout: verbose, debug, info
 - stderr: warn, error
2023-03-10 22:22:15 +01:00
Romain Vimont
bb56472d4e Print server logs and newline in one call
System.out.println() first prints the message, then the new line.
Between these two calls, the client might print a message, breaking
formatting.

Instead, call System.out.print() with '\n' appended to the message.
2023-03-10 22:22:15 +01:00
Romain Vimont
7da45c246e Warn on ignored audio options
For raw audio codec, some audio options are ignored.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
d2952c7e93 Add --audio-codec=raw option
Add support for raw (PCM S16 LE) audio codec (a raw decoder is included
in FFmpeg).

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
66b6c06443 Add raw audio recorder
Add an alternative AudioRecorder to stream raw packets without encoding.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
dc228eaad0 Extract async processor interface
On the server side, several components are started, stopped and joined.
Extract an interface to handle them generically.

This will help to support both encoded and raw audio stream, because
they will be two different concrete components, but implementing the
same interface.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
65cc9d765d Extract audio capture
The audio capture was implemented in AudioEncoder.

In order to reuse it without encoding, extract it to a separate class.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
02dd1be4a1 Stop on decoder frame push error
On push, frame sinks report downstream errors to stop upstream
components. Do not ignore the error.
2023-03-10 22:22:15 +01:00
Romain Vimont
df55bc2683 Add --audio-buffer
Expose an option to add a buffering delay (in milliseconds) before
playing audio.

This is similar to the options --display-buffer and --v4l2-buffer for
video frames.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
d66b0b3dcc Add compat support for FFmpeg < 5.1
The new chlayout API has been introduced in FFmpeg 5.1. Use the old
channel_layout API on older versions.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
fbe0f951e1 Add audio player
Play the decoded audio using SDL.

The audio player frame sink receives the audio frames, resample them
and write them to a byte buffer (introduced by this commit).

On SDL audio callback (from an internal SDL thread), copy samples from
this byte buffer to the SDL audio buffer.

The byte buffer is protected by the SDL_AudioDeviceLock(), but it has
been designed so that the producer and the consumer may write and read
in parallel, provided that they don't access the same slices of the
ring-buffer buffer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-03-10 22:22:15 +01:00
Romain Vimont
e1333f6f3b Optionally do not delay the first frame
A delay buffer delayed all the frames except the first one, to open the
scrcpy window immediately and get a picture.

Make this feature optional, so that the delay buffer might also be used
for audio (especially for simulating a high delay for debugging).

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
9b3ca208bf Accept clock estimation with a single point
If there is only one point, assume the slope is 1.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
48a537d45c Remove anonymous struct in delay buffer
For clarity, the fields used only when a delay was set were wrapped in
an anonymous structure.

Now that the delay buffer has been extracted to a separate component,
the delay is necessarily set (it may not be 0), so the fields are always
used.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
1230149fdd Use delay buffer as a frame source/sink
The components needing delayed frames (sc_screen and sc_v4l2_sink)
managed a sc_video_buffer instance, which itself embedded a
sc_frame_buffer instance (to keep only the most recent frame).

In theory, these components should not be aware of delaying: they should
just receive AVFrames later, and only handle a sc_frame_buffer.

Therefore, refactor sc_delay_buffer as a frame source (it consumes)
frames) and a frame sink (it produces frames, after some delay), and
plug an instance in the pipeline only when a delay is requested.

This also removes the need for a specific sc_video_buffer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
974227a3fc Use frame source trait in decoder 2023-03-10 22:22:15 +01:00
Romain Vimont
6543964f12 Introduce frame source trait
There was a frame sink trait, implemented by components able to receive
AVFrames, but each frame source had to manually send frame to sinks.

In order to mutualise sink management, add a frame sink trait.
2023-03-10 22:22:15 +01:00
Romain Vimont
f3197e178d Use packet source trait in demuxer 2023-03-10 22:22:15 +01:00
Romain Vimont
c39054a63d Introduce packet source trait
There was a packet sink trait, implemented by components able to
receive AVPackets, but each packet source had to manually send packets
to sinks.

In order to mutualise sink management, add a packet source trait.
2023-03-10 22:22:15 +01:00
Romain Vimont
f410f2bdc4 Extract sc_delay_buffer
A video buffer had 2 responsibilities:
 - handle the frame delaying mechanism (queuing packets and pushing them
   after the expected delay);
 - keep only the most recent frame (using a sc_frame_buffer).

In order to be able to reuse only the frame delaying mechanism, extract
it to a separate component, sc_delay_buffer.
2023-03-10 22:22:15 +01:00
Romain Vimont
6379c08012 Fix buffering pts conversion
The mistake had no effect, because tick is also internally expressed in
microseconds.
2023-03-10 22:22:15 +01:00
Romain Vimont
4540f1d69e Report video buffer downstream errors
Make the video buffer stop if its consumer could not receive a frame.
2023-03-10 22:22:15 +01:00
Romain Vimont
ad94ccca0b Stop the video buffer on error
If an error occurs from the video buffer thread (typically an
out-of-memory error), then stop.
2023-03-10 22:22:15 +01:00
Romain Vimont
a3703340fc Fix possible race condition on video_buffer end
The video_buffer thread clears the queue once it is stopped, but new
frames might still be pushed asynchronously.

To avoid the problem, do not push any frame once the video_buffer is
stopped.
2023-03-10 22:22:15 +01:00
Romain Vimont
6f38c6311b Remove sc_queue
All uses have been replaced by VecDeque.
2023-03-10 22:22:15 +01:00
Romain Vimont
338310677e Remove cbuf
All uses have been replaced by VecDeque.
2023-03-10 22:22:15 +01:00
Romain Vimont
f978e4d6de Use VecDeque in aoa_hid
Replace cbuf by VecDeque in aoa_hid
2023-03-10 22:22:15 +01:00
Romain Vimont
a0a65b3c4d Use VecDeque in file_pusher
Replace cbuf by VecDeque in file_pusher.

As a side-effect, the new implementation does not limit the queue to an
arbitrary value.
2023-03-10 22:22:15 +01:00
Romain Vimont
4d989de9ae Use VecDeque in controller
Replace cbuf by VecDeque in controller.
2023-03-10 22:22:15 +01:00
Romain Vimont
f25a67f342 Use VecDeque in video_buffer
The packets queued for buffering were wrapped in a dynamically allocated
structure with a "next" field.

To avoid this additional layer of allocation and indirection, use a
VecDeque.
2023-03-10 22:22:15 +01:00
Romain Vimont
efc15744da Use VecDeque in recorder
The packets queued for recording were wrapped in a dynamically allocated
structure with a "next" field.

To avoid this additional layer of allocation and indirection, use a
VecDeque.
2023-03-10 22:22:15 +01:00
Romain Vimont
33df484912 Introduce VecDeque
Introduce a double-ended queue implemented with a growable ring buffer.

Inspired from the Rust VecDeque type:
<https://doc.rust-lang.org/std/collections/struct.VecDeque.html>
2023-03-10 22:22:15 +01:00
Romain Vimont
457385d5f4 Add sc_allocarray() util
Add a function to allocate an array, which fails safely in the case
where the multiplication would overflow.
2023-03-10 22:22:15 +01:00
Romain Vimont
c735b8c127 Use reallocarray() in sc_vector
This fails safely in case of overflow.
2023-03-10 22:22:15 +01:00
Romain Vimont
6dceb32817 Add compat for reallocarray()
This function fails safely in the case where the multiplication would
overflow.
2023-03-10 22:22:15 +01:00
Romain Vimont
6e05d7047a Call avcodec_receive_frame() in a loop
Since in scrcpy a video packet passed to avcodec_send_packet() is always
a complete video frame, it is sufficient to call avcodec_receive_frame()
exactly once.

In practice, it also works for audio packets: the decoder produces
exactly 1 frame for 1 input packet.

In theory, it is an implementation detail though, so
avcodec_receive_frame() should be called in a loop.
2023-03-10 22:22:15 +01:00
Romain Vimont
c1528cdca9 Add --require-audio
By default, scrcpy mirrors only the video when audio capture fails on
the device. Add an option to force scrcpy to fail if audio is enabled
but does not work.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Simon Chan
de40cac6ad Add workaround to capture audio on Android 11
On Android 11, it is possible to start the capture only when the running
app is in foreground. But scrcpy is not an app, it's a Java application
started from shell.

As a workaround, start an existing Android shell existing activity just
to start the capture, then close it immediately.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-10 22:22:15 +01:00
Romain Vimont
b60a8aa657 Add two-step write feature to bytebuf
If there is exactly one producer, then it can assume that the remaining
space in the buffer will only increase until it writes something.

This assumption may allow the producer to write to the buffer (up to a
known safe size) without any synchronization mechanism, thus allowing
to read and write different parts of the buffer in parallel.

The producer can then commit the write with a lock held, and update its
knowledge of the safe empty remaining space.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
20d41fdd7e Introduce bytebuf util
Add a ring-buffer for bytes. It will be useful for audio buffering.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
619730edaf Pass AVCodecContext to frame sinks
Frame consumers may need details about the frame format.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
e22660d698 Add an audio decoder
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
05f0e35d2a Give a name to decoder instances
This will be useful in logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
99837fa600 Rename decoder to video_decoder
This prepares the introduction of audio_decoder.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
a205ff6c8b Log display sizes in display list
This is more convenient than just the display id alone.
2023-03-10 22:22:15 +01:00
Romain Vimont
b65301f672 Add --list-displays
Add an option to list the device displays properly.
2023-03-10 22:22:15 +01:00
Romain Vimont
2596ca02f0 Move log message helpers to LogUtils
This class will also contain other log helpers.
2023-03-10 22:22:15 +01:00
Romain Vimont
50d56a9a2b Quit on audio configuration failure
When audio capture fails on the device, scrcpy continues mirroring the
video stream. This allows to enable audio by default only when
supported.

However, if an audio configuration occurs (for example the user
explicitly selected an unknown audio encoder), this must be treated as
an error and scrcpy must exit.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
9196dc1563 Add --list-encoders
Add an option to list the device encoders properly.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
b7e5284adf Move await_for_server() logs
Print the logs on the caller side. This will allow to call the function
in another context without printing the logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
f9960e959f Add --audio-encoder
Similar to --video-encoder, but for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
6f332a2bc7 Extract unknown encoder error message
This will allow to reuse the same code for audio encoder selection.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
b03c864c70 Add --audio-codec-options
Similar to --video-codec-options, but for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
58cf8e5401 Extract application of codec options
This will allow to reuse the same code for audio codec options.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
4601735e51 Add support for AAC audio codec
Add option --audio-codec=aac.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
839b842aa7 Add --audio-codec
Introduce the selection mechanism. Alternative codecs will be added
later.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
0870b8c8be Add --audio-bit-rate
Add an option to configure the audio bit-rate.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
8e640dc90f Disable MethodLength checkstyle on createOptions()
This method will grow as needed to initialize options.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
e694619d53 Rename --encoder to --video-encoder
This prepares the introduction of --audio-encoder.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
31555fa530 Rename --codec-options to --video-codec-options
This prepares the introduction of --audio-codec-options.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
9087e85c3f Rename --bit-rate to --video-bit-rate
This prepares the introduction of --audio-bit-rate.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
cee40ca047 Rename --codec to --video-codec
This prepares the introduction of --audio-codec.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
a1802dab76 Remove default bit-rate on client side
If no bit-rate is passed, let the server use the default value (8Mbps).

This avoids to define a default value on both sides, and to pass the
default bit-rate as an argument when starting the server.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
17d5301c0f Record at least video packets on stop
If the recorder is stopped while it has not received any audio packet
yet, make sure the video stream is correctly recorded.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
80c0780b77 Disable audio before Android 11
The permission "android.permission.RECORD_AUDIO" has been added for
shell in Android 11.

Moreover, on lower versions, it may make the server segfault on the
device (happened on a Nexus 5 with Android 6.0.1).

Refs <4feeee8891%5E%21/>
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
13a3395a33 Disable audio on initialization error
By default, audio is enabled (--no-audio must be explicitly passed to
disable it).

However, some devices may not support audio capture (typically devices
below Android 11, or Android 11 when the shell application is not
foreground on start).

In that case, make the server notify the client to dynamically disable
audio forwarding so that it does not wait indefinitely for an audio
stream.

Also disable audio on unknown codec or missing decoder on the
client-side, for the same reasons.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
7de0622214 Add audio recording support
Make the recorder accept two input sources (video and audio), and mux
them into a single file.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
3d29f6ef06 Rename video-specific variables in recorder
This paves the way to add audio-specific variables.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
609b098a97 Do not merge config audio packets
For video streams (at least H.264 and H.265), the config packet
containing SPS/PPS must be prepended to the next packet (the following
keyframe).

For audio streams (at least OPUS), they must not be merged.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
de430bc4aa Add an audio demuxer
Add a demuxer which will read the stream from the audio socket.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
f60b5767f4 Force --no-audio if no display and no recording
The client does not use the audio stream if there is no display and no
recording (i.e. only V4L2), so disable audio so that the device does not
attempt to capture it.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
d499f890e7 Give a name to demuxer instances
This will be useful in logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
e9876788c9 Rename demuxer to video_demuxer
There will be another demuxer instance for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
15556d1f3b Extract OPUS extradata
For OPUS codec, FFmpeg expects the raw extradata, but MediaCodec wraps
it in some structure.

Fix the config packet to send only the raw extradata.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
7cf5cf5875 Use a streamer to send the audio stream
Send each encoded audio packet using a streamer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
5eed2c52c2 Encode recorded audio on the device
For now, the encoded packets are just logged into the console.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
464a35b05e Make streamer more generic
Expose a method to write a packet from raw metadata (without
BufferInfo).
2023-03-10 22:22:15 +01:00
Simon Chan
11d32616a9 Capture device audio
Create an AudioRecorder to capture the audio source REMOTE_SUBMIX.

For now, the captured packets are just logged into the console.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-10 22:22:15 +01:00
Simon Chan
e841241a8e Add a new socket for audio stream
When audio is enabled, open a new socket to send the audio stream from
the device to the client.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-10 22:22:15 +01:00
Simon Chan
3cf03e4a4b Add --no-audio option
Audio will be enabled by default (when supported). Add an option to
disable it.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-10 22:22:15 +01:00
Romain Vimont
8487ddba64 Use FakeContext for Application instance
This will expose the correct package name and UID to the application
context.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
84ba6435bb Use shell package name for workarounds
For consistency.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
42285ae869 Use ROOT_UID from FakeContext
Remove USER_ID from ServiceManager, and replace it by a constant in
FakeContext.

This is the same as android.os.Process.ROOT_UID, but this constant has
been introduced in API 29.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
820189b6a6 Use PACKAGE_NAME from FakeContext
Remove duplicated constant.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
9a815ceba8 Use AttributionSource from FakeContext
FakeContext already provides an AttributeSource instance.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-03-10 22:22:15 +01:00
Simon Chan
a5541b3476 Add a fake Android Context
Since scrcpy-server is not an Android application (it's a java
executable), it has no Context.

Some features will require a Context instance to get the package name
and the UID. Add a FakeContext for this purpose.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-10 22:22:15 +01:00
Romain Vimont
10ef8da95d Improve error message for unknown encoder
The provided encoder name depends on the selected codec. Improve the
error message and the suggestions.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
90d88a6927 Rename "codec" variable to "mediaCodec"
This will allow to use "codec" for the Codec type.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
10ce0f376a Make streamer independent of codec type
Rename VideoStreamer to Streamer, and extract a Codec interface which
will also support audio codecs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-10 22:22:15 +01:00
Romain Vimont
c226797991 Pass all args to ScreenEncoder constructor
There is no good reason to pass some of them in the constructor and some
others as parameters of the streamScreen() method.
2023-03-10 22:22:15 +01:00
Romain Vimont
ae9b08b905 Move screen encoder initialization
This prepares further refactors.
2023-03-10 22:22:15 +01:00
Romain Vimont
51628201b7 Write streamer header from ScreenEncoder
The screen encoder is responsible for writing data to the video
streamer.
2023-03-10 22:22:15 +01:00
Romain Vimont
ae29e5b562 Use VideoStreamer directly from ScreenEncoder
The Callbacks interface notifies new packets. But in addition, the
screen encoder will need to write headers on start.

We could add a function onStart(), but for simplicity, just remove the
interface, which brings no value, and call the streamer directly.

Refs 87972e2022
2023-03-10 22:22:15 +01:00
Romain Vimont
5b2ec66222 Simplify error handling on socket creation
On any error, all previously opened sockets must be closed.

Handle these errors in a single catch-block. Currently, there are only 2
sockets, but this will simplify even more with more sockets.

Note: this commit is better displayed with --ignore-space-change (-b).
2023-03-10 22:22:15 +01:00
Romain Vimont
ef6a3b97a7 Reorder initialization
Initialize components in the pipeline order: demuxer first, decoder and
recorder second.
2023-03-10 22:22:15 +01:00
Romain Vimont
f9efe48aac Refactor recorder logic
Process the initial config packet (necessary to write the header)
separately.
2023-03-10 22:22:15 +01:00
Romain Vimont
4b246cd963 Move last packet recording
Write the last packet at the end.
2023-03-10 22:22:15 +01:00
Romain Vimont
3c407773e9 Add start() function for recorder
For consistency with the other components, do not start the internal
thread from an init() function.
2023-03-10 22:22:15 +01:00
Romain Vimont
a039124d5d Open recording file from the recorder thread
The recorder opened the target file from the packet sink open()
callback, called by the demuxer. Only then the recorder thread was
started.

One golden rule for the recorder is to never block the demuxer for I/O,
because it would impact mirroring. This rule is respected on recording
packets, but not for the initial recorder opening.

Therefore, start the recorder thread from sc_recorder_init(), open the
file immediately from the recorder thread, then make it wait for the
stream to start (on packet sink open()).

Now that the recorder can report errors directly (rather than making the
demuxer call fail), it is possible to report file opening error even
before the packet sink is open.
2023-03-10 22:22:15 +01:00
Romain Vimont
6b5dfef923 Inline packet_sink impl in recorder
Remove useless wrappers.
2023-03-10 22:22:15 +01:00
Romain Vimont
fb29135591 Initialize recorder fields from init()
The recorder has two initialization phases: one to initialize the
concrete recorder object, and one to open its packet_sink trait.

Initialize mutex and condvar as part of the object initialization.

If there were several packet_sink traits (spoiler: one for video, one
for audio), then the mutex and condvar would still be initialized only
once.
2023-03-10 22:22:15 +01:00
Romain Vimont
b1b33e3eaf Report recorder errors
Stop scrcpy on recorder errors.

It was previously indirectly stopped by the demuxer, which failed to
push packets to a recorder in error. Report it directly instead:
 - it avoids to wait for the next demuxer call;
 - it will allow to open the target file from a separate thread and stop
   immediately on any I/O error.
2023-03-10 22:22:15 +01:00
Romain Vimont
db5751a76a Move previous packet to a local variable
It is only used from run_recorder().
2023-03-10 22:22:15 +01:00
Romain Vimont
b6744e7887 Move pts_origin to a local variable
It is only used from run_recorder().
2023-03-10 22:22:15 +01:00
Romain Vimont
181fb555bb Change PTS origin type from uint64_t to int64_t
It is initialized from AVPacket.pts, which is an int64_t.
2023-03-10 22:22:15 +01:00
Romain Vimont
fa99763668 Fix --encoder documentation
Mention that it depends on the codec provided by --codec (which is not
necessarily H264 anymore).
2023-03-10 22:22:15 +01:00
Romain Vimont
b43938fa66 Do not print stacktraces when unnecessary
User-friendly error messages are printed on specific configuration
exceptions. In that case, do not print the stacktrace.

Also handle the user-friendly error message directly where the error
occurs, and print multiline messages in a single log call, to avoid
confusing interleaving.
2023-03-10 22:22:15 +01:00
Romain Vimont
9f8e96e895 Fix --no-clipboard-autosync bash completion
Fix typo.
2023-03-10 22:22:15 +01:00
Romain Vimont
c78254fcd1 Split server stop() and join()
For consistency with the other components, call stop() and join()
separately.

This allows to stop all components, then join them all.
2023-03-10 22:22:15 +01:00
Romain Vimont
e30e692b36 Print FFmpeg logs
FFmpeg logs are redirected to a specific SDL log category.

Initialize the log level for this category to print them as expected.
2023-03-10 22:22:15 +01:00
Romain Vimont
10e8295aea Move FFmpeg callback initialization
Configure FFmpeg log redirection on start from a log helper.
2023-03-10 22:22:15 +01:00
Romain Vimont
f30fd963a1 Upgrade FFmpeg custom builds for Windows
Use a build which includes the pcm_s16le decoder, to support RAW audio.

Refs <https://github.com/rom1v/scrcpy-deps/commits/6.0-scrcpy-2>
2023-03-10 22:22:15 +01:00
Romain Vimont
9d60d7880b Upgrade FFmpeg (6.0) for Windows
Use the latest version (specifically built for scrcpy).

Refs <https://www.ffmpeg.org/download.html#release_6.0>
2023-03-10 22:22:15 +01:00
Romain Vimont
0fc62bfcd6 Use minimal prebuilt FFmpeg for Windows
On the scrcpy-deps repo, I built FFmpeg 5.1.2 binaries for Windows with
only the features used by scrcpy.

For comparison, here are the sizes of the dll for FFmpeg 5.1.2:
 - before: 89M
 - after: 4.7M

It also allows to upgrade the old FFmpeg version (4.3.1) used for win32.

Refs <https://github.com/rom1v/scrcpy-deps>
Refs <https://github.com/Genymobile/scrcpy/issues/1753>
2023-03-10 22:22:15 +01:00
Romain Vimont
a20615066d Simplify libusb prebuilt scripts
In theory, include/ might be slightly different for win32 and win64
builds. Use each one separately to simplify.
2023-03-10 22:22:15 +01:00
32 changed files with 452 additions and 323 deletions

View File

@@ -78,7 +78,7 @@ _scrcpy() {
return
;;
--audio-codec)
COMPREPLY=($(compgen -W 'raw opus aac' -- "$cur"))
COMPREPLY=($(compgen -W 'opus aac raw' -- "$cur"))
return
;;
--lock-video-orientation)

View File

@@ -10,7 +10,7 @@ local arguments
arguments=(
'--always-on-top[Make scrcpy window always on top \(above other windows\)]'
'--audio-bit-rate=[Encode the audio at the given bit-rate]'
'--audio-codec=[Select the audio codec]:codec:(raw opus aac)'
'--audio-codec=[Select the audio codec]:codec:(opus aac raw)'
'--audio-codec-options=[Set a list of comma-separated key\:type=value options for the device audio encoder]'
'--audio-encoder=[Use a specific MediaCodec audio encoder]'
{-b,--video-bit-rate=}'[Encode the video at the given bit-rate]'

View File

@@ -23,20 +23,32 @@ Make scrcpy window always on top (above other windows).
.BI "\-\-audio\-bit\-rate " value
Encode the audio at the given bit\-rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
Default is 196K (196000).
Default is 128K (128000).
.TP
.BI "\-\-audio\-buffer ms
Add a buffering delay (in milliseconds) before playing audio. This increases latency to compensate for jitter.
Configure the audio buffering delay (in milliseconds).
Default is 0 (no buffering).
Lower values decrease the latency, but increase the likelyhood of buffer underrun (causing audio glitches).
Default is 50.
.TP
.BI "\-\-audio\-codec " name
Select an audio codec (raw, opus or aac).
Select an audio codec (opus, aac or raw).
Default is opus.
.TP
.BI "\-\-audio\-codec\-options " key\fR[:\fItype\fR]=\fIvalue\fR[,...]
Set a list of comma-separated key:type=value options for the device audio encoder.
The possible values for 'type' are 'int' (default), 'long', 'float' and 'string'.
The list of possible codec options is available in the Android documentation
.UR https://d.android.com/reference/android/media/MediaFormat
.UE .
.TP
.BI "\-\-audio\-encoder " name
Use a specific MediaCodec audio encoder (depending on the codec provided by \fB\-\-audio\-codec\fR).
@@ -270,7 +282,7 @@ Supported names are currently "direct3d", "opengl", "opengles2", "opengles", "me
.TP
.B \-\-require\-audio
By default, scrcpy mirrors only the video if audio capture fails on the device. This flag makes scrcpy fail if audio is enabled but does not work.
By default, scrcpy mirrors only the video if audio capture fails on the device. This option makes scrcpy fail if audio is enabled but does not work.
.TP
.BI "\-\-rotation " value

View File

@@ -4,7 +4,7 @@
#include "util/log.h"
//#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
/** Downcast frame_sink to sc_audio_player */
#define DOWNCAST(SINK) container_of(SINK, struct sc_audio_player, frame_sink)
@@ -12,29 +12,20 @@
#define SC_AV_SAMPLE_FMT AV_SAMPLE_FMT_FLT
#define SC_SDL_SAMPLE_FMT AUDIO_F32
#define SC_AUDIO_OUTPUT_BUFFER_SAMPLES 480 // 10ms at 48000Hz
#define SC_AUDIO_OUTPUT_BUFFER_SAMPLES 240 // 5ms at 48000Hz
// The target number of buffered samples between the producer and the consumer.
// This value is directly use for compensation.
#define SC_TARGET_BUFFERED_SAMPLES (3 * SC_AUDIO_OUTPUT_BUFFER_SAMPLES)
// Use a ring-buffer of 1 second (at 48000Hz) between the producer and the
// consumer. It too big, but it guarantees that the producer and the consumer
// will be able to access it in parallel without locking.
#define SC_BYTEBUF_SIZE_IN_SAMPLES 48000
static inline size_t
static inline uint32_t
bytes_to_samples(struct sc_audio_player *ap, size_t bytes) {
assert(bytes % (ap->nb_channels * ap->out_bytes_per_sample) == 0);
return bytes / (ap->nb_channels * ap->out_bytes_per_sample);
}
static inline size_t
samples_to_bytes(struct sc_audio_player *ap, size_t samples) {
samples_to_bytes(struct sc_audio_player *ap, uint32_t samples) {
return samples * ap->nb_channels * ap->out_bytes_per_sample;
}
void
static void SDLCALL
sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
struct sc_audio_player *ap = userdata;
@@ -45,34 +36,56 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
size_t len = len_int;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] SDL callback requests %" SC_PRIsizet " samples",
LOGD("[Audio] SDL callback requests %" PRIu32 " samples",
bytes_to_samples(ap, len));
#endif
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
if (!ap->played) {
uint32_t buffered_samples = bytes_to_samples(ap, read_avail);
// Part of the buffering is handled by inserting initial silence. The
// remaining (margin) last samples will be handled by compensation.
uint32_t margin = 30 * ap->sample_rate / 1000; // 30ms
if (buffered_samples + margin < ap->target_buffering) {
LOGV("[Audio] Inserting initial buffering silence: %" PRIu32
" samples", bytes_to_samples(ap, len));
// Delay playback starting to reach the target buffering. Fill the
// whole buffer with silence (len is small compared to the
// arbitrary margin value).
memset(stream, 0, len);
return;
}
}
size_t read = MIN(read_avail, len);
if (read) {
sc_bytebuf_read(&ap->buf, stream, read);
}
if (read < len) {
// Insert silence
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Buffer underflow, inserting silence: %" SC_PRIsizet
" samples", bytes_to_samples(ap, len - read));
#endif
memset(stream + read, 0, len - read);
// If the first frame has not been received yet, it's not an underflow
size_t silence_bytes = len - read;
uint32_t silence_samples = bytes_to_samples(ap, silence_bytes);
// Insert silence. In theory, the inserted silent samples replace the
// missing real samples, which will arrive later, so they should be
// dropped to keep the latency minimal. However, this would cause very
// audible glitches, so let the clock compensation restore the target
// latency.
LOGD("[Audio] Buffer underflow, inserting silence: %" PRIu32 " samples",
silence_samples);
memset(stream + read, 0, silence_bytes);
if (ap->received) {
ap->underflow += bytes_to_samples(ap, len - read);
// Inserting additional samples immediately increases buffering
ap->avg_buffering.avg += silence_samples;
}
}
ap->last_consumed = sc_tick_now();
ap->played = true;
}
static uint8_t *
sc_audio_player_get_swr_buf(struct sc_audio_player *ap, size_t min_samples) {
sc_audio_player_get_swr_buf(struct sc_audio_player *ap, uint32_t min_samples) {
size_t min_buf_size = samples_to_bytes(ap, min_samples);
if (min_buf_size > ap->swr_buf_alloc_size) {
size_t new_size = min_buf_size + 4096;
@@ -89,6 +102,186 @@ sc_audio_player_get_swr_buf(struct sc_audio_player *ap, size_t min_samples) {
return ap->swr_buf;
}
static bool
sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
const AVFrame *frame) {
struct sc_audio_player *ap = DOWNCAST(sink);
SwrContext *swr_ctx = ap->swr_ctx;
int64_t swr_delay = swr_get_delay(swr_ctx, ap->sample_rate);
// No need to av_rescale_rnd(), input and output sample rates are the same.
// Add more space (256) for clock compensation.
int dst_nb_samples = swr_delay + frame->nb_samples + 256;
uint8_t *swr_buf = sc_audio_player_get_swr_buf(ap, dst_nb_samples);
if (!swr_buf) {
return false;
}
int ret = swr_convert(swr_ctx, &swr_buf, dst_nb_samples,
(const uint8_t **) frame->data, frame->nb_samples);
if (ret < 0) {
LOGE("Resampling failed: %d", ret);
return false;
}
// swr_convert() returns the number of samples which would have been
// written if the buffer was big enough.
uint32_t samples_written = MIN(ret, dst_nb_samples);
size_t swr_buf_size = samples_to_bytes(ap, samples_written);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] %" PRIu32 " samples written to buffer", samples_written);
#endif
// Since this function is the only writer, the current available space is
// at least the previous available space. In practice, it should almost
// always be possible to write without lock.
bool lockless_write = swr_buf_size <= ap->previous_write_avail;
if (lockless_write) {
sc_bytebuf_prepare_write(&ap->buf, swr_buf, swr_buf_size);
}
SDL_LockAudioDevice(ap->device);
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
uint32_t buffered_samples = bytes_to_samples(ap, read_avail);
if (lockless_write) {
sc_bytebuf_commit_write(&ap->buf, swr_buf_size);
} else {
// Take care to keep full samples
size_t align = ap->nb_channels * ap->out_bytes_per_sample;
size_t write_avail =
sc_bytebuf_write_available(&ap->buf) / align * align;
if (swr_buf_size > write_avail) {
// Entering this branch is very unlikely, the ring-buffer (bytebuf)
// is allocated with a size sufficient to store 1 second more than
// the target buffering. If this happens, though, we have to skip
// old samples.
size_t cap = sc_bytebuf_capacity(&ap->buf) / align * align;
if (swr_buf_size > cap) {
// Very very unlikely: a single resampled frame should never
// exceed the ring-buffer size (or something is very wrong).
// Ignore the first bytes in swr_buf
swr_buf += swr_buf_size - cap;
swr_buf_size = cap;
// This change in samples_written will impact the
// instant_compensation below
samples_written -= bytes_to_samples(ap, swr_buf_size - cap);
}
assert(swr_buf_size >= write_avail);
if (swr_buf_size > write_avail) {
sc_bytebuf_skip(&ap->buf, swr_buf_size - write_avail);
uint32_t skip_samples =
bytes_to_samples(ap, swr_buf_size - write_avail);
assert(buffered_samples >= skip_samples);
buffered_samples -= skip_samples;
if (ap->played) {
// Dropping input samples instantly decreases buffering
ap->avg_buffering.avg -= skip_samples;
}
}
// It should remain exactly the expected size to write the new
// samples.
assert((sc_bytebuf_write_available(&ap->buf) / align * align)
== swr_buf_size);
}
sc_bytebuf_write(&ap->buf, swr_buf, swr_buf_size);
}
buffered_samples += samples_written;
assert(samples_to_bytes(ap, buffered_samples)
== sc_bytebuf_read_available(&ap->buf));
// Read with lock held, to be used after unlocking
bool played = ap->played;
if (played) {
uint32_t max_buffered_samples = ap->target_buffering
+ 12 * SC_AUDIO_OUTPUT_BUFFER_SAMPLES
+ ap->target_buffering / 10;
if (buffered_samples > max_buffered_samples) {
uint32_t skip_samples = buffered_samples - max_buffered_samples;
size_t skip_bytes = samples_to_bytes(ap, skip_samples);
sc_bytebuf_skip(&ap->buf, skip_bytes);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Buffering threshold exceeded, skipping %" PRIu32
" samples", skip_samples);
#endif
}
// Number of samples added (or removed, if negative) for compensation
int32_t instant_compensation =
(int32_t) samples_written - frame->nb_samples;
// The compensation must apply instantly, it must not be smoothed
ap->avg_buffering.avg += instant_compensation;
// However, the buffering level must be smoothed
sc_average_push(&ap->avg_buffering, buffered_samples);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] buffered_samples=%" PRIu32 " avg_buffering=%f",
buffered_samples, sc_average_get(&ap->avg_buffering));
#endif
} else {
// SDL playback not started yet, do not accumulate more than
// max_initial_buffering samples, this would cause unnecessary delay
// (and glitches to compensate) on start.
uint32_t max_initial_buffering = ap->target_buffering
+ 2 * SC_AUDIO_OUTPUT_BUFFER_SAMPLES;
if (buffered_samples > max_initial_buffering) {
uint32_t skip_samples = buffered_samples - max_initial_buffering;
size_t skip_bytes = samples_to_bytes(ap, skip_samples);
sc_bytebuf_skip(&ap->buf, skip_bytes);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Playback not started, skipping %" PRIu32 " samples",
skip_samples);
#endif
}
}
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
ap->received = true;
SDL_UnlockAudioDevice(ap->device);
if (played) {
ap->samples_since_resync += samples_written;
if (ap->samples_since_resync >= ap->sample_rate) {
// Recompute compensation every second
ap->samples_since_resync = 0;
float avg = sc_average_get(&ap->avg_buffering);
int diff = ap->target_buffering - avg;
if (diff < 0 && buffered_samples < ap->target_buffering) {
// Do not accelerate if the instant buffering level is below
// the average, this would increase underflow
diff = 0;
}
// Compensate the diff over 4 seconds (but will be recomputed after
// 1 second)
int distance = 4 * ap->sample_rate;
// Limit compensation rate to 2%
int abs_max_diff = distance / 50;
diff = CLAMP(diff, -abs_max_diff, abs_max_diff);
LOGV("[Audio] Buffering: target=%" PRIu32 " avg=%f cur=%" PRIu32
" compensation=%d", ap->target_buffering, avg,
buffered_samples, diff);
int ret = swr_set_compensation(swr_ctx, diff, distance);
if (ret < 0) {
LOGW("Resampling compensation failed: %d", ret);
// not fatal
}
}
}
return true;
}
static bool
sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
const AVCodecContext *ctx) {
@@ -127,7 +320,6 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
assert(ctx->sample_rate > 0);
assert(!av_sample_fmt_is_planar(SC_AV_SAMPLE_FMT));
int out_bytes_per_sample = av_get_bytes_per_sample(SC_AV_SAMPLE_FMT);
assert(out_bytes_per_sample > 0);
@@ -157,7 +349,15 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
ap->nb_channels = nb_channels;
ap->out_bytes_per_sample = out_bytes_per_sample;
size_t bytebuf_size = samples_to_bytes(ap, SC_BYTEBUF_SIZE_IN_SAMPLES);
ap->target_buffering = ap->target_buffering_delay * ap->sample_rate
/ SC_TICK_FREQ;
// Use a ring-buffer of the target buffering size plus 1 second between the
// producer and the consumer. It's too big on purpose, to guarantee that
// the producer and the consumer will be able to access it in parallel
// without locking.
size_t bytebuf_samples = ap->target_buffering + ap->sample_rate;
size_t bytebuf_size = samples_to_bytes(ap, bytebuf_samples);
bool ok = sc_bytebuf_init(&ap->buf, bytebuf_size);
if (!ok) {
@@ -174,12 +374,21 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
sc_average_init(&ap->avg_buffering, 8);
// Samples are produced and consumed by blocks, so the buffering must be
// smoothed to get a relatively stable value.
sc_average_init(&ap->avg_buffering, 32);
ap->samples_since_resync = 0;
ap->last_consumed = 0;
ap->underflow = 0;
ap->received = 0;
ap->received = false;
ap->played = false;
// The thread calling open() is the thread calling push(), which fills the
// audio buffer consumed by the SDL audio thread.
ok = sc_thread_set_priority(SC_THREAD_PRIORITY_TIME_CRITICAL);
if (!ok) {
ok = sc_thread_set_priority(SC_THREAD_PRIORITY_HIGH);
(void) ok; // We don't care if it worked, at least we tried
}
SDL_PauseAudioDevice(ap->device, 0);
@@ -208,166 +417,10 @@ sc_audio_player_frame_sink_close(struct sc_frame_sink *sink) {
swr_free(&ap->swr_ctx);
}
static bool
sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
const AVFrame *frame) {
struct sc_audio_player *ap = DOWNCAST(sink);
SwrContext *swr_ctx = ap->swr_ctx;
int64_t delay = swr_get_delay(swr_ctx, ap->sample_rate);
// No need to av_rescale_rnd(), input and output sample rates are the same
// Add more space (256) for clock compensation
int dst_nb_samples = delay + frame->nb_samples + 256;
uint8_t *swr_buf = sc_audio_player_get_swr_buf(ap, dst_nb_samples);
if (!swr_buf) {
return false;
}
int ret = swr_convert(swr_ctx, &swr_buf, dst_nb_samples,
(const uint8_t **) frame->data, frame->nb_samples);
if (ret < 0) {
LOGE("Resampling failed: %d", ret);
return false;
}
// swr_convert() returns the number of samples which would have been
// written if the buffer was big enough.
size_t samples_written = MIN(ret, dst_nb_samples);
size_t swr_buf_size = samples_to_bytes(ap, samples_written);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] %" SC_PRIsizet " samples written to buffer", samples_written);
#endif
// Since this function is the only writer, the current available space is
// at least the previous available space. In practice, it should almost
// always be possible to write without lock.
bool lockless_write = swr_buf_size <= ap->previous_write_avail;
if (lockless_write) {
sc_bytebuf_prepare_write(&ap->buf, swr_buf, swr_buf_size);
}
SDL_LockAudioDevice(ap->device);
// The consumer requests audio samples blocks (e.g. 480 samples).
// Convert the duration since the last consumption into samples.
size_t extrapolated = 0;
if (ap->last_consumed) {
sc_tick now = sc_tick_now();
assert(now >= ap->last_consumed);
extrapolated = (now - ap->last_consumed) * ap->sample_rate
/ SC_TICK_FREQ;
}
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
// The consumer may not increase underflow value if there are still samples
// available
assert(read_avail == 0 || ap->underflow == 0);
size_t buffered_samples = bytes_to_samples(ap, read_avail);
// Underflow caused silence samples in excess (so it adds buffering).
// Extrapolated samples must be considered consumed for smoothing (so it
// removes buffering).
float buffering = (float) buffered_samples + ap->underflow - extrapolated;
sc_average_push(&ap->avg_buffering, buffering);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] buffered_samples=%" SC_PRIsizet
" underflow=%" SC_PRIsizet
" extrapolated=%" SC_PRIsizet
" buffering=%f avg_buffering=%f",
buffered_samples, ap->underflow, extrapolated, buffering,
sc_average_get(&ap->avg_buffering));
#endif
if (lockless_write) {
sc_bytebuf_commit_write(&ap->buf, swr_buf_size);
} else {
// Take care to keep full samples
size_t align = ap->nb_channels * ap->out_bytes_per_sample;
size_t write_avail =
sc_bytebuf_write_available(&ap->buf) / align * align;
if (swr_buf_size > write_avail) {
// Skip old samples
size_t cap = sc_bytebuf_capacity(&ap->buf) / align * align;
if (swr_buf_size > cap) {
// Ignore the first bytes in swr_buf
swr_buf += swr_buf_size - cap;
swr_buf_size = cap;
}
assert(swr_buf_size > write_avail);
if (swr_buf_size - write_avail > 0) {
sc_bytebuf_skip(&ap->buf, swr_buf_size - write_avail);
}
}
sc_bytebuf_write(&ap->buf, swr_buf, swr_buf_size);
}
// On buffer underflow, typically because a packet is late, silence is
// inserted. In that case, the late samples must be ignored when they
// arrive, otherwise they will delay playback.
//
// As an improvement, instead of naively skipping the silence duration, we
// can absorb it if it helps clock compensation.
if (ap->underflow) {
size_t avg = sc_average_get(&ap->avg_buffering);
if (avg > SC_TARGET_BUFFERED_SAMPLES) {
size_t diff = SC_TARGET_BUFFERED_SAMPLES - avg;
if (ap->underflow > diff) {
// Partially absorb underflow for clock compensation (only keep
// the diff with the target buffering level).
ap->underflow -= diff;
} else {
// Totally absorb underflow for clock compensation
ap->underflow = 0;
}
size_t skip_samples = MIN(ap->underflow, buffered_samples);
if (skip_samples) {
size_t skip_bytes = samples_to_bytes(ap, skip_samples);
sc_bytebuf_skip(&ap->buf, skip_bytes);
read_avail -= skip_bytes;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Skipping %" SC_PRIsizet " samples", skip_samples);
#endif
}
} else {
// Totally absorb underflow for clock compensation
ap->underflow = 0;
}
}
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
ap->received = true;
SDL_UnlockAudioDevice(ap->device);
ap->samples_since_resync += samples_written;
if (ap->samples_since_resync >= ap->sample_rate) {
// Resync every second
ap->samples_since_resync = 0;
float avg = sc_average_get(&ap->avg_buffering);
int diff = SC_TARGET_BUFFERED_SAMPLES - avg;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Average buffering=%f, compensation %d", avg, diff);
#endif
// Compensate the diff over 3 seconds (but will be recomputed after
// 1 second)
int ret = swr_set_compensation(swr_ctx, diff, 3 * ap->sample_rate);
if (ret < 0) {
LOGW("Resampling compensation failed: %d", ret);
// not fatal
}
}
return true;
}
void
sc_audio_player_init(struct sc_audio_player *ap) {
sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering) {
ap->target_buffering_delay = target_buffering;
static const struct sc_frame_sink_ops ops = {
.open = sc_audio_player_frame_sink_open,
.close = sc_audio_player_frame_sink_close,

View File

@@ -8,6 +8,7 @@
#include <util/average.h>
#include <util/bytebuf.h>
#include <util/thread.h>
#include <util/tick.h>
#include <libavformat/avformat.h>
#include <libswresample/swresample.h>
@@ -18,10 +19,23 @@ struct sc_audio_player {
SDL_AudioDeviceID device;
// protected by SDL_AudioDeviceLock()
// The target buffering between the producer and the consumer. This value
// is directly use for compensation.
// Since audio capture and/or encoding on the device typically produce
// blocks of 960 samples (20ms) or 1024 samples (~21.3ms), this target
// value should be higher.
sc_tick target_buffering_delay;
uint32_t target_buffering; // in samples
// Audio buffer to communicate between the receiver and the SDL audio
// callback (protected by SDL_AudioDeviceLock())
struct sc_bytebuf buf;
// The previous number of bytes available in the buffer (only used by the
// receiver thread)
size_t previous_write_avail;
// Resampler (only used from the receiver thread)
struct SwrContext *swr_ctx;
// The sample rate is the same for input and output
@@ -31,22 +45,25 @@ struct sc_audio_player {
// The number of bytes per sample for a single channel
unsigned out_bytes_per_sample;
// Target buffer for resampling
// Target buffer for resampling (only used by the receiver thread)
uint8_t *swr_buf;
size_t swr_buf_alloc_size;
// Number of buffered samples (may be negative on underflow)
// Number of buffered samples (may be negative on underflow) (only used by
// the receiver thread)
struct sc_average avg_buffering;
// Count the number of samples to trigger a compensation update regularly
size_t samples_since_resync;
// (only used by the receiver thread)
uint32_t samples_since_resync;
// The last date a sample has been consumed by the audio output
sc_tick last_consumed;
// Number of silence samples inserted to be compensated
size_t underflow;
// Set to true the first time a sample is received (protected by
// SDL_AudioDeviceLock())
bool received;
// Set to true the first time the SDL callback is called (protected by
// SDL_AudioDeviceLock())
bool played;
const struct sc_audio_player_callbacks *cbs;
void *cbs_userdata;
};
@@ -56,6 +73,6 @@ struct sc_audio_player_callbacks {
};
void
sc_audio_player_init(struct sc_audio_player *ap);
sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering);
#endif

View File

@@ -19,6 +19,7 @@
enum {
OPT_RENDER_EXPIRED_FRAMES = 1000,
OPT_BIT_RATE,
OPT_WINDOW_TITLE,
OPT_PUSH_TARGET,
OPT_ALWAYS_ON_TOP,
@@ -118,21 +119,22 @@ static const struct sc_option options[] = {
.argdesc = "value",
.text = "Encode the audio at the given bit-rate, expressed in bits/s. "
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is 196K (196000).",
"Default is 128K (128000).",
},
{
.longopt_id = OPT_AUDIO_BUFFER,
.longopt = "audio-buffer",
.argdesc = "ms",
.text = "Add a buffering delay (in milliseconds) before playing audio. "
"This increases latency to compensate for jitter.\n"
"Default is 0 (no buffering).",
.text = "Configure the audio buffering delay (in milliseconds).\n"
"Lower values decrease the latency, but increase the "
"likelyhood of buffer underrun (causing audio glitches).\n"
"Default is 50.",
},
{
.longopt_id = OPT_AUDIO_CODEC,
.longopt = "audio-codec",
.argdesc = "name",
.text = "Select an audio codec (raw, opus or aac).\n"
.text = "Select an audio codec (opus, aac or raw).\n"
"Default is opus.",
},
{
@@ -163,6 +165,12 @@ static const struct sc_option options[] = {
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is 8M (8000000).",
},
{
// deprecated
.longopt_id = OPT_BIT_RATE,
.longopt = "bit-rate",
.argdesc = "value",
},
{
// Not really deprecated (--codec has never been released), but without
// declaring an explicit --codec option, getopt_long() partial matching
@@ -472,8 +480,8 @@ static const struct sc_option options[] = {
.longopt_id = OPT_REQUIRE_AUDIO,
.longopt = "require-audio",
.text = "By default, scrcpy mirrors only the video when audio capture "
"fails on the device. This flag makes scrcpy fail if audio is "
"enabled but does not work."
"fails on the device. This option makes scrcpy fail if audio "
"is enabled but does not work."
},
{
.longopt_id = OPT_ROTATION,
@@ -1506,10 +1514,6 @@ parse_video_codec(const char *optarg, enum sc_codec *codec) {
static bool
parse_audio_codec(const char *optarg, enum sc_codec *codec) {
if (!strcmp(optarg, "raw")) {
*codec = SC_CODEC_RAW;
return true;
}
if (!strcmp(optarg, "opus")) {
*codec = SC_CODEC_OPUS;
return true;
@@ -1518,7 +1522,11 @@ parse_audio_codec(const char *optarg, enum sc_codec *codec) {
*codec = SC_CODEC_AAC;
return true;
}
LOGE("Unsupported audio codec: %s (expected raw, opus or aac)", optarg);
if (!strcmp(optarg, "raw")) {
*codec = SC_CODEC_RAW;
return true;
}
LOGE("Unsupported audio codec: %s (expected opus, aac or raw)", optarg);
return false;
}
@@ -1532,6 +1540,9 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
int c;
while ((c = getopt_long(argc, argv, optstring, longopts, NULL)) != -1) {
switch (c) {
case OPT_BIT_RATE:
LOGW("--bit-rate is deprecated, use --video-bit-rate instead.");
// fall through
case 'b':
if (!parse_bit_rate(optarg, &opts->video_bit_rate)) {
return false;
@@ -1933,18 +1944,6 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
}
}
if (opts->audio_codec == SC_CODEC_RAW) {
if (opts->audio_bit_rate) {
LOGW("--audio-bit-rate is ignored for raw audio codec");
}
if (opts->audio_codec_options) {
LOGW("--audio-codec-options is ignored for raw audio codec");
}
if (opts->audio_encoder) {
LOGW("--audio-encoder is ignored for raw audio codec");
}
}
if (!opts->control) {
if (opts->turn_screen_off) {
LOGE("Could not request to turn screen off if control is disabled");

View File

@@ -105,8 +105,7 @@ sc_clock_update(struct sc_clock *clock, sc_tick system, sc_tick stream) {
sc_clock_estimate(clock, &clock->slope, &clock->offset);
#ifndef SC_CLOCK_NDEBUG
LOGD("Clock estimation: %f * pts + %" PRItick,
clock->slope, clock->offset);
LOGD("Clock estimation: %f * pts + %" PRItick, clock->slope, clock->offset);
#endif
}

View File

@@ -54,6 +54,10 @@
# define SCRCPY_SDL_HAS_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR
#endif
#if SDL_VERSION_ATLEAST(2, 0, 16)
# define SCRCPY_SDL_HAS_THREAD_PRIORITY_TIME_CRITICAL
#endif
#ifndef HAVE_STRDUP
char *strdup(const char *s);
#endif

View File

@@ -58,7 +58,7 @@ run_buffering(void *data) {
sc_tick max_deadline = sc_tick_now() + db->delay;
// PTS (written by the server) are expressed in microseconds
sc_tick pts = SC_TICK_TO_US(dframe.frame->pts);
sc_tick pts = SC_TICK_FROM_US(dframe.frame->pts);
bool timed_out = false;
while (!db->stopped && !timed_out) {

View File

@@ -23,9 +23,9 @@ sc_demuxer_to_avcodec_id(uint32_t codec_id) {
#define SC_CODEC_ID_H264 UINT32_C(0x68323634) // "h264" in ASCII
#define SC_CODEC_ID_H265 UINT32_C(0x68323635) // "h265" in ASCII
#define SC_CODEC_ID_AV1 UINT32_C(0x00617631) // "av1" in ASCII
#define SC_CODEC_ID_RAW UINT32_C(0x00726177) // "raw" in ASCII
#define SC_CODEC_ID_OPUS UINT32_C(0x6f707573) // "opus" in ASCII
#define SC_CODEC_ID_AAC UINT32_C(0x00616163) // "aac in ASCII"
#define SC_CODEC_ID_RAW UINT32_C(0x00726177) // "raw" in ASCII
switch (codec_id) {
case SC_CODEC_ID_H264:
return AV_CODEC_ID_H264;
@@ -33,12 +33,12 @@ sc_demuxer_to_avcodec_id(uint32_t codec_id) {
return AV_CODEC_ID_HEVC;
case SC_CODEC_ID_AV1:
return AV_CODEC_ID_AV1;
case SC_CODEC_ID_RAW:
return AV_CODEC_ID_PCM_S16LE;
case SC_CODEC_ID_OPUS:
return AV_CODEC_ID_OPUS;
case SC_CODEC_ID_AAC:
return AV_CODEC_ID_AAC;
case SC_CODEC_ID_RAW:
return AV_CODEC_ID_PCM_S16LE;
default:
LOGE("Unknown codec id 0x%08" PRIx32, codec_id);
return AV_CODEC_ID_NONE;
@@ -200,7 +200,7 @@ run_demuxer(void *data) {
ok = sc_packet_source_sinks_push(&demuxer->packet_source, packet);
av_packet_unref(packet);
if (!ok) {
LOGE("Demuxer '%s': could not process packet", demuxer->name);
// The sink already logged its concrete error
break;
}
}

View File

@@ -43,7 +43,7 @@ const struct scrcpy_options scrcpy_options_default = {
.display_id = 0,
.display_buffer = 0,
.v4l2_buffer = 0,
.audio_buffer = 0,
.audio_buffer = SC_TICK_FROM_MS(50),
#ifdef HAVE_USB
.otg = false,
#endif

View File

@@ -27,9 +27,9 @@ enum sc_codec {
SC_CODEC_H264,
SC_CODEC_H265,
SC_CODEC_AV1,
SC_CODEC_RAW,
SC_CODEC_OPUS,
SC_CODEC_AAC,
SC_CODEC_RAW,
};
enum sc_lock_video_orientation {

View File

@@ -240,7 +240,8 @@ sc_recorder_process_header(struct sc_recorder *recorder) {
sc_cond_wait(&recorder->queue_cond, &recorder->mutex);
}
if (recorder->stopped && sc_vecdeque_is_empty(&recorder->video_queue)) {
if (sc_vecdeque_is_empty(&recorder->video_queue)) {
assert(recorder->stopped);
// If the recorder is stopped, don't process anything if there are not
// at least video packets
sc_mutex_unlock(&recorder->mutex);
@@ -394,10 +395,6 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
error = true;
goto end;
}
// If the recorder is stopped while one of the streams has no
// packets, then we must avoid a live-loop and correctly record
// the stream having packets.
pts_origin = video_pkt ? video_pkt->pts : audio_pkt->pts;
} else {
// We need both video and audio packets to initialize pts_origin
continue;
@@ -508,6 +505,10 @@ static int
run_recorder(void *data) {
struct sc_recorder *recorder = data;
// Recording is a background task
bool ok = sc_thread_set_priority(SC_THREAD_PRIORITY_LOW);
(void) ok; // We don't care if it worked
bool success = sc_recorder_record(recorder);
sc_mutex_lock(&recorder->mutex);
@@ -583,7 +584,7 @@ sc_recorder_video_packet_sink_push(struct sc_packet_sink *sink,
return false;
}
rec->stream_index = 0;
rec->stream_index = recorder->video_stream_index;
bool ok = sc_vecdeque_push(&recorder->video_queue, rec);
if (!ok) {
@@ -652,7 +653,7 @@ sc_recorder_audio_packet_sink_push(struct sc_packet_sink *sink,
return false;
}
rec->stream_index = 1;
rec->stream_index = recorder->audio_stream_index;
bool ok = sc_vecdeque_push(&recorder->audio_queue, rec);
if (!ok) {

View File

@@ -43,7 +43,6 @@ struct scrcpy {
struct sc_server server;
struct sc_screen screen;
struct sc_audio_player audio_player;
struct sc_delay_buffer audio_buffer;
struct sc_demuxer video_demuxer;
struct sc_demuxer audio_demuxer;
struct sc_decoder video_decoder;
@@ -246,13 +245,6 @@ sc_audio_demuxer_on_ended(struct sc_demuxer *demuxer,
// Contrary to the video demuxer, keep mirroring if only the audio fails
// (unless --require-audio is set).
// 'eos' is true on end-of-stream, including when audio capture is not
// possible on the device (so that scrcpy continue to mirror video without
// failing).
// However, if an audio configuration failure occurs (for example the user
// explicitly selected an unknown audio encoder), 'eos' is false and scrcpy
// must exit.
if (status == SC_DEMUXER_STATUS_ERROR
|| (status == SC_DEMUXER_STATUS_DISABLED
&& options->require_audio)) {
@@ -695,16 +687,9 @@ aoa_hid_end:
sc_frame_source_add_sink(src, &s->screen.frame_sink);
if (options->audio) {
struct sc_frame_source *src = &s->audio_decoder.frame_source;
if (options->audio_buffer) {
sc_delay_buffer_init(&s->audio_buffer, options->audio_buffer,
false);
sc_frame_source_add_sink(src, &s->audio_buffer.frame_sink);
src = &s->audio_buffer.frame_source;
}
sc_audio_player_init(&s->audio_player);
sc_frame_source_add_sink(src, &s->audio_player.frame_sink);
sc_audio_player_init(&s->audio_player, options->audio_buffer);
sc_frame_source_add_sink(&s->audio_decoder.frame_source,
&s->audio_player.frame_sink);
}
}

View File

@@ -169,12 +169,12 @@ sc_server_get_codec_name(enum sc_codec codec) {
return "h265";
case SC_CODEC_AV1:
return "av1";
case SC_CODEC_RAW:
return "raw";
case SC_CODEC_OPUS:
return "opus";
case SC_CODEC_AAC:
return "aac";
case SC_CODEC_RAW:
return "raw";
default:
return NULL;
}

View File

@@ -9,7 +9,6 @@
bool
sc_bytebuf_init(struct sc_bytebuf *buf, size_t alloc_size) {
assert(alloc_size);
// sufficient, but use more for alignment.
buf->data = malloc(alloc_size);
if (!buf->data) {
LOG_OOM();
@@ -64,8 +63,8 @@ sc_bytebuf_write_step0(struct sc_bytebuf *buf, const uint8_t *from,
if (len < right_len) {
right_len = len;
}
memcpy(buf->data + buf->head, from, right_len);
if (len > right_len) {
memcpy(buf->data, from + right_len, len - right_len);
}

View File

@@ -14,7 +14,7 @@ struct sc_bytebuf {
size_t head; // writter cursor
size_t tail; // reader cursor
// empty: tail == head
// full: (tail + 1) % allocated == head
// full: ((tail + 1) % alloc_size) == head
};
bool
@@ -37,12 +37,10 @@ sc_bytebuf_read(struct sc_bytebuf *buf, uint8_t *to, size_t len);
* The caller must check that len <= sc_bytebuf_read_available() (it is an
* error to attempt to skip more bytes than available).
*
* This function is guaranteed not to change the head.
*
* This function is guaranteed to not change the head.
* This function is guaranteed not to write to buf->head.
*
* It is equivalent to call sc_bytebuf_read() to some array and discard the
* array (but more efficient since there is no copy).
* array (but this function is more efficient since there is no copy).
*/
void
sc_bytebuf_skip(struct sc_bytebuf *buf, size_t len);

View File

@@ -125,8 +125,30 @@ sc_av_log_callback(void *avcl, int level, const char *fmt, va_list vl) {
free(local_fmt);
}
static const char *const sc_sdl_log_priority_names[SDL_NUM_LOG_PRIORITIES] = {
[SDL_LOG_PRIORITY_VERBOSE] = "VERBOSE",
[SDL_LOG_PRIORITY_DEBUG] = "DEBUG",
[SDL_LOG_PRIORITY_INFO] = "INFO",
[SDL_LOG_PRIORITY_WARN] = "WARN",
[SDL_LOG_PRIORITY_ERROR] = "ERROR",
[SDL_LOG_PRIORITY_CRITICAL] = "CRITICAL",
};
static void SDLCALL
sc_sdl_log_print(void *userdata, int category, SDL_LogPriority priority,
const char *message) {
(void) userdata;
(void) category;
FILE *out = priority < SDL_LOG_PRIORITY_WARN ? stdout : stderr;
assert(priority < SDL_NUM_LOG_PRIORITIES);
const char *prio_name = sc_sdl_log_priority_names[priority];
fprintf(out, "%s: %s\n", prio_name, message);
}
void
sc_log_configure() {
SDL_LogSetOutputFunction(sc_sdl_log_print, NULL);
// Redirect FFmpeg logs to SDL logs
av_log_set_callback(sc_av_log_callback);
}

View File

@@ -3,7 +3,10 @@
#include <stddef.h>
/* Like calloc(), but without initialization.
/**
* Allocate an array of `nmemb` items of `size` bytes each
*
* Like calloc(), but without initialization.
* Like reallocarray(), but without reallocation.
*/
void *

View File

@@ -23,6 +23,39 @@ sc_thread_create(sc_thread *thread, sc_thread_fn fn, const char *name,
return true;
}
static SDL_ThreadPriority
to_sdl_thread_priority(enum sc_thread_priority priority) {
switch (priority) {
case SC_THREAD_PRIORITY_TIME_CRITICAL:
#ifdef SCRCPY_SDL_HAS_THREAD_PRIORITY_TIME_CRITICAL
return SDL_THREAD_PRIORITY_TIME_CRITICAL;
#else
// fall through
#endif
case SC_THREAD_PRIORITY_HIGH:
return SDL_THREAD_PRIORITY_HIGH;
case SC_THREAD_PRIORITY_NORMAL:
return SDL_THREAD_PRIORITY_NORMAL;
case SC_THREAD_PRIORITY_LOW:
return SDL_THREAD_PRIORITY_LOW;
default:
assert(!"Unknown thread priority");
return 0;
}
}
bool
sc_thread_set_priority(enum sc_thread_priority priority) {
SDL_ThreadPriority sdl_priority = to_sdl_thread_priority(priority);
int r = SDL_SetThreadPriority(sdl_priority);
if (r) {
LOGD("Could not set thread priority: %s", SDL_GetError());
return false;
}
return true;
}
void
sc_thread_join(sc_thread *thread, int *status) {
SDL_WaitThread(thread->thread, status);

View File

@@ -21,6 +21,13 @@ typedef struct sc_thread {
SDL_Thread *thread;
} sc_thread;
enum sc_thread_priority {
SC_THREAD_PRIORITY_LOW,
SC_THREAD_PRIORITY_NORMAL,
SC_THREAD_PRIORITY_HIGH,
SC_THREAD_PRIORITY_TIME_CRITICAL,
};
typedef struct sc_mutex {
SDL_mutex *mutex;
#ifndef NDEBUG
@@ -39,6 +46,9 @@ sc_thread_create(sc_thread *thread, sc_thread_fn fn, const char *name,
void
sc_thread_join(sc_thread *thread, int *status);
bool
sc_thread_set_priority(enum sc_thread_priority priority);
bool
sc_mutex_init(sc_mutex *mutex);

View File

@@ -52,10 +52,10 @@
*/
#define sc_vecdeque_init(pv) \
({ \
(pv)->data = NULL; \
(pv)->cap = 0; \
(pv)->origin = 0; \
(pv)->size = 0; \
(pv)->data = NULL; \
})
/**
@@ -128,7 +128,7 @@
* \param item_size the size of one item (the generic type is unknown from this
* function)
* \param pcap a pointer to the `cap` field of the SC_VECDEQUE [IN/OUT]
* \param porigin a pointer to pv->origin (will be read and updated)
* \param porigin a pointer to pv->origin [IN/OUT]
* \param size the `size` field of the SC_VECDEQUE
* \return the new array to assign to the `data` field of the SC_VECDEQUE (if
* not NULL)
@@ -312,7 +312,7 @@ sc_vecdeque_growsize_(size_t value)
*
* If the VecDeque is full, it is resized.
*
* This function returns either a valid non-nULL pointer to the uninitialized
* This function returns either a valid non-NULL pointer to the uninitialized
* item just pushed, or NULL on reallocation failure.
*/
#define sc_vecdeque_push_hole(pv) \
@@ -369,7 +369,7 @@ sc_vecdeque_growsize_(size_t value)
})
/**
* Pop an item and returns it
* Pop an item and return it
*
* It is an error to call this function if the VecDeque is empty.
*/

View File

@@ -0,0 +1,7 @@
package com.genymobile.scrcpy;
public interface AsyncProcessor {
void start();
void stop();
void join() throws InterruptedException;
}

View File

@@ -129,8 +129,8 @@ public final class AudioCapture {
pts = nextPts;
}
long durationMs = r * 1000 / CHANNELS / SAMPLE_RATE;
nextPts = pts + durationMs;
long durationUs = r * 1000000 / (CHANNELS * BYTES_PER_SAMPLE * SAMPLE_RATE);
nextPts = pts + durationUs;
if (previousPts != 0 && pts < previousPts) {
// Audio PTS may come from two sources:

View File

@@ -3,9 +3,9 @@ package com.genymobile.scrcpy;
import android.media.MediaFormat;
public enum AudioCodec implements Codec {
RAW(0x00_72_61_77, "raw", MediaFormat.MIMETYPE_AUDIO_RAW),
OPUS(0x6f_70_75_73, "opus", MediaFormat.MIMETYPE_AUDIO_OPUS),
AAC(0x00_61_61_63, "aac", MediaFormat.MIMETYPE_AUDIO_AAC);
AAC(0x00_61_61_63, "aac", MediaFormat.MIMETYPE_AUDIO_AAC),
RAW(0x00_72_61_77, "raw", MediaFormat.MIMETYPE_AUDIO_RAW);
private final int id; // 4-byte ASCII representation of the name
private final String name;

View File

@@ -14,7 +14,7 @@ import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
public final class AudioEncoder implements AudioRecorder {
public final class AudioEncoder implements AsyncProcessor {
private static class InputTask {
private final int index;

View File

@@ -5,7 +5,7 @@ import android.media.MediaCodec;
import java.io.IOException;
import java.nio.ByteBuffer;
public final class AudioRawRecorder implements AudioRecorder {
public final class AudioRawRecorder implements AsyncProcessor {
private final Streamer streamer;

View File

@@ -1,12 +0,0 @@
package com.genymobile.scrcpy;
/**
* A component able to record audio asynchronously
*
* The implementation is responsible to send packets.
*/
public interface AudioRecorder {
void start();
void stop();
void join() throws InterruptedException;
}

View File

@@ -14,7 +14,7 @@ import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
public class Controller {
public class Controller implements AsyncProcessor {
private static final int DEFAULT_DEVICE_ID = 0;

View File

@@ -39,28 +39,28 @@ public final class Ln {
public static void v(String message) {
if (isEnabled(Level.VERBOSE)) {
Log.v(TAG, message);
System.out.println(PREFIX + "VERBOSE: " + message);
System.out.print(PREFIX + "VERBOSE: " + message + '\n');
}
}
public static void d(String message) {
if (isEnabled(Level.DEBUG)) {
Log.d(TAG, message);
System.out.println(PREFIX + "DEBUG: " + message);
System.out.print(PREFIX + "DEBUG: " + message + '\n');
}
}
public static void i(String message) {
if (isEnabled(Level.INFO)) {
Log.i(TAG, message);
System.out.println(PREFIX + "INFO: " + message);
System.out.print(PREFIX + "INFO: " + message + '\n');
}
}
public static void w(String message, Throwable throwable) {
if (isEnabled(Level.WARN)) {
Log.w(TAG, message, throwable);
System.out.println(PREFIX + "WARN: " + message);
System.err.print(PREFIX + "WARN: " + message + '\n');
if (throwable != null) {
throwable.printStackTrace();
}
@@ -74,7 +74,7 @@ public final class Ln {
public static void e(String message, Throwable throwable) {
if (isEnabled(Level.ERROR)) {
Log.e(TAG, message, throwable);
System.out.println(PREFIX + "ERROR: " + message);
System.err.print(PREFIX + "ERROR: " + message + "\n");
if (throwable != null) {
throwable.printStackTrace();
}

View File

@@ -13,7 +13,7 @@ public class Options {
private VideoCodec videoCodec = VideoCodec.H264;
private AudioCodec audioCodec = AudioCodec.OPUS;
private int videoBitRate = 8000000;
private int audioBitRate = 196000;
private int audioBitRate = 128000;
private int maxFps;
private int lockVideoOrientation = -1;
private boolean tunnelForward;

View File

@@ -5,6 +5,7 @@ import android.os.BatteryManager;
import android.os.Build;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -91,8 +92,7 @@ public final class Server {
Workarounds.fillAppInfo();
}
Controller controller = null;
AudioRecorder audioRecorder = null;
List<AsyncProcessor> asyncProcessors = new ArrayList<>();
try (DesktopConnection connection = DesktopConnection.open(scid, tunnelForward, audio, control, sendDummyByte)) {
if (options.getSendDeviceMeta()) {
@@ -101,29 +101,34 @@ public final class Server {
}
if (control) {
controller = new Controller(device, connection, options.getClipboardAutosync(), options.getPowerOn());
controller.start();
final Controller controllerRef = controller;
device.setClipboardListener(text -> controllerRef.getSender().pushClipboardText(text));
Controller controller = new Controller(device, connection, options.getClipboardAutosync(), options.getPowerOn());
device.setClipboardListener(text -> controller.getSender().pushClipboardText(text));
asyncProcessors.add(controller);
}
if (audio) {
AudioCodec audioCodec = options.getAudioCodec();
Streamer audioStreamer = new Streamer(connection.getAudioFd(), audioCodec, options.getSendCodecId(), options.getSendFrameMeta());
Streamer audioStreamer = new Streamer(connection.getAudioFd(), audioCodec, options.getSendCodecId(),
options.getSendFrameMeta());
AsyncProcessor audioRecorder;
if (audioCodec == AudioCodec.RAW) {
audioRecorder = new AudioRawRecorder(audioStreamer);
} else {
audioRecorder = new AudioEncoder(audioStreamer, options.getAudioBitRate(), options.getAudioCodecOptions(),
options.getAudioEncoder());
}
audioRecorder.start();
asyncProcessors.add(audioRecorder);
}
Streamer videoStreamer = new Streamer(connection.getVideoFd(), options.getVideoCodec(), options.getSendCodecId(),
options.getSendFrameMeta());
ScreenEncoder screenEncoder = new ScreenEncoder(device, videoStreamer, options.getVideoBitRate(), options.getMaxFps(),
options.getVideoCodecOptions(), options.getVideoEncoder(), options.getDownsizeOnError());
for (AsyncProcessor asyncProcessor : asyncProcessors) {
asyncProcessor.start();
}
try {
// synchronous
screenEncoder.streamScreen();
@@ -136,20 +141,14 @@ public final class Server {
} finally {
Ln.d("Screen streaming stopped");
initThread.interrupt();
if (audioRecorder != null) {
audioRecorder.stop();
}
if (controller != null) {
controller.stop();
for (AsyncProcessor asyncProcessor : asyncProcessors) {
asyncProcessor.stop();
}
try {
initThread.join();
if (audioRecorder != null) {
audioRecorder.join();
}
if (controller != null) {
controller.join();
for (AsyncProcessor asyncProcessor : asyncProcessors) {
asyncProcessor.join();
}
} catch (InterruptedException e) {
// ignore