Compare commits

..

111 Commits

Author SHA1 Message Date
Romain Vimont
c1e9d837e0 Warn on ignored audio options
For raw audio codec, some audio options are ignored.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
fd5000a3c2 Add --audio-codec=raw option
Add support for raw (PCM S16 LE) audio codec (a raw decoder is included
in FFmpeg).

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
7292f0f013 Add raw audio recorder
Add an alternative AudioRecorder to stream raw packets without encoding.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
7c6b7cd952 Extract audio recorder interface
In order to support both encoded and raw audio stream, extract a
interface (very minimal, but sufficient to just start and stop).

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
df12314656 Extract audio capture
The audio capture was implemented in AudioEncoder.

In order to reuse it without encoding, extract it to a separate class.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
f71f7e7edd Stop on decoder frame push error
On push, frame sinks report downstream errors to stop upstream
components. Do not ignore the error.
2023-03-06 09:58:02 +01:00
Romain Vimont
2f84a4012b Add --audio-buffer
Expose an option to add a buffering delay (in milliseconds) before
playing audio.

This is similar to the options --display-buffer and --v4l2-buffer for
video frames.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
d93680976b Optionally do not delay the first frame
A delay buffer delayed all the frames except the first one, to open the
scrcpy window immediately and get a picture.

Make this feature optional, so that the delay buffer might also be used
for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
02c3c6148f Accept clock estimation with a single point
If there is only one point, assume the slope is 1.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
a417285c41 Use delay buffer as a frame source/sink
The components needing delayed frames (sc_screen and sc_v4l2_sink)
managed a sc_video_buffer instance, which itself embedded a
sc_frame_buffer instance (to keep only the most recent frame).

In theory, these components should not be aware of delaying: they should
just receive AVFrames later, and only handle a sc_frame_buffer.

Therefore, refactor sc_delay_buffer as a frame source (it consumes)
frames) and a frame sink (it produces frames, after some delay), and
plug an instance in the pipeline only when a delay is requested.

This also removes the need for a specific sc_video_buffer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
9b19de4262 Use frame source trait in decoder 2023-03-06 09:58:02 +01:00
Romain Vimont
3cbe9c0cc2 Introduce frame source trait
There was a frame sink trait, implemented by components able to receive
AVFrames, but each frame source had to manually send to frame sinks.

In order to mutualise sink management, add a frame sink trait.
2023-03-06 09:58:02 +01:00
Romain Vimont
a30e662107 Use packet source trait in demuxer 2023-03-06 09:58:02 +01:00
Romain Vimont
356c247368 Introduce packet source trait
There was a packet sink trait, implemented by components able to
receive AVPackets, but each packet source had to manually send to packet
sinks.

In order to mutualise sink management, add a packet source trait.
2023-03-06 09:58:02 +01:00
Romain Vimont
a70ca0996c Extract sc_delay_buffer
A video buffer had 2 responsibilities:
 - handle the frame delaying mechanism (queuing packets and pushing them
   after the expected delay);
 - keep only the most recent frame (using a sc_frame_buffer).

In order to reuse only the frame delaying mechanism, extract it to a
separate component, sc_delay_buffer.
2023-03-06 09:58:02 +01:00
Romain Vimont
e97cd27af2 Report video buffer downstream errors
Make the video buffer stop if its consumer could not receive a frame.
2023-03-06 09:58:02 +01:00
Romain Vimont
16eece8093 Stop the video buffer on error
If an error occurs from the video buffer thread (typically an
out-of-memory error), then stop.
2023-03-06 09:58:02 +01:00
Romain Vimont
a4f9cca08c Fix possible race condition on video_buffer end
The video_buffer thread clears the queue once it is stopped, but new
frames might still be pushed asynchronously.

To avoid the problem, do not push any frame once the video_buffer is
stopped.
2023-03-06 09:58:02 +01:00
Romain Vimont
c5cca0302d Remove sc_queue
All uses have been replaced by VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
c92a037029 Remove cbuf
All uses have been replaced by VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
a47debe016 Use VecDeque in aoa_hid
Replace cbuf by VecDeque in aoa_hid
2023-03-06 09:58:02 +01:00
Romain Vimont
ba44947066 Use VecDeque in file_pusher
Replace cbuf by VecDeque in file_pusher.

As a side-effect, the new implementation does not limit the queue to an
arbitrary value.
2023-03-06 09:58:02 +01:00
Romain Vimont
b65bd2b87c Use VecDeque in controller
Replace cbuf by VecDeque in controller.
2023-03-06 09:58:02 +01:00
Romain Vimont
9bb7d0f3d6 Use VecDeque in video_buffer
The packets queued for buffering were wrapped in a dynamically allocated
structure with a "next" field.

To avoid this additional layer of allocation and indirection, use a
VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
9389b6fc65 Use VecDeque in recorder
The packets queued for recording were wrapped in a dynamically allocated
structure with a "next" field.

To avoid this additional layer of allocation and indirection, use a
VecDeque.
2023-03-06 09:58:02 +01:00
Romain Vimont
2e041db876 Introduce VecDeque
Introduce a double-ended queue implemented with a growable ring buffer.

Inspired from the Rust VecDeque type:
<https://doc.rust-lang.org/std/collections/struct.VecDeque.html>
2023-03-06 09:58:02 +01:00
Romain Vimont
0b39c613a8 Add sc_allocarray() util
Add a function to allocate an array, which fails safely in the case
where the multiplication would overflow.
2023-03-06 09:58:02 +01:00
Romain Vimont
7e725ae55f Use reallocarray() in sc_vector
This fails safely in case of overflow.
2023-03-06 09:58:02 +01:00
Romain Vimont
cd99f1b643 Add compat for reallocarray()
This function fails safely in the case where the multiplication would
overflow.
2023-03-06 09:58:02 +01:00
Romain Vimont
e4cfad2cd2 Call avcodec_receive_frame() in a loop
Since in scrcpy a video packet passed to avcodec_send_packet() is always
a complete video frame, it is sufficient to call avcodec_receive_frame()
exactly once.

In practice, it also works for audio packets: the decoder produces
exactly 1 frame for 1 input packet.

In theory, it is an implementation detail though, so
avcodec_receive_frame() should be called in a loop.
2023-03-06 09:58:02 +01:00
Romain Vimont
aeea39e589 Add --require-audio
By default, scrcpy mirrors only the video when audio capture fails on
the device. Add a flag to force scrcpy to fail if audio is enabled but
does not work.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
af577f23fe Add compat support for FFmpeg < 5.1
The new chlayout API has been introduced in FFmpeg 5.1. Use the old
channel_layout API on older versions.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Simon Chan
0af893fd73 Add workaround to capture audio on Android 11
On Android 11, it is possible to start the capture only when the running
app is in foreground. But scrcpy is not an app, it's a Java application
started from shell.

As a workaround, start an existing Android shell existing activity just
to start the capture, then close it immediately.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-06 09:58:02 +01:00
Romain Vimont
047b54b9fe Add audio player
Play the decoded audio using SDL.

The audio player frame sink receives the audio frames, resample them
and write them to a byte buffer (introduced by this commit).

On SDL audio callback (from an internal SDL thread), copy samples from
this byte buffer to the SDL audio buffer.

The byte buffer is protected by the SDL_AudioDeviceLock(), but it has
been designed so that the producer and the consumer may write and read
in parallel, provided that they don't access the same slices of the
ring-buffer buffer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-03-06 09:58:02 +01:00
Romain Vimont
a01a257ddb Add two-step write feature to bytebuf
If there is exactly one producer, then it can assume that the remaining
space in the buffer will only increase until it write something.

This assumption may allow the producer to write to the buffer (up to a
known safe size) without any synchronization mechanism, thus allowing
to read and write different parts of the buffer in parallel.

The producer can then commit the write with lock held, and update its
knowledge of the safe empty remaining space.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
8bf9f200cc Introduce bytebuf util
Add a ring-buffer for bytes. It will be useful for buffering audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
134f48534a Pass AVCodecContext to frame sinks
Frame consumers may need details about the frame format.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
37e01e331f Add an audio decoder
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
59fc45b7cc Give a name to decoder instances
This will be useful in logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
84755698fd Rename decoder to video_decoder
This prepares the introduction of audio_decoder.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
cfdb85de1b Log display sizes in display list
This is more convenient than just the display id alone.
2023-03-06 09:58:02 +01:00
Romain Vimont
04b662b2f0 Add --list-displays 2023-03-06 09:58:02 +01:00
Romain Vimont
b00c393cd0 Move log message helpers to LogUtils
This class will also contain other log helpers.
2023-03-06 09:58:02 +01:00
Romain Vimont
94fef53781 Quit on audio configuration failure
When audio capture fails on the device, scrcpy continue mirroring the
video stream. This allows to enable audio by default only when
supported.

However, if an audio configuration occurs (for example the user
explicitly selected an unknown audio encoder), this must be treated as
an error and scrcpy must exit.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
9670645818 Add --list-encoders
Add an option to list the device encoders properly.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
42aa59f2c2 Move await_for_server() logs
Print the logs on the caller side. This will allow to call the function
in another context without printing the logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:02 +01:00
Romain Vimont
ea35c23b6b Add --audio-encoder
Similar to --video-encoder, but for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:58:01 +01:00
Romain Vimont
aa9860a9f9 Extract unknown encoder error message
This will allow to reuse the same code for audio encoder selection.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:57:42 +01:00
Romain Vimont
400173c280 Add --audio-codec-options
Similar to --video-codec-options, but for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:57:36 +01:00
Romain Vimont
3cd76e0bac Extract application of codec options
This will allow to reuse the same code for audio codec options.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
0cd6e4d5dd Add support for AAC audio codec
Add option --audio-codec=aac.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
56c65bd4c6 Add --audio-codec
Introduce the selection mechanism. Alternative codecs will be added
later.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
78dcc01dcf Add --audio-bit-rate
Add an option to configure the audio bit-rate.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
b9e0803806 Disable MethodLength checkstyle on createOptions()
This method will grow as needed to initialize options.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
879f086e6a Rename --encoder to --video-encoder
This prepares the introduction of --audio-encoder.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
0828631b16 Rename --codec-options to --video-codec-options
This prepares the introduction of --audio-codec-options.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
7653158e5a Rename --bit-rate to --video-bit-rate
This prepares the introduction of --audio-bit-rate.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
42082f622f Rename --codec to --video-codec
This prepares the introduction of --audio-codec.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
5a02005fc3 Remove default bit-rate on client side
If no bit-rate is passed, let the server use the default value (8Mbps).

This avoids to define a default value on both sides, and to pass the
default bit-rate as an argument when starting the server.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
8c680d391d Record at least video packets on stop
If the recorder is stopped while it has not received any audio packet
yet, make sure the video stream is correctly recorded.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
a9ea8b7c1f Disable audio before Android 11
The permission "android.permission.RECORD_AUDIO" has been added for
shell in Android 11.

Moreover, on lower versions, it may make the server segfault on the
device (happened on a Nexus 5 with Android 6.0.1).

Refs <4feeee8891%5E%21/>
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
4be7925667 Disable audio on initialization error
By default, audio is enabled (--no-audio must be explicitly passed to
disable it).

However, some devices may not support audio capture (typically devices
below Android 11, or Android 11 when the shell application is not
foreground on start).

In that case, make the server notify the client to dynamically disable
audio forwarding so that it does not wait indefinitely for an audio
stream.

Also disable audio on unknown codec or missing decoder on the
client-side, for the same reasons.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
43d6d0d4bc Add record audio support
Make the recorder accept two input sources (video and audio), and mux
them into a single file.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
7a4fe1e8f6 Rename video-specific variables in recorder
This paves the way to add audio-specific variables.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
75cd0ea3b5 Do not merge config audio packets
For video streams (at least H.264 and H.265), the config packet
containing SPS/PPS must be prepended to the next packet (the following
keyframe).

For audio streams (at least OPUS), they must not be merged.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
1f9523dd67 Add an audio demuxer
Add a demuxer which will read the stream from the audio socket.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
f0b74e2ed8 Force --no-audio if no display and no recording
The client does not use the audio stream if there is no display and no
recording (i.e. only V4L2), so disable audio so that the device does not
attempt to capture it.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
6da741177f Give a name to demuxer instances
This will be useful in logs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
f8231417aa Rename demuxer to video_demuxer
There will be another demuxer instance for audio.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
f230db9476 Extract OPUS extradata
For OPUS codec, FFmpeg expects the raw extradata, but MediaCodec wraps
it in some structure.

Fix the config packet to send only the raw extradata.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
24a904800f Use a streamer to send the audio stream
Send each encoded audio packet using a streamer.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:26:29 +01:00
Romain Vimont
9bc0998c09 Encode recorded audio on the device
For now, the encoded packets are just logged into the console.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-06 09:25:46 +01:00
Romain Vimont
cd7bdabc84 Make streamer more generic
Expose a method to write a packet from raw metadata (without
BufferInfo).
2023-03-05 21:08:42 +01:00
Simon Chan
901fdc6bf6 Capture device audio
Create an AudioRecorder to capture the audio source REMOTE_SUBMIX.

For now, the captured packets are just logged into the console.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-05 18:13:25 +01:00
Simon Chan
2d7630882a Add a new socket for audio stream
When audio is enabled, open a new socket to send the audio stream from
the device to the client.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-05 18:13:25 +01:00
Simon Chan
0457253655 Add --no-audio option
Audio will be enabled by default (when supported). Add an option to
disable it.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-03 21:43:57 +01:00
Romain Vimont
484f38be1c Use FakeContext for Application instance
This will expose the correct package name and UID to the application
context.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-03 21:43:54 +01:00
Romain Vimont
013bf96cd0 Use shell package name for workarounds
For consistency.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-03 21:43:50 +01:00
Romain Vimont
3c090b3d9e Use ROOT_UID from FakeContext
Remove USER_ID from ServiceManager, and replace it by a constant in
FakeContext.

This is the same as android.os.Process.ROOT_UID, but this constant has
been introduced in API 29.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-03 21:43:44 +01:00
Romain Vimont
31068ee607 Use PACKAGE_NAME from FakeContext
Remove duplicated constant.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-03 21:43:41 +01:00
Romain Vimont
e23366fb4e Use AttributionSource from FakeContext
FakeContext already provides an AttributeSource instance.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2023-03-03 21:43:33 +01:00
Simon Chan
b5ae5bf6bb Add a fake Android Context
Since scrcpy-server is not an Android application (it's a java
executable), it has no Context.

Some features will require a Context instance to get the package name
and the UID. Add a FakeContext for this purpose.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-03 21:43:11 +01:00
Romain Vimont
e068fe43cf Improve error message for unknown encoder
The provided encoder name depends on the selected codec. Improve the
error message and the suggestions.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-03 21:43:06 +01:00
Romain Vimont
883a998c10 Rename "codec" variable to "mediaCodec"
This will allow to use "codec" for the Codec type.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-03 21:43:04 +01:00
Romain Vimont
3c670dc52a Make streamer independent of codec type
Rename VideoStreamer to Streamer, and extract a Codec interface which
will also support audio codecs.

PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
2023-03-03 21:42:47 +01:00
Romain Vimont
af1f00bece Pass all args to ScreenEncoder constructor
There is no good reason to pass some of them in the constructor and some
others as parameters of the streamScreen() method.
2023-03-03 21:09:21 +01:00
Romain Vimont
7853c4c303 Move screen encoder initialization
This prepares further refactors.
2023-03-03 21:09:21 +01:00
Romain Vimont
bbb025bcf3 Write streamer header from ScreenEncoder
The screen encoder is responsible to write data to the video streamer.
2023-03-03 21:09:21 +01:00
Romain Vimont
9e4d7f59d7 Use VideoStreamer directly from ScreenEncoder
The Callbacks interface notifies new packets. But in addition, the
screen encoder will need to write headers on start.

We could add a function onStart(), but for simplicity, just remove the
interface, which brings no value, and call the streamer directly.

Refs 87972e2022
2023-03-03 21:09:21 +01:00
Romain Vimont
89c3de5498 Simplify error handling on socket creation
On any error, all previously opened sockets must be closed.

Handle these errors in a single catch-block. Currently, there are only 2
sockets, but this will simplify even more with more sockets.

Note: this commit is better displayed with --ignore-space-change (-b).
2023-03-03 21:09:21 +01:00
Romain Vimont
0d8644d3ff Reorder initialization
Initialize components in the pipeline order: demuxer first, decoder and
recorder second.
2023-03-03 21:09:21 +01:00
Romain Vimont
f0660df102 Refactor recorder logic
Process the initial config packet (necessary to write the header)
separately.
2023-03-03 21:09:21 +01:00
Romain Vimont
ddd9c8b4a8 Move last packet recording
Write the last packet at the end.
2023-03-03 21:09:21 +01:00
Romain Vimont
ed14c56be4 Add start() function for recorder
For consistency with the other components, do not start the internal
thread from an init() function.
2023-03-03 21:09:21 +01:00
Romain Vimont
0af71d2bd8 Open recording file from the recorder thread
The recorder opened the target file from the packet sink open()
callback, called by the demuxer. Only then the recorder thread was
started.

One golden rule for the recorder is to never block the demuxer for I/O,
because it would impact mirroring. This rule is respected on recording
packets, but not for the initial recorder opening.

Therefore, start the recorder thread from sc_recorder_init(), open the
file immediately from the recorder thread, then make it wait for the
stream to start (on packet sink open()).

Now that the recorder can report errors directly (rather than making the
demuxer call fail), it is possible to report file opening error even
before the packet sink is open.
2023-03-03 21:09:21 +01:00
Romain Vimont
e1deb7077c Inline packet_sink impl in recorder
Remove useless wrappers.
2023-03-03 21:09:21 +01:00
Romain Vimont
92b5c297b4 Initialize recorder fields from init()
The recorder has two initialization phases: one to initialize the
concrete recorder object, and one to open its packet_sink trait.

Initialize mutex and condvar as part of the object initialization.

If there were several packet_sink traits, the mutex and condvar would
still be initialized only once.
2023-03-03 21:09:21 +01:00
Romain Vimont
92bc1a37ae Report recorder errors
Stop scrcpy on recorder errors.

It was previously indirectly stopped by the demuxer, which failed to
push packets to a recorder in error. Report it directly instead:
 - it avoids to wait for the next demuxer call;
 - it will allow to open the target file from a separate thread and stop
   immediately on any I/O error.
2023-03-03 21:09:21 +01:00
Romain Vimont
37c9c3cb50 Move previous packet to a local variable
It is only used from run_recorder().
2023-03-03 21:09:21 +01:00
Romain Vimont
83c20a10db Move pts_origin to a local variable
It is only used from run_recorder().
2023-03-03 21:09:21 +01:00
Romain Vimont
c6cd4ff8fe Change PTS origin type from uint64_t to int64_t
It is initialized from AVPacket.pts, which is an int64_t.
2023-03-03 21:09:21 +01:00
Romain Vimont
8eef96012b Fix --encoder documentation
Mention that it depends on the codec provided by --codec (which is not
necessarily H264 anymore).
2023-03-03 21:09:21 +01:00
Romain Vimont
3619efa7d4 Do not print stacktraces when unnecessary
User-friendly error messages are printed on specific configuration
exceptions. In that case, do not print the stacktrace.

Also handle the user-friendly error message directly where the error
occurs, and print multiline messages in a single log call, to avoid
confusing interleaving.
2023-03-03 21:09:21 +01:00
Romain Vimont
4fecf4e49e Fix --no-clipboard-autosync bash completion
Fix typo.
2023-03-03 21:09:21 +01:00
Romain Vimont
3015224135 Split server stop() and join()
For consistency with the other components, call stop() and join()
separately.

This allows to stop all components, then join them all.
2023-03-03 21:09:21 +01:00
Romain Vimont
2381a61798 Print FFmpeg logs
FFmpeg logs are redirected to a specific SDL log category.

Initialize the log level for this category to print them as expected.
2023-03-03 21:09:21 +01:00
Romain Vimont
3ab2840fd5 Move FFmpeg callback initialization
Configure FFmpeg log redirection on start from a log helper.
2023-03-03 21:09:21 +01:00
Romain Vimont
2bdee9b31c Upgrade FFmpeg custom builds for Windows
Use a build which includes the pcm_s16le decoder, to support RAW audio.

Refs <https://github.com/rom1v/scrcpy-deps/commits/6.0-scrcpy-2>
2023-03-03 21:09:21 +01:00
Romain Vimont
8131d1f00e Upgrade FFmpeg (6.0) for Windows
Use the latest version (specifically built for scrcpy).

Refs <https://www.ffmpeg.org/download.html#release_6.0>
2023-03-03 21:09:20 +01:00
Romain Vimont
17b9d149cf Use minimal prebuilt FFmpeg for Windows
On the scrcpy-deps repo, I built FFmpeg 5.1.2 binaries for Windows with
only the features used by scrcpy.

For comparison, here are the sizes of the dll for FFmpeg 5.1.2:
 - before: 89M
 - after: 4.7M

It also allows to upgrade the old FFmpeg version (4.3.1) used for win32.

Refs <https://github.com/rom1v/scrcpy-deps>
Refs <https://github.com/Genymobile/scrcpy/issues/1753>
2023-03-03 21:08:42 +01:00
Romain Vimont
c8b0e3682c Simplify libusb prebuilt scripts
In theory, include/ might be slightly different for win32 and win64
builds. Use each one separately to simplify.
2023-03-03 21:06:03 +01:00
32 changed files with 323 additions and 442 deletions

View File

@@ -78,7 +78,7 @@ _scrcpy() {
return
;;
--audio-codec)
COMPREPLY=($(compgen -W 'opus aac raw' -- "$cur"))
COMPREPLY=($(compgen -W 'raw opus aac' -- "$cur"))
return
;;
--lock-video-orientation)

View File

@@ -10,7 +10,7 @@ local arguments
arguments=(
'--always-on-top[Make scrcpy window always on top \(above other windows\)]'
'--audio-bit-rate=[Encode the audio at the given bit-rate]'
'--audio-codec=[Select the audio codec]:codec:(opus aac raw)'
'--audio-codec=[Select the audio codec]:codec:(raw opus aac)'
'--audio-codec-options=[Set a list of comma-separated key\:type=value options for the device audio encoder]'
'--audio-encoder=[Use a specific MediaCodec audio encoder]'
{-b,--video-bit-rate=}'[Encode the video at the given bit-rate]'

View File

@@ -23,19 +23,17 @@ Make scrcpy window always on top (above other windows).
.BI "\-\-audio\-bit\-rate " value
Encode the audio at the given bit\-rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
Default is 128K (128000).
Default is 196K (196000).
.TP
.BI "\-\-audio\-buffer ms
Configure the audio buffering delay (in milliseconds).
Add a buffering delay (in milliseconds) before playing audio. This increases latency to compensate for jitter.
Lower values decrease the latency, but increase the likelyhood of buffer underrun (causing audio glitches).
Default is 50.
Default is 0 (no buffering).
.TP
.BI "\-\-audio\-codec " name
Select an audio codec (opus, aac or raw).
Select an audio codec (raw, opus or aac).
Default is opus.
@@ -282,7 +280,7 @@ Supported names are currently "direct3d", "opengl", "opengles2", "opengles", "me
.TP
.B \-\-require\-audio
By default, scrcpy mirrors only the video if audio capture fails on the device. This option makes scrcpy fail if audio is enabled but does not work.
By default, scrcpy mirrors only the video if audio capture fails on the device. This flag makes scrcpy fail if audio is enabled but does not work.
.TP
.BI "\-\-rotation " value

View File

@@ -4,7 +4,7 @@
#include "util/log.h"
#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
//#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
/** Downcast frame_sink to sc_audio_player */
#define DOWNCAST(SINK) container_of(SINK, struct sc_audio_player, frame_sink)
@@ -12,20 +12,29 @@
#define SC_AV_SAMPLE_FMT AV_SAMPLE_FMT_FLT
#define SC_SDL_SAMPLE_FMT AUDIO_F32
#define SC_AUDIO_OUTPUT_BUFFER_SAMPLES 240 // 5ms at 48000Hz
#define SC_AUDIO_OUTPUT_BUFFER_SAMPLES 480 // 10ms at 48000Hz
static inline uint32_t
// The target number of buffered samples between the producer and the consumer.
// This value is directly use for compensation.
#define SC_TARGET_BUFFERED_SAMPLES (3 * SC_AUDIO_OUTPUT_BUFFER_SAMPLES)
// Use a ring-buffer of 1 second (at 48000Hz) between the producer and the
// consumer. It too big, but it guarantees that the producer and the consumer
// will be able to access it in parallel without locking.
#define SC_BYTEBUF_SIZE_IN_SAMPLES 48000
static inline size_t
bytes_to_samples(struct sc_audio_player *ap, size_t bytes) {
assert(bytes % (ap->nb_channels * ap->out_bytes_per_sample) == 0);
return bytes / (ap->nb_channels * ap->out_bytes_per_sample);
}
static inline size_t
samples_to_bytes(struct sc_audio_player *ap, uint32_t samples) {
samples_to_bytes(struct sc_audio_player *ap, size_t samples) {
return samples * ap->nb_channels * ap->out_bytes_per_sample;
}
static void SDLCALL
void
sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
struct sc_audio_player *ap = userdata;
@@ -36,56 +45,34 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
size_t len = len_int;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] SDL callback requests %" PRIu32 " samples",
LOGD("[Audio] SDL callback requests %" SC_PRIsizet " samples",
bytes_to_samples(ap, len));
#endif
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
if (!ap->played) {
uint32_t buffered_samples = bytes_to_samples(ap, read_avail);
// Part of the buffering is handled by inserting initial silence. The
// remaining (margin) last samples will be handled by compensation.
uint32_t margin = 30 * ap->sample_rate / 1000; // 30ms
if (buffered_samples + margin < ap->target_buffering) {
LOGV("[Audio] Inserting initial buffering silence: %" PRIu32
" samples", bytes_to_samples(ap, len));
// Delay playback starting to reach the target buffering. Fill the
// whole buffer with silence (len is small compared to the
// arbitrary margin value).
memset(stream, 0, len);
return;
}
}
size_t read = MIN(read_avail, len);
if (read) {
sc_bytebuf_read(&ap->buf, stream, read);
}
if (read < len) {
size_t silence_bytes = len - read;
uint32_t silence_samples = bytes_to_samples(ap, silence_bytes);
// Insert silence. In theory, the inserted silent samples replace the
// missing real samples, which will arrive later, so they should be
// dropped to keep the latency minimal. However, this would cause very
// audible glitches, so let the clock compensation restore the target
// latency.
LOGD("[Audio] Buffer underflow, inserting silence: %" PRIu32 " samples",
silence_samples);
memset(stream + read, 0, silence_bytes);
// Insert silence
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Buffer underflow, inserting silence: %" SC_PRIsizet
" samples", bytes_to_samples(ap, len - read));
#endif
memset(stream + read, 0, len - read);
// If the first frame has not been received yet, it's not an underflow
if (ap->received) {
// Inserting additional samples immediately increases buffering
ap->avg_buffering.avg += silence_samples;
ap->underflow += bytes_to_samples(ap, len - read);
}
}
ap->played = true;
ap->last_consumed = sc_tick_now();
}
static uint8_t *
sc_audio_player_get_swr_buf(struct sc_audio_player *ap, uint32_t min_samples) {
sc_audio_player_get_swr_buf(struct sc_audio_player *ap, size_t min_samples) {
size_t min_buf_size = samples_to_bytes(ap, min_samples);
if (min_buf_size > ap->swr_buf_alloc_size) {
size_t new_size = min_buf_size + 4096;
@@ -102,186 +89,6 @@ sc_audio_player_get_swr_buf(struct sc_audio_player *ap, uint32_t min_samples) {
return ap->swr_buf;
}
static bool
sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
const AVFrame *frame) {
struct sc_audio_player *ap = DOWNCAST(sink);
SwrContext *swr_ctx = ap->swr_ctx;
int64_t swr_delay = swr_get_delay(swr_ctx, ap->sample_rate);
// No need to av_rescale_rnd(), input and output sample rates are the same.
// Add more space (256) for clock compensation.
int dst_nb_samples = swr_delay + frame->nb_samples + 256;
uint8_t *swr_buf = sc_audio_player_get_swr_buf(ap, dst_nb_samples);
if (!swr_buf) {
return false;
}
int ret = swr_convert(swr_ctx, &swr_buf, dst_nb_samples,
(const uint8_t **) frame->data, frame->nb_samples);
if (ret < 0) {
LOGE("Resampling failed: %d", ret);
return false;
}
// swr_convert() returns the number of samples which would have been
// written if the buffer was big enough.
uint32_t samples_written = MIN(ret, dst_nb_samples);
size_t swr_buf_size = samples_to_bytes(ap, samples_written);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] %" PRIu32 " samples written to buffer", samples_written);
#endif
// Since this function is the only writer, the current available space is
// at least the previous available space. In practice, it should almost
// always be possible to write without lock.
bool lockless_write = swr_buf_size <= ap->previous_write_avail;
if (lockless_write) {
sc_bytebuf_prepare_write(&ap->buf, swr_buf, swr_buf_size);
}
SDL_LockAudioDevice(ap->device);
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
uint32_t buffered_samples = bytes_to_samples(ap, read_avail);
if (lockless_write) {
sc_bytebuf_commit_write(&ap->buf, swr_buf_size);
} else {
// Take care to keep full samples
size_t align = ap->nb_channels * ap->out_bytes_per_sample;
size_t write_avail =
sc_bytebuf_write_available(&ap->buf) / align * align;
if (swr_buf_size > write_avail) {
// Entering this branch is very unlikely, the ring-buffer (bytebuf)
// is allocated with a size sufficient to store 1 second more than
// the target buffering. If this happens, though, we have to skip
// old samples.
size_t cap = sc_bytebuf_capacity(&ap->buf) / align * align;
if (swr_buf_size > cap) {
// Very very unlikely: a single resampled frame should never
// exceed the ring-buffer size (or something is very wrong).
// Ignore the first bytes in swr_buf
swr_buf += swr_buf_size - cap;
swr_buf_size = cap;
// This change in samples_written will impact the
// instant_compensation below
samples_written -= bytes_to_samples(ap, swr_buf_size - cap);
}
assert(swr_buf_size >= write_avail);
if (swr_buf_size > write_avail) {
sc_bytebuf_skip(&ap->buf, swr_buf_size - write_avail);
uint32_t skip_samples =
bytes_to_samples(ap, swr_buf_size - write_avail);
assert(buffered_samples >= skip_samples);
buffered_samples -= skip_samples;
if (ap->played) {
// Dropping input samples instantly decreases buffering
ap->avg_buffering.avg -= skip_samples;
}
}
// It should remain exactly the expected size to write the new
// samples.
assert((sc_bytebuf_write_available(&ap->buf) / align * align)
== swr_buf_size);
}
sc_bytebuf_write(&ap->buf, swr_buf, swr_buf_size);
}
buffered_samples += samples_written;
assert(samples_to_bytes(ap, buffered_samples)
== sc_bytebuf_read_available(&ap->buf));
// Read with lock held, to be used after unlocking
bool played = ap->played;
if (played) {
uint32_t max_buffered_samples = ap->target_buffering
+ 12 * SC_AUDIO_OUTPUT_BUFFER_SAMPLES
+ ap->target_buffering / 10;
if (buffered_samples > max_buffered_samples) {
uint32_t skip_samples = buffered_samples - max_buffered_samples;
size_t skip_bytes = samples_to_bytes(ap, skip_samples);
sc_bytebuf_skip(&ap->buf, skip_bytes);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Buffering threshold exceeded, skipping %" PRIu32
" samples", skip_samples);
#endif
}
// Number of samples added (or removed, if negative) for compensation
int32_t instant_compensation =
(int32_t) samples_written - frame->nb_samples;
// The compensation must apply instantly, it must not be smoothed
ap->avg_buffering.avg += instant_compensation;
// However, the buffering level must be smoothed
sc_average_push(&ap->avg_buffering, buffered_samples);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] buffered_samples=%" PRIu32 " avg_buffering=%f",
buffered_samples, sc_average_get(&ap->avg_buffering));
#endif
} else {
// SDL playback not started yet, do not accumulate more than
// max_initial_buffering samples, this would cause unnecessary delay
// (and glitches to compensate) on start.
uint32_t max_initial_buffering = ap->target_buffering
+ 2 * SC_AUDIO_OUTPUT_BUFFER_SAMPLES;
if (buffered_samples > max_initial_buffering) {
uint32_t skip_samples = buffered_samples - max_initial_buffering;
size_t skip_bytes = samples_to_bytes(ap, skip_samples);
sc_bytebuf_skip(&ap->buf, skip_bytes);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Playback not started, skipping %" PRIu32 " samples",
skip_samples);
#endif
}
}
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
ap->received = true;
SDL_UnlockAudioDevice(ap->device);
if (played) {
ap->samples_since_resync += samples_written;
if (ap->samples_since_resync >= ap->sample_rate) {
// Recompute compensation every second
ap->samples_since_resync = 0;
float avg = sc_average_get(&ap->avg_buffering);
int diff = ap->target_buffering - avg;
if (diff < 0 && buffered_samples < ap->target_buffering) {
// Do not accelerate if the instant buffering level is below
// the average, this would increase underflow
diff = 0;
}
// Compensate the diff over 4 seconds (but will be recomputed after
// 1 second)
int distance = 4 * ap->sample_rate;
// Limit compensation rate to 2%
int abs_max_diff = distance / 50;
diff = CLAMP(diff, -abs_max_diff, abs_max_diff);
LOGV("[Audio] Buffering: target=%" PRIu32 " avg=%f cur=%" PRIu32
" compensation=%d", ap->target_buffering, avg,
buffered_samples, diff);
int ret = swr_set_compensation(swr_ctx, diff, distance);
if (ret < 0) {
LOGW("Resampling compensation failed: %d", ret);
// not fatal
}
}
}
return true;
}
static bool
sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
const AVCodecContext *ctx) {
@@ -320,6 +127,7 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
assert(ctx->sample_rate > 0);
assert(!av_sample_fmt_is_planar(SC_AV_SAMPLE_FMT));
int out_bytes_per_sample = av_get_bytes_per_sample(SC_AV_SAMPLE_FMT);
assert(out_bytes_per_sample > 0);
@@ -349,15 +157,7 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
ap->nb_channels = nb_channels;
ap->out_bytes_per_sample = out_bytes_per_sample;
ap->target_buffering = ap->target_buffering_delay * ap->sample_rate
/ SC_TICK_FREQ;
// Use a ring-buffer of the target buffering size plus 1 second between the
// producer and the consumer. It's too big on purpose, to guarantee that
// the producer and the consumer will be able to access it in parallel
// without locking.
size_t bytebuf_samples = ap->target_buffering + ap->sample_rate;
size_t bytebuf_size = samples_to_bytes(ap, bytebuf_samples);
size_t bytebuf_size = samples_to_bytes(ap, SC_BYTEBUF_SIZE_IN_SAMPLES);
bool ok = sc_bytebuf_init(&ap->buf, bytebuf_size);
if (!ok) {
@@ -374,21 +174,12 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
// Samples are produced and consumed by blocks, so the buffering must be
// smoothed to get a relatively stable value.
sc_average_init(&ap->avg_buffering, 32);
sc_average_init(&ap->avg_buffering, 8);
ap->samples_since_resync = 0;
ap->received = false;
ap->played = false;
// The thread calling open() is the thread calling push(), which fills the
// audio buffer consumed by the SDL audio thread.
ok = sc_thread_set_priority(SC_THREAD_PRIORITY_TIME_CRITICAL);
if (!ok) {
ok = sc_thread_set_priority(SC_THREAD_PRIORITY_HIGH);
(void) ok; // We don't care if it worked, at least we tried
}
ap->last_consumed = 0;
ap->underflow = 0;
ap->received = 0;
SDL_PauseAudioDevice(ap->device, 0);
@@ -417,10 +208,166 @@ sc_audio_player_frame_sink_close(struct sc_frame_sink *sink) {
swr_free(&ap->swr_ctx);
}
void
sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering) {
ap->target_buffering_delay = target_buffering;
static bool
sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
const AVFrame *frame) {
struct sc_audio_player *ap = DOWNCAST(sink);
SwrContext *swr_ctx = ap->swr_ctx;
int64_t delay = swr_get_delay(swr_ctx, ap->sample_rate);
// No need to av_rescale_rnd(), input and output sample rates are the same
// Add more space (256) for clock compensation
int dst_nb_samples = delay + frame->nb_samples + 256;
uint8_t *swr_buf = sc_audio_player_get_swr_buf(ap, dst_nb_samples);
if (!swr_buf) {
return false;
}
int ret = swr_convert(swr_ctx, &swr_buf, dst_nb_samples,
(const uint8_t **) frame->data, frame->nb_samples);
if (ret < 0) {
LOGE("Resampling failed: %d", ret);
return false;
}
// swr_convert() returns the number of samples which would have been
// written if the buffer was big enough.
size_t samples_written = MIN(ret, dst_nb_samples);
size_t swr_buf_size = samples_to_bytes(ap, samples_written);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] %" SC_PRIsizet " samples written to buffer", samples_written);
#endif
// Since this function is the only writer, the current available space is
// at least the previous available space. In practice, it should almost
// always be possible to write without lock.
bool lockless_write = swr_buf_size <= ap->previous_write_avail;
if (lockless_write) {
sc_bytebuf_prepare_write(&ap->buf, swr_buf, swr_buf_size);
}
SDL_LockAudioDevice(ap->device);
// The consumer requests audio samples blocks (e.g. 480 samples).
// Convert the duration since the last consumption into samples.
size_t extrapolated = 0;
if (ap->last_consumed) {
sc_tick now = sc_tick_now();
assert(now >= ap->last_consumed);
extrapolated = (now - ap->last_consumed) * ap->sample_rate
/ SC_TICK_FREQ;
}
size_t read_avail = sc_bytebuf_read_available(&ap->buf);
// The consumer may not increase underflow value if there are still samples
// available
assert(read_avail == 0 || ap->underflow == 0);
size_t buffered_samples = bytes_to_samples(ap, read_avail);
// Underflow caused silence samples in excess (so it adds buffering).
// Extrapolated samples must be considered consumed for smoothing (so it
// removes buffering).
float buffering = (float) buffered_samples + ap->underflow - extrapolated;
sc_average_push(&ap->avg_buffering, buffering);
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] buffered_samples=%" SC_PRIsizet
" underflow=%" SC_PRIsizet
" extrapolated=%" SC_PRIsizet
" buffering=%f avg_buffering=%f",
buffered_samples, ap->underflow, extrapolated, buffering,
sc_average_get(&ap->avg_buffering));
#endif
if (lockless_write) {
sc_bytebuf_commit_write(&ap->buf, swr_buf_size);
} else {
// Take care to keep full samples
size_t align = ap->nb_channels * ap->out_bytes_per_sample;
size_t write_avail =
sc_bytebuf_write_available(&ap->buf) / align * align;
if (swr_buf_size > write_avail) {
// Skip old samples
size_t cap = sc_bytebuf_capacity(&ap->buf) / align * align;
if (swr_buf_size > cap) {
// Ignore the first bytes in swr_buf
swr_buf += swr_buf_size - cap;
swr_buf_size = cap;
}
assert(swr_buf_size > write_avail);
if (swr_buf_size - write_avail > 0) {
sc_bytebuf_skip(&ap->buf, swr_buf_size - write_avail);
}
}
sc_bytebuf_write(&ap->buf, swr_buf, swr_buf_size);
}
// On buffer underflow, typically because a packet is late, silence is
// inserted. In that case, the late samples must be ignored when they
// arrive, otherwise they will delay playback.
//
// As an improvement, instead of naively skipping the silence duration, we
// can absorb it if it helps clock compensation.
if (ap->underflow) {
size_t avg = sc_average_get(&ap->avg_buffering);
if (avg > SC_TARGET_BUFFERED_SAMPLES) {
size_t diff = SC_TARGET_BUFFERED_SAMPLES - avg;
if (ap->underflow > diff) {
// Partially absorb underflow for clock compensation (only keep
// the diff with the target buffering level).
ap->underflow -= diff;
} else {
// Totally absorb underflow for clock compensation
ap->underflow = 0;
}
size_t skip_samples = MIN(ap->underflow, buffered_samples);
if (skip_samples) {
size_t skip_bytes = samples_to_bytes(ap, skip_samples);
sc_bytebuf_skip(&ap->buf, skip_bytes);
read_avail -= skip_bytes;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Skipping %" SC_PRIsizet " samples", skip_samples);
#endif
}
} else {
// Totally absorb underflow for clock compensation
ap->underflow = 0;
}
}
ap->previous_write_avail = sc_bytebuf_write_available(&ap->buf);
ap->received = true;
SDL_UnlockAudioDevice(ap->device);
ap->samples_since_resync += samples_written;
if (ap->samples_since_resync >= ap->sample_rate) {
// Resync every second
ap->samples_since_resync = 0;
float avg = sc_average_get(&ap->avg_buffering);
int diff = SC_TARGET_BUFFERED_SAMPLES - avg;
#ifndef SC_AUDIO_PLAYER_NDEBUG
LOGD("[Audio] Average buffering=%f, compensation %d", avg, diff);
#endif
// Compensate the diff over 3 seconds (but will be recomputed after
// 1 second)
int ret = swr_set_compensation(swr_ctx, diff, 3 * ap->sample_rate);
if (ret < 0) {
LOGW("Resampling compensation failed: %d", ret);
// not fatal
}
}
return true;
}
void
sc_audio_player_init(struct sc_audio_player *ap) {
static const struct sc_frame_sink_ops ops = {
.open = sc_audio_player_frame_sink_open,
.close = sc_audio_player_frame_sink_close,

View File

@@ -8,7 +8,6 @@
#include <util/average.h>
#include <util/bytebuf.h>
#include <util/thread.h>
#include <util/tick.h>
#include <libavformat/avformat.h>
#include <libswresample/swresample.h>
@@ -19,23 +18,10 @@ struct sc_audio_player {
SDL_AudioDeviceID device;
// The target buffering between the producer and the consumer. This value
// is directly use for compensation.
// Since audio capture and/or encoding on the device typically produce
// blocks of 960 samples (20ms) or 1024 samples (~21.3ms), this target
// value should be higher.
sc_tick target_buffering_delay;
uint32_t target_buffering; // in samples
// Audio buffer to communicate between the receiver and the SDL audio
// callback (protected by SDL_AudioDeviceLock())
// protected by SDL_AudioDeviceLock()
struct sc_bytebuf buf;
// The previous number of bytes available in the buffer (only used by the
// receiver thread)
size_t previous_write_avail;
// Resampler (only used from the receiver thread)
struct SwrContext *swr_ctx;
// The sample rate is the same for input and output
@@ -45,25 +31,22 @@ struct sc_audio_player {
// The number of bytes per sample for a single channel
unsigned out_bytes_per_sample;
// Target buffer for resampling (only used by the receiver thread)
// Target buffer for resampling
uint8_t *swr_buf;
size_t swr_buf_alloc_size;
// Number of buffered samples (may be negative on underflow) (only used by
// the receiver thread)
// Number of buffered samples (may be negative on underflow)
struct sc_average avg_buffering;
// Count the number of samples to trigger a compensation update regularly
// (only used by the receiver thread)
uint32_t samples_since_resync;
size_t samples_since_resync;
// Set to true the first time a sample is received (protected by
// SDL_AudioDeviceLock())
// The last date a sample has been consumed by the audio output
sc_tick last_consumed;
// Number of silence samples inserted to be compensated
size_t underflow;
bool received;
// Set to true the first time the SDL callback is called (protected by
// SDL_AudioDeviceLock())
bool played;
const struct sc_audio_player_callbacks *cbs;
void *cbs_userdata;
};
@@ -73,6 +56,6 @@ struct sc_audio_player_callbacks {
};
void
sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering);
sc_audio_player_init(struct sc_audio_player *ap);
#endif

View File

@@ -19,7 +19,6 @@
enum {
OPT_RENDER_EXPIRED_FRAMES = 1000,
OPT_BIT_RATE,
OPT_WINDOW_TITLE,
OPT_PUSH_TARGET,
OPT_ALWAYS_ON_TOP,
@@ -119,22 +118,21 @@ static const struct sc_option options[] = {
.argdesc = "value",
.text = "Encode the audio at the given bit-rate, expressed in bits/s. "
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is 128K (128000).",
"Default is 196K (196000).",
},
{
.longopt_id = OPT_AUDIO_BUFFER,
.longopt = "audio-buffer",
.argdesc = "ms",
.text = "Configure the audio buffering delay (in milliseconds).\n"
"Lower values decrease the latency, but increase the "
"likelyhood of buffer underrun (causing audio glitches).\n"
"Default is 50.",
.text = "Add a buffering delay (in milliseconds) before playing audio. "
"This increases latency to compensate for jitter.\n"
"Default is 0 (no buffering).",
},
{
.longopt_id = OPT_AUDIO_CODEC,
.longopt = "audio-codec",
.argdesc = "name",
.text = "Select an audio codec (opus, aac or raw).\n"
.text = "Select an audio codec (raw, opus or aac).\n"
"Default is opus.",
},
{
@@ -165,12 +163,6 @@ static const struct sc_option options[] = {
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is 8M (8000000).",
},
{
// deprecated
.longopt_id = OPT_BIT_RATE,
.longopt = "bit-rate",
.argdesc = "value",
},
{
// Not really deprecated (--codec has never been released), but without
// declaring an explicit --codec option, getopt_long() partial matching
@@ -480,8 +472,8 @@ static const struct sc_option options[] = {
.longopt_id = OPT_REQUIRE_AUDIO,
.longopt = "require-audio",
.text = "By default, scrcpy mirrors only the video when audio capture "
"fails on the device. This option makes scrcpy fail if audio "
"is enabled but does not work."
"fails on the device. This flag makes scrcpy fail if audio is "
"enabled but does not work."
},
{
.longopt_id = OPT_ROTATION,
@@ -1514,6 +1506,10 @@ parse_video_codec(const char *optarg, enum sc_codec *codec) {
static bool
parse_audio_codec(const char *optarg, enum sc_codec *codec) {
if (!strcmp(optarg, "raw")) {
*codec = SC_CODEC_RAW;
return true;
}
if (!strcmp(optarg, "opus")) {
*codec = SC_CODEC_OPUS;
return true;
@@ -1522,11 +1518,7 @@ parse_audio_codec(const char *optarg, enum sc_codec *codec) {
*codec = SC_CODEC_AAC;
return true;
}
if (!strcmp(optarg, "raw")) {
*codec = SC_CODEC_RAW;
return true;
}
LOGE("Unsupported audio codec: %s (expected opus, aac or raw)", optarg);
LOGE("Unsupported audio codec: %s (expected raw, opus or aac)", optarg);
return false;
}
@@ -1540,9 +1532,6 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
int c;
while ((c = getopt_long(argc, argv, optstring, longopts, NULL)) != -1) {
switch (c) {
case OPT_BIT_RATE:
LOGW("--bit-rate is deprecated, use --video-bit-rate instead.");
// fall through
case 'b':
if (!parse_bit_rate(optarg, &opts->video_bit_rate)) {
return false;
@@ -1944,6 +1933,18 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
}
}
if (opts->audio_codec == SC_CODEC_RAW) {
if (opts->audio_bit_rate) {
LOGW("--audio-bit-rate is ignored for raw audio codec");
}
if (opts->audio_codec_options) {
LOGW("--audio-codec-options is ignored for raw audio codec");
}
if (opts->audio_encoder) {
LOGW("--audio-encoder is ignored for raw audio codec");
}
}
if (!opts->control) {
if (opts->turn_screen_off) {
LOGE("Could not request to turn screen off if control is disabled");

View File

@@ -105,7 +105,8 @@ sc_clock_update(struct sc_clock *clock, sc_tick system, sc_tick stream) {
sc_clock_estimate(clock, &clock->slope, &clock->offset);
#ifndef SC_CLOCK_NDEBUG
LOGD("Clock estimation: %f * pts + %" PRItick, clock->slope, clock->offset);
LOGD("Clock estimation: %f * pts + %" PRItick,
clock->slope, clock->offset);
#endif
}

View File

@@ -54,10 +54,6 @@
# define SCRCPY_SDL_HAS_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR
#endif
#if SDL_VERSION_ATLEAST(2, 0, 16)
# define SCRCPY_SDL_HAS_THREAD_PRIORITY_TIME_CRITICAL
#endif
#ifndef HAVE_STRDUP
char *strdup(const char *s);
#endif

View File

@@ -58,7 +58,7 @@ run_buffering(void *data) {
sc_tick max_deadline = sc_tick_now() + db->delay;
// PTS (written by the server) are expressed in microseconds
sc_tick pts = SC_TICK_FROM_US(dframe.frame->pts);
sc_tick pts = SC_TICK_TO_US(dframe.frame->pts);
bool timed_out = false;
while (!db->stopped && !timed_out) {

View File

@@ -23,9 +23,9 @@ sc_demuxer_to_avcodec_id(uint32_t codec_id) {
#define SC_CODEC_ID_H264 UINT32_C(0x68323634) // "h264" in ASCII
#define SC_CODEC_ID_H265 UINT32_C(0x68323635) // "h265" in ASCII
#define SC_CODEC_ID_AV1 UINT32_C(0x00617631) // "av1" in ASCII
#define SC_CODEC_ID_RAW UINT32_C(0x00726177) // "raw" in ASCII
#define SC_CODEC_ID_OPUS UINT32_C(0x6f707573) // "opus" in ASCII
#define SC_CODEC_ID_AAC UINT32_C(0x00616163) // "aac in ASCII"
#define SC_CODEC_ID_RAW UINT32_C(0x00726177) // "raw" in ASCII
switch (codec_id) {
case SC_CODEC_ID_H264:
return AV_CODEC_ID_H264;
@@ -33,12 +33,12 @@ sc_demuxer_to_avcodec_id(uint32_t codec_id) {
return AV_CODEC_ID_HEVC;
case SC_CODEC_ID_AV1:
return AV_CODEC_ID_AV1;
case SC_CODEC_ID_RAW:
return AV_CODEC_ID_PCM_S16LE;
case SC_CODEC_ID_OPUS:
return AV_CODEC_ID_OPUS;
case SC_CODEC_ID_AAC:
return AV_CODEC_ID_AAC;
case SC_CODEC_ID_RAW:
return AV_CODEC_ID_PCM_S16LE;
default:
LOGE("Unknown codec id 0x%08" PRIx32, codec_id);
return AV_CODEC_ID_NONE;
@@ -200,7 +200,7 @@ run_demuxer(void *data) {
ok = sc_packet_source_sinks_push(&demuxer->packet_source, packet);
av_packet_unref(packet);
if (!ok) {
// The sink already logged its concrete error
LOGE("Demuxer '%s': could not process packet", demuxer->name);
break;
}
}

View File

@@ -43,7 +43,7 @@ const struct scrcpy_options scrcpy_options_default = {
.display_id = 0,
.display_buffer = 0,
.v4l2_buffer = 0,
.audio_buffer = SC_TICK_FROM_MS(50),
.audio_buffer = 0,
#ifdef HAVE_USB
.otg = false,
#endif

View File

@@ -27,9 +27,9 @@ enum sc_codec {
SC_CODEC_H264,
SC_CODEC_H265,
SC_CODEC_AV1,
SC_CODEC_RAW,
SC_CODEC_OPUS,
SC_CODEC_AAC,
SC_CODEC_RAW,
};
enum sc_lock_video_orientation {

View File

@@ -240,8 +240,7 @@ sc_recorder_process_header(struct sc_recorder *recorder) {
sc_cond_wait(&recorder->queue_cond, &recorder->mutex);
}
if (sc_vecdeque_is_empty(&recorder->video_queue)) {
assert(recorder->stopped);
if (recorder->stopped && sc_vecdeque_is_empty(&recorder->video_queue)) {
// If the recorder is stopped, don't process anything if there are not
// at least video packets
sc_mutex_unlock(&recorder->mutex);
@@ -395,6 +394,10 @@ sc_recorder_process_packets(struct sc_recorder *recorder) {
error = true;
goto end;
}
// If the recorder is stopped while one of the streams has no
// packets, then we must avoid a live-loop and correctly record
// the stream having packets.
pts_origin = video_pkt ? video_pkt->pts : audio_pkt->pts;
} else {
// We need both video and audio packets to initialize pts_origin
continue;
@@ -505,10 +508,6 @@ static int
run_recorder(void *data) {
struct sc_recorder *recorder = data;
// Recording is a background task
bool ok = sc_thread_set_priority(SC_THREAD_PRIORITY_LOW);
(void) ok; // We don't care if it worked
bool success = sc_recorder_record(recorder);
sc_mutex_lock(&recorder->mutex);
@@ -584,7 +583,7 @@ sc_recorder_video_packet_sink_push(struct sc_packet_sink *sink,
return false;
}
rec->stream_index = recorder->video_stream_index;
rec->stream_index = 0;
bool ok = sc_vecdeque_push(&recorder->video_queue, rec);
if (!ok) {
@@ -653,7 +652,7 @@ sc_recorder_audio_packet_sink_push(struct sc_packet_sink *sink,
return false;
}
rec->stream_index = recorder->audio_stream_index;
rec->stream_index = 1;
bool ok = sc_vecdeque_push(&recorder->audio_queue, rec);
if (!ok) {

View File

@@ -43,6 +43,7 @@ struct scrcpy {
struct sc_server server;
struct sc_screen screen;
struct sc_audio_player audio_player;
struct sc_delay_buffer audio_buffer;
struct sc_demuxer video_demuxer;
struct sc_demuxer audio_demuxer;
struct sc_decoder video_decoder;
@@ -245,6 +246,13 @@ sc_audio_demuxer_on_ended(struct sc_demuxer *demuxer,
// Contrary to the video demuxer, keep mirroring if only the audio fails
// (unless --require-audio is set).
// 'eos' is true on end-of-stream, including when audio capture is not
// possible on the device (so that scrcpy continue to mirror video without
// failing).
// However, if an audio configuration failure occurs (for example the user
// explicitly selected an unknown audio encoder), 'eos' is false and scrcpy
// must exit.
if (status == SC_DEMUXER_STATUS_ERROR
|| (status == SC_DEMUXER_STATUS_DISABLED
&& options->require_audio)) {
@@ -687,9 +695,16 @@ aoa_hid_end:
sc_frame_source_add_sink(src, &s->screen.frame_sink);
if (options->audio) {
sc_audio_player_init(&s->audio_player, options->audio_buffer);
sc_frame_source_add_sink(&s->audio_decoder.frame_source,
&s->audio_player.frame_sink);
struct sc_frame_source *src = &s->audio_decoder.frame_source;
if (options->audio_buffer) {
sc_delay_buffer_init(&s->audio_buffer, options->audio_buffer,
false);
sc_frame_source_add_sink(src, &s->audio_buffer.frame_sink);
src = &s->audio_buffer.frame_source;
}
sc_audio_player_init(&s->audio_player);
sc_frame_source_add_sink(src, &s->audio_player.frame_sink);
}
}

View File

@@ -169,12 +169,12 @@ sc_server_get_codec_name(enum sc_codec codec) {
return "h265";
case SC_CODEC_AV1:
return "av1";
case SC_CODEC_RAW:
return "raw";
case SC_CODEC_OPUS:
return "opus";
case SC_CODEC_AAC:
return "aac";
case SC_CODEC_RAW:
return "raw";
default:
return NULL;
}

View File

@@ -9,6 +9,7 @@
bool
sc_bytebuf_init(struct sc_bytebuf *buf, size_t alloc_size) {
assert(alloc_size);
// sufficient, but use more for alignment.
buf->data = malloc(alloc_size);
if (!buf->data) {
LOG_OOM();
@@ -63,8 +64,8 @@ sc_bytebuf_write_step0(struct sc_bytebuf *buf, const uint8_t *from,
if (len < right_len) {
right_len = len;
}
memcpy(buf->data + buf->head, from, right_len);
memcpy(buf->data + buf->head, from, right_len);
if (len > right_len) {
memcpy(buf->data, from + right_len, len - right_len);
}

View File

@@ -14,7 +14,7 @@ struct sc_bytebuf {
size_t head; // writter cursor
size_t tail; // reader cursor
// empty: tail == head
// full: ((tail + 1) % alloc_size) == head
// full: (tail + 1) % allocated == head
};
bool
@@ -37,10 +37,12 @@ sc_bytebuf_read(struct sc_bytebuf *buf, uint8_t *to, size_t len);
* The caller must check that len <= sc_bytebuf_read_available() (it is an
* error to attempt to skip more bytes than available).
*
* This function is guaranteed not to write to buf->head.
* This function is guaranteed not to change the head.
*
* This function is guaranteed to not change the head.
*
* It is equivalent to call sc_bytebuf_read() to some array and discard the
* array (but this function is more efficient since there is no copy).
* array (but more efficient since there is no copy).
*/
void
sc_bytebuf_skip(struct sc_bytebuf *buf, size_t len);

View File

@@ -125,30 +125,8 @@ sc_av_log_callback(void *avcl, int level, const char *fmt, va_list vl) {
free(local_fmt);
}
static const char *const sc_sdl_log_priority_names[SDL_NUM_LOG_PRIORITIES] = {
[SDL_LOG_PRIORITY_VERBOSE] = "VERBOSE",
[SDL_LOG_PRIORITY_DEBUG] = "DEBUG",
[SDL_LOG_PRIORITY_INFO] = "INFO",
[SDL_LOG_PRIORITY_WARN] = "WARN",
[SDL_LOG_PRIORITY_ERROR] = "ERROR",
[SDL_LOG_PRIORITY_CRITICAL] = "CRITICAL",
};
static void SDLCALL
sc_sdl_log_print(void *userdata, int category, SDL_LogPriority priority,
const char *message) {
(void) userdata;
(void) category;
FILE *out = priority < SDL_LOG_PRIORITY_WARN ? stdout : stderr;
assert(priority < SDL_NUM_LOG_PRIORITIES);
const char *prio_name = sc_sdl_log_priority_names[priority];
fprintf(out, "%s: %s\n", prio_name, message);
}
void
sc_log_configure() {
SDL_LogSetOutputFunction(sc_sdl_log_print, NULL);
// Redirect FFmpeg logs to SDL logs
av_log_set_callback(sc_av_log_callback);
}

View File

@@ -3,10 +3,7 @@
#include <stddef.h>
/**
* Allocate an array of `nmemb` items of `size` bytes each
*
* Like calloc(), but without initialization.
/* Like calloc(), but without initialization.
* Like reallocarray(), but without reallocation.
*/
void *

View File

@@ -23,39 +23,6 @@ sc_thread_create(sc_thread *thread, sc_thread_fn fn, const char *name,
return true;
}
static SDL_ThreadPriority
to_sdl_thread_priority(enum sc_thread_priority priority) {
switch (priority) {
case SC_THREAD_PRIORITY_TIME_CRITICAL:
#ifdef SCRCPY_SDL_HAS_THREAD_PRIORITY_TIME_CRITICAL
return SDL_THREAD_PRIORITY_TIME_CRITICAL;
#else
// fall through
#endif
case SC_THREAD_PRIORITY_HIGH:
return SDL_THREAD_PRIORITY_HIGH;
case SC_THREAD_PRIORITY_NORMAL:
return SDL_THREAD_PRIORITY_NORMAL;
case SC_THREAD_PRIORITY_LOW:
return SDL_THREAD_PRIORITY_LOW;
default:
assert(!"Unknown thread priority");
return 0;
}
}
bool
sc_thread_set_priority(enum sc_thread_priority priority) {
SDL_ThreadPriority sdl_priority = to_sdl_thread_priority(priority);
int r = SDL_SetThreadPriority(sdl_priority);
if (r) {
LOGD("Could not set thread priority: %s", SDL_GetError());
return false;
}
return true;
}
void
sc_thread_join(sc_thread *thread, int *status) {
SDL_WaitThread(thread->thread, status);

View File

@@ -21,13 +21,6 @@ typedef struct sc_thread {
SDL_Thread *thread;
} sc_thread;
enum sc_thread_priority {
SC_THREAD_PRIORITY_LOW,
SC_THREAD_PRIORITY_NORMAL,
SC_THREAD_PRIORITY_HIGH,
SC_THREAD_PRIORITY_TIME_CRITICAL,
};
typedef struct sc_mutex {
SDL_mutex *mutex;
#ifndef NDEBUG
@@ -46,9 +39,6 @@ sc_thread_create(sc_thread *thread, sc_thread_fn fn, const char *name,
void
sc_thread_join(sc_thread *thread, int *status);
bool
sc_thread_set_priority(enum sc_thread_priority priority);
bool
sc_mutex_init(sc_mutex *mutex);

View File

@@ -52,10 +52,10 @@
*/
#define sc_vecdeque_init(pv) \
({ \
(pv)->data = NULL; \
(pv)->cap = 0; \
(pv)->origin = 0; \
(pv)->size = 0; \
(pv)->data = NULL; \
})
/**
@@ -128,7 +128,7 @@
* \param item_size the size of one item (the generic type is unknown from this
* function)
* \param pcap a pointer to the `cap` field of the SC_VECDEQUE [IN/OUT]
* \param porigin a pointer to pv->origin [IN/OUT]
* \param porigin a pointer to pv->origin (will be read and updated)
* \param size the `size` field of the SC_VECDEQUE
* \return the new array to assign to the `data` field of the SC_VECDEQUE (if
* not NULL)
@@ -312,7 +312,7 @@ sc_vecdeque_growsize_(size_t value)
*
* If the VecDeque is full, it is resized.
*
* This function returns either a valid non-NULL pointer to the uninitialized
* This function returns either a valid non-nULL pointer to the uninitialized
* item just pushed, or NULL on reallocation failure.
*/
#define sc_vecdeque_push_hole(pv) \
@@ -369,7 +369,7 @@ sc_vecdeque_growsize_(size_t value)
})
/**
* Pop an item and return it
* Pop an item and returns it
*
* It is an error to call this function if the VecDeque is empty.
*/

View File

@@ -1,7 +0,0 @@
package com.genymobile.scrcpy;
public interface AsyncProcessor {
void start();
void stop();
void join() throws InterruptedException;
}

View File

@@ -129,8 +129,8 @@ public final class AudioCapture {
pts = nextPts;
}
long durationUs = r * 1000000 / (CHANNELS * BYTES_PER_SAMPLE * SAMPLE_RATE);
nextPts = pts + durationUs;
long durationMs = r * 1000 / CHANNELS / SAMPLE_RATE;
nextPts = pts + durationMs;
if (previousPts != 0 && pts < previousPts) {
// Audio PTS may come from two sources:

View File

@@ -3,9 +3,9 @@ package com.genymobile.scrcpy;
import android.media.MediaFormat;
public enum AudioCodec implements Codec {
RAW(0x00_72_61_77, "raw", MediaFormat.MIMETYPE_AUDIO_RAW),
OPUS(0x6f_70_75_73, "opus", MediaFormat.MIMETYPE_AUDIO_OPUS),
AAC(0x00_61_61_63, "aac", MediaFormat.MIMETYPE_AUDIO_AAC),
RAW(0x00_72_61_77, "raw", MediaFormat.MIMETYPE_AUDIO_RAW);
AAC(0x00_61_61_63, "aac", MediaFormat.MIMETYPE_AUDIO_AAC);
private final int id; // 4-byte ASCII representation of the name
private final String name;

View File

@@ -14,7 +14,7 @@ import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
public final class AudioEncoder implements AsyncProcessor {
public final class AudioEncoder implements AudioRecorder {
private static class InputTask {
private final int index;

View File

@@ -5,7 +5,7 @@ import android.media.MediaCodec;
import java.io.IOException;
import java.nio.ByteBuffer;
public final class AudioRawRecorder implements AsyncProcessor {
public final class AudioRawRecorder implements AudioRecorder {
private final Streamer streamer;

View File

@@ -0,0 +1,12 @@
package com.genymobile.scrcpy;
/**
* A component able to record audio asynchronously
*
* The implementation is responsible to send packets.
*/
public interface AudioRecorder {
void start();
void stop();
void join() throws InterruptedException;
}

View File

@@ -14,7 +14,7 @@ import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
public class Controller implements AsyncProcessor {
public class Controller {
private static final int DEFAULT_DEVICE_ID = 0;

View File

@@ -39,28 +39,28 @@ public final class Ln {
public static void v(String message) {
if (isEnabled(Level.VERBOSE)) {
Log.v(TAG, message);
System.out.print(PREFIX + "VERBOSE: " + message + '\n');
System.out.println(PREFIX + "VERBOSE: " + message);
}
}
public static void d(String message) {
if (isEnabled(Level.DEBUG)) {
Log.d(TAG, message);
System.out.print(PREFIX + "DEBUG: " + message + '\n');
System.out.println(PREFIX + "DEBUG: " + message);
}
}
public static void i(String message) {
if (isEnabled(Level.INFO)) {
Log.i(TAG, message);
System.out.print(PREFIX + "INFO: " + message + '\n');
System.out.println(PREFIX + "INFO: " + message);
}
}
public static void w(String message, Throwable throwable) {
if (isEnabled(Level.WARN)) {
Log.w(TAG, message, throwable);
System.err.print(PREFIX + "WARN: " + message + '\n');
System.out.println(PREFIX + "WARN: " + message);
if (throwable != null) {
throwable.printStackTrace();
}
@@ -74,7 +74,7 @@ public final class Ln {
public static void e(String message, Throwable throwable) {
if (isEnabled(Level.ERROR)) {
Log.e(TAG, message, throwable);
System.err.print(PREFIX + "ERROR: " + message + "\n");
System.out.println(PREFIX + "ERROR: " + message);
if (throwable != null) {
throwable.printStackTrace();
}

View File

@@ -13,7 +13,7 @@ public class Options {
private VideoCodec videoCodec = VideoCodec.H264;
private AudioCodec audioCodec = AudioCodec.OPUS;
private int videoBitRate = 8000000;
private int audioBitRate = 128000;
private int audioBitRate = 196000;
private int maxFps;
private int lockVideoOrientation = -1;
private boolean tunnelForward;

View File

@@ -5,7 +5,6 @@ import android.os.BatteryManager;
import android.os.Build;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -92,7 +91,8 @@ public final class Server {
Workarounds.fillAppInfo();
}
List<AsyncProcessor> asyncProcessors = new ArrayList<>();
Controller controller = null;
AudioRecorder audioRecorder = null;
try (DesktopConnection connection = DesktopConnection.open(scid, tunnelForward, audio, control, sendDummyByte)) {
if (options.getSendDeviceMeta()) {
@@ -101,34 +101,29 @@ public final class Server {
}
if (control) {
Controller controller = new Controller(device, connection, options.getClipboardAutosync(), options.getPowerOn());
device.setClipboardListener(text -> controller.getSender().pushClipboardText(text));
asyncProcessors.add(controller);
controller = new Controller(device, connection, options.getClipboardAutosync(), options.getPowerOn());
controller.start();
final Controller controllerRef = controller;
device.setClipboardListener(text -> controllerRef.getSender().pushClipboardText(text));
}
if (audio) {
AudioCodec audioCodec = options.getAudioCodec();
Streamer audioStreamer = new Streamer(connection.getAudioFd(), audioCodec, options.getSendCodecId(),
options.getSendFrameMeta());
AsyncProcessor audioRecorder;
Streamer audioStreamer = new Streamer(connection.getAudioFd(), audioCodec, options.getSendCodecId(), options.getSendFrameMeta());
if (audioCodec == AudioCodec.RAW) {
audioRecorder = new AudioRawRecorder(audioStreamer);
} else {
audioRecorder = new AudioEncoder(audioStreamer, options.getAudioBitRate(), options.getAudioCodecOptions(),
options.getAudioEncoder());
}
asyncProcessors.add(audioRecorder);
audioRecorder.start();
}
Streamer videoStreamer = new Streamer(connection.getVideoFd(), options.getVideoCodec(), options.getSendCodecId(),
options.getSendFrameMeta());
ScreenEncoder screenEncoder = new ScreenEncoder(device, videoStreamer, options.getVideoBitRate(), options.getMaxFps(),
options.getVideoCodecOptions(), options.getVideoEncoder(), options.getDownsizeOnError());
for (AsyncProcessor asyncProcessor : asyncProcessors) {
asyncProcessor.start();
}
try {
// synchronous
screenEncoder.streamScreen();
@@ -141,14 +136,20 @@ public final class Server {
} finally {
Ln.d("Screen streaming stopped");
initThread.interrupt();
for (AsyncProcessor asyncProcessor : asyncProcessors) {
asyncProcessor.stop();
if (audioRecorder != null) {
audioRecorder.stop();
}
if (controller != null) {
controller.stop();
}
try {
initThread.join();
for (AsyncProcessor asyncProcessor : asyncProcessors) {
asyncProcessor.join();
if (audioRecorder != null) {
audioRecorder.join();
}
if (controller != null) {
controller.join();
}
} catch (InterruptedException e) {
// ignore