Reduce latency by 1 frame

To packetize the H.264 raw stream, av_parser_parse2() (called by
av_read_frame()) knows that it has received a full frame only after it
has received some data for the next frame. As a consequence, the client
always waited until the next frame before sending the current frame to
the decoder!

On the device side, we know packets boundaries. To reduce latency,
make the device always transmit the "frame meta" to packetize the stream
manually (it was already implemented to send PTS, but only enabled on
recording).

On the client side, replace av_read_frame() by manual packetizing and
parsing.

<https://stackoverflow.com/questions/50682518/replacing-av-read-frame-to-reduce-delay>
<https://trac.ffmpeg.org/ticket/3354>
This commit is contained in:
Romain Vimont
2019-07-31 01:55:32 +02:00
parent a90ccbdf3b
commit 63af7fbafe
6 changed files with 180 additions and 187 deletions

View File

@@ -3,6 +3,7 @@
#include <stdbool.h>
#include <stdint.h>
#include <libavformat/avformat.h>
#include <SDL2/SDL_atomic.h>
#include <SDL2/SDL_thread.h>
@@ -10,23 +11,18 @@
struct video_buffer;
struct frame_meta {
uint64_t pts;
struct frame_meta *next;
};
struct stream {
socket_t socket;
struct video_buffer *video_buffer;
SDL_Thread *thread;
SDL_atomic_t stopped;
struct decoder *decoder;
struct recorder *recorder;
struct receiver_state {
// meta (in order) for frames not consumed yet
struct frame_meta *frame_meta_queue;
size_t remaining; // remaining bytes to receive for the current frame
} receiver_state;
AVCodecContext *codec_ctx;
AVCodecParserContext *parser;
// successive packets may need to be concatenated, until a non-config
// packet is available
bool has_pending;
AVPacket pending;
};
void