libobs: Fix back-to-back GPU encoder sessions breaking

Reset frame_rate_divisor_counter to 0 on encoder shutdown.

After starting and stopping a GPU encoder session, obs_encoder_shutdown
would set frame_rate_divisor_counter to 1. When the next GPU encoder
session was started, in libobs/obs-video-gpu-encode.c, gpu_encode_thread
would set skip to this value (1), and increment
frame_rate_divisor_counter to 2. This causes the next check to fail, as
frame_rate_divisor is 1 by default (2 == 1 is false), so
frame_rate_divisor_counter retains its value. Since skip is non-zero,
the next check, if(skip), passes, and skip to the next loop iteration.

This will continue forever, because frame_rate_divisor_counter will
continue to increment, so it will never hold the same value as
frame_rate_divisor. This means that send_off_encoder_packet is never
called, so the muxer never receives encoded packets.

To the end-user, this manifests as their second encoder session being
impossible to stop. They then have to force quit OBS and the
obs-ffmpeg-mux process. This change prevents that from occurring and
allows multiple back-to-back GPU encoder sessions to be completed.
This commit is contained in:
Ryan Foster 2023-07-14 10:06:18 -04:00
parent 578dc46a79
commit 80864197fb

View file

@ -668,7 +668,7 @@ void obs_encoder_shutdown(obs_encoder_t *encoder)
encoder->first_received = false;
encoder->offset_usec = 0;
encoder->start_ts = 0;
encoder->frame_rate_divisor_counter = 1;
encoder->frame_rate_divisor_counter = 0;
maybe_clear_encoder_core_video_mix(encoder);
}
obs_encoder_set_last_error(encoder, NULL);