The existing caption insertion implementation is to add H.264 specific
SEI NALs, because of this we will skip caption insertion unless the
video stream is H.264. This prevents corruption of AV1/HEVC/etc.
We will now keep a per track list of caption data. When caption
insertion is triggered we will add that to the list for each actively
encoding video track. When doing interleaved send, we pull the data
from the caption data for the corresponding video. This ensures that
captions get copied to all video tracks.
Calling `set_higher_ts` before offsets are known pollutes
`highest_video_ts` with timestamps that are out of range of actual
timestamps. They're generally somewhere high above 0 until an
offset for all streams is found, where they are then reset to 0 or
slightly below 0 in the presence of b-frames.
`highest_video_ts` also needs to start below 0 for the same reason.
Even with timestamps being reset to close to 0, b-frames will
cause initial DTS to drop below 0, thus we need a value that should
"always" be below any "real" timestamps observed.
`highest_video_ts` tracking now only starts once all input streams
are ready, and is computed based on all buffered packets at that
point.
The API `obs_output_video` and `obs_output_audio` returned valid
pointers until the commit fb57eff21 and 645e31fa1.
The API `obs_output_video` is used by some plugins such as obs-websocket
and obs-midi to calculate the duration of streaming and recording.
To have a similar behavior, return the media (video and audio,
respectively) from the encoder.
When `obs_output_t` is an encoded output, `obs_output_set_media` will
ignore the video and audio so that `obs_output_t` will keep holding the
`video_t` and `audio_t` pointer when the output was created.
By this commit, `video` and `audio` member variables in `obs_output_t`
will never set if it is an encoded output.
In the auto-configuration wizard, `video_t` is released and created to
have a different video size while `obs_output_t` is not released. This
resulted in accessing the released `video_t` pointer.
Initialize arrays to 0, as otherwise these can get initialized with
garbage data or potentially Visual Studio's default debug marker,
which is a problem if they're being checked against `NULL` later.
Effectively reverting parts of d314d47, this commit removes the new
functions that got added to remove the flags parameter. Instead, it just
marks the parameter as unused and documents this. Having what is
effectively an API break just to remove a parameter is a bit overkill.
The other parts of d314d47 which cleaned up the usage of the flags
parameter are untouched here.
#8873 accidentally used `log_flag_encoded()` instead of `flag_encoded()` in `obs_output_begin_data_capture()`, which was causing a warning to be logged when it shouldn't.
This deprecates the following functions, replacing them with new
versions:
- `obs_output_can_begin_data_capture()` - now `*capture2()`
- `obs_output_initialize_encoders()` - now `*encoders2()`
- `obs_output_begin_data_capture()` - now `*capture2()`
The flags parameter was initially designed to support audio-only or
video-only operation of an output which had the `OBS_OUTPUT_AV` flag,
however, full support for that was never implemented, and there are
likely fundamental issues with an implementation, mainly that most
outputs are programmed assuming that there will always be at least one
audio and one video track. This requires new flags specifying support
for optional audio/video, among other things.
An implementation to allow audio/video to be optional is best done
using the flag technique above, with audio/video enablement specified
by whether media (raw, `video_t/audio_t`) or encoder (`obs_encoder_t`)
objects are specified.
Since every implementation I could find always specifies `flags` as 0,
I was able to safely conclude that immediately removing the parameter's
functionality is safe to do.
Since OBS outputs can have any combination of flags for encoded,
video, audio, and service, there are a number of cases where it would
be a good idea to validate that we're not allowing output changes that
risk undefined behavior given the supported flags.
This also does a mild amount of code cleanup, adding inline functions
for checking the flags mentioned above.
Since 65eb3c0815 we tried to get as close
a sync between audio and video tracks as we could before starting to
send frames to the output.
But in be717dbb2c when multi-track audio
was considered for synchronization it continued to try and line ALL
audio packets up with one video packet. While audio and video packets
are similar size this will work out. But once video packets duration is
smaller than 1/2 audio packet duration this may fail forever. In
practice with AAC's 20-23ms frame duration we can often get tracks
producing frames 5-10ms out of phase. For 60fps video this mostly fine
as 16ms frame duration will cover the gap, but for framerates as low as
100 its possible to fail to synchronize. In practice this is ~50%
for 6 audio tracks and 120fps video on my system, or 100% at 240fps.
There was quite a bit of conflated usage of mixes (which refers
to raw audio) and encoder counts. This fully separates the two
and makes a distinct separation when iterating over mixes vs
encoders.
- Changes the default base exponent value to 1.5 from 2.0
- Applies a random skew of +-0.05 to the exponent to lessen the
"water hammer" effect caused by predictable backoff techniques
- Fixes the logging associated with exponential backoff to log the
true reconnect delay value
With this, you can now cast normal obs objects (services, outputs,
sources, encoders) to an obs_object_t, and then use obs_object_*
functions to get references, release references, and similar for weak
object references as well. This allows the ability for the frontend to
use an object of any of those types interchangeably in certain
situations without having to handle each specific type individually.
This is useful because the properties view in particular doesn't care
what type of object it uses, it just needs to be able to hold weak
references to abstract OBS objects.
(This commit also modifies UI)
This makes it more trivial for encoder plugins to communicate to users
why specifically an encoder error might have occurred mid-stream.
This isn't particularly needed, as a service with multiple tracks won't
be using multiple tracks to begin with anyway. This might change later,
but for now just mark it deprecated.
Similar to how outputs can pass errors, add the same functionality for
encoders so that if an output encoder has an error, it is made available
to the output and eventually the UI / user.
