Discussion:
[FFmpeg-devel] FIX: HLS Discontinuity / Non-Monotonous DTS
Joe Koberg
2018-03-16 13:58:17 UTC
Permalink
Hello,

This patch adds code to track and correct timestamp discontinuities, fixing
"non-monotonous dts" errors and timing issues with HLS playlists.

For some time, FFmpeg has not properly handled discontinuous timestamps in
the MPEG-TS stream from Apple HLS playlists. Symptoms include inability to
remux to containers like MP4 or stream to RTMP endpoints; choppy, broken,
or sped-up playback; and voluminous "Non-monotonous DTS", "went backwards",
and "out of order" messages logged. This is particularly troublesome in
the context of HLS, as the format allows insertion of independently-sourced
fragments for ad-insertion or to concatenate ongoing video to a stream.

Over the years, many users have encountered and reported this issue in
different ways.

* https://trac.ffmpeg.org/ticket/5419 "HLS EXT-X-DISCONTINUITY tag is not
supported"
* https://trac.ffmpeg.org/ticket/6613
* https://trac.ffmpeg.org/ticket/6810
* https://trac.ffmpeg.org/ticket/6709
* https://trac.ffmpeg.org/ticket/6365
* https://trac.ffmpeg.org/ticket/5236
* https://github.com/Bilibili/ijkplayer/issues/2565
* https://stackoverflow.com/questions/49289394/downloading-ts-stream-with-
ext-x-discontinuity-sequence-ffmpeg
* etc...


It appears that the -dts_delta_threshold checking code is quite similar in
intent, and adjacent to this code, but the HLS demuxer lacks the
AVFMT_TS_DISCONT flag, effectively disabling that option. Even when
AVFMT_TS_DISCONT is set on the input format (e.g. MPEG-TS),
"-dts_delta_threshold 0" produces out-of-sync audio in our tests. That
code has been left as-is for now. The flag probably should be set on the
HLS format, though.

In this fix, any backwards DTS jump is considered an error, as is any
forward jump of DTS or PTS by more than `dts_monotonicity_threshold`
microseconds (really, AV_TIME_BASE units, defaulting to 1000000 = 1
second). The delta from the predicted value is updated and applied to all
subsequent packets, with the assumption that video was stitched together
from continuous runs.

With this patch in place and the -force_dts_monotonicity flag set, we can
successfuly transcode arbitrary HLS playlists to MP4, FLV, or other
timestamped containers when we could not before. The playback presentation
of the result is smooth, properly timed, and in-sync in Quicktime Player,
VLC, and ffplay with correctly reported and the file seekable.

Interaction with flags such as -copyts, -copytb, and others has not been
fully analyzed.

We have prepared a test HLS playlist that reveals the issue.

How to reproduce:

ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4

With patch:

ffmpeg -force_dts_monotonicity -i https://s3.amazonaws.com/
playon-test-videos/discont_test_new/discont_test.m3u8 -c copy -y output.mp4


Please let me know if I've missed anything!

----

Joe Koberg
***@gmail.com
Carl Eugen Hoyos
2018-03-16 14:47:10 UTC
Permalink
Post by Joe Koberg
Hello,
This patch adds code to track and correct timestamp discontinuities, fixing
"non-monotonous dts" errors and timing issues with HLS playlists.
For some time, FFmpeg has not properly handled discontinuous timestamps in
the MPEG-TS stream from Apple HLS playlists. Symptoms include inability to
remux to containers like MP4 or stream to RTMP endpoints; choppy, broken,
or sped-up playback; and voluminous "Non-monotonous DTS", "went backwards",
and "out of order" messages logged. This is particularly troublesome in
the context of HLS, as the format allows insertion of independently-sourced
fragments for ad-insertion or to concatenate ongoing video to a stream.
Over the years, many users have encountered and reported this issue in
different ways.
* https://trac.ffmpeg.org/ticket/5419 "HLS EXT-X-DISCONTINUITY tag is not
supported"
* https://trac.ffmpeg.org/ticket/6613
* https://trac.ffmpeg.org/ticket/6810
* https://trac.ffmpeg.org/ticket/6709
* https://trac.ffmpeg.org/ticket/6365
* https://trac.ffmpeg.org/ticket/5236
* https://github.com/Bilibili/ijkplayer/issues/2565
*
https://stackoverflow.com/questions/49289394/downloading-ts-stream-with-
ext-x-discontinuity-sequence-ffmpeg
* etc...
It appears that the -dts_delta_threshold checking code is quite similar in
intent, and adjacent to this code, but the HLS demuxer lacks the
AVFMT_TS_DISCONT flag, effectively disabling that option. Even when
AVFMT_TS_DISCONT is set on the input format (e.g. MPEG-TS),
"-dts_delta_threshold 0" produces out-of-sync audio in our tests. That
code has been left as-is for now. The flag probably should be set on the
HLS format, though.
In this fix, any backwards DTS jump is considered an error, as is any
forward jump of DTS or PTS by more than `dts_monotonicity_threshold`
microseconds (really, AV_TIME_BASE units, defaulting to 1000000 = 1
second). The delta from the predicted value is updated and applied to all
subsequent packets, with the assumption that video was stitched together
from continuous runs.
With this patch in place and the -force_dts_monotonicity flag set, we can
successfuly transcode arbitrary HLS playlists to MP4, FLV, or other
timestamped containers when we could not before. The playback presentation
of the result is smooth, properly timed, and in-sync in Quicktime Player,
VLC, and ffplay with correctly reported and the file seekable.
Interaction with flags such as -copyts, -copytb, and others has not been
fully analyzed.
We have prepared a test HLS playlist that reveals the issue.
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
ffmpeg -force_dts_monotonicity -i https://s3.amazonaws.com/
playon-test-videos/discont_test_new/discont_test.m3u8 -c copy -y output.mp4
Please let me know if I've missed anything!
----
Joe Koberg
Carl Eugen Hoyos
2018-03-16 14:48:17 UTC
Permalink
Post by Joe Koberg
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
Is the issue only reproducible if you use hls input or is it also
reproducible if you first concatenate the ts files and use the
complete file as input?

Carl Eugen
Joe Koberg
2018-03-16 15:07:53 UTC
Permalink
Post by Carl Eugen Hoyos
Post by Joe Koberg
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
Is the issue only reproducible if you use hls input or is it also
reproducible if you first concatenate the ts files and use the
complete file as input?
Very interesting, this case seems to work perfectly with ffmpeg 3.4.2.
In previous versions, I don't think it did.

Any hints on getting that to apply to the HLS demuxer?


test concat file:

file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file '2befd3aa94_20180313T082811_000016.ts'
file '2befd3aa94_20180313T082811_000017.ts'
file '2befd3aa94_20180313T082811_000018.ts'
file '2befd3aa94_20180313T082811_000019.ts'
file '2befd3aa94_20180313T082811_000020.ts'
file '2befd3aa94_20180313T082811_000021.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file '2befd3aa94_20180313T082811_000022.ts'
file '2befd3aa94_20180313T082811_000023.ts'
file '2befd3aa94_20180313T082811_000024.ts'
file '2befd3aa94_20180313T082811_000025.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'
file 'b1dc7f03bcce8b554f5f840cf5018b0e_20171009T151157Z_000000.ts'



with command:
$ ffmpeg -f concat -i test.concat -c copy -y outputconcat.mp4

----

Joe Koberg
***@gmail.com
Carl Eugen Hoyos
2018-03-16 15:14:46 UTC
Permalink
Post by Joe Koberg
Post by Carl Eugen Hoyos
Post by Joe Koberg
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
Is the issue only reproducible if you use hls input or is it also
reproducible if you first concatenate the ts files and use the
complete file as input?
Very interesting, this case seems to work perfectly with ffmpeg 3.4.2.
Only current FFmpeg is relevant.
Post by Joe Koberg
In previous versions, I don't think it did.
Any hints on getting that to apply to the HLS demuxer?
I meant:
What happens if you concat all these files (with "cat") before
feeding the resulting large file to ffmpeg?

Carl Eugen
Joe Koberg
2018-03-16 15:24:06 UTC
Permalink
Post by Carl Eugen Hoyos
Post by Joe Koberg
Post by Carl Eugen Hoyos
Post by Joe Koberg
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
Is the issue only reproducible if you use hls input or is it also
reproducible if you first concatenate the ts files and use the
complete file as input?
Very interesting, this case seems to work perfectly with ffmpeg 3.4.2.
Only current FFmpeg is relevant.
Post by Joe Koberg
In previous versions, I don't think it did.
Any hints on getting that to apply to the HLS demuxer?
What happens if you concat all these files (with "cat") before
feeding the resulting large file to ffmpeg?
That also seems to work fine! But the duration of the output file
differs a bit (it's correct, 112.112s) from the concat demuxer, which
I didn't expect
Joe Koberg
2018-03-20 14:46:58 UTC
Permalink
Post by Carl Eugen Hoyos
Post by Joe Koberg
Post by Carl Eugen Hoyos
Is the issue only reproducible if you use hls input or is it also
reproducible if you first concatenate the ts files and use the
complete file as input?
Very interesting, this case seems to work perfectly with ffmpeg 3.4.2.
Only current FFmpeg is relevant.
Post by Joe Koberg
In previous versions, I don't think it did.
Any hints on getting that to apply to the HLS demuxer?
What happens if you concat all these files (with "cat") before
feeding the resulting large file to ffmpeg?
Carl Eugen
When I do that, a few messages are printed but otherwise everything
works perfectly and the output appears correctly timed when written to
MP4.

How do we get similar performance from HLS input? I think hls.c just
calls av_probe_input_buffer to open the sub-demuxers (this is my first
time trying to read this code, so please correct me), so I'm not sure
why the MPEG-TS demuxer wouldn't handle timestamps the same way in
both situations.

for reference, new command was:

cat `grep '.ts$' discont_test.m3u8` | ffmpeg -f mpegts -i - -c
copy output000.mp4



Thanks,

Joe Koberg
***@koberg.us
Aman Gupta
2018-05-15 23:50:34 UTC
Permalink
Post by Joe Koberg
Post by Carl Eugen Hoyos
Post by Joe Koberg
Post by Carl Eugen Hoyos
Is the issue only reproducible if you use hls input or is it also
reproducible if you first concatenate the ts files and use the
complete file as input?
Very interesting, this case seems to work perfectly with ffmpeg 3.4.2.
Only current FFmpeg is relevant.
Post by Joe Koberg
In previous versions, I don't think it did.
Any hints on getting that to apply to the HLS demuxer?
What happens if you concat all these files (with "cat") before
feeding the resulting large file to ffmpeg?
Carl Eugen
When I do that, a few messages are printed but otherwise everything
works perfectly and the output appears correctly timed when written to
MP4.
How do we get similar performance from HLS input? I think hls.c just
calls av_probe_input_buffer to open the sub-demuxers (this is my first
time trying to read this code, so please correct me), so I'm not sure
why the MPEG-TS demuxer wouldn't handle timestamps the same way in
both situations.
I tried adding AVFMT_TS_DISCONT to ff_hls_demuxer locally, and most of the
warnings in your remux test went away. The output.mp4 appears to play back
fine.

AFAIK there is no other difference between the mpegts demuxer and the hls
demuxer, and your description of how hls.c is correct.

Aman
Post by Joe Koberg
cat `grep '.ts$' discont_test.m3u8` | ffmpeg -f mpegts -i - -c
copy output000.mp4
Thanks,
Joe Koberg
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Joe Koberg
2018-05-02 17:43:20 UTC
Permalink
Post by Carl Eugen Hoyos
What happens if you concat all these files (with "cat") before
feeding the resulting large file to ffmpeg?
Carl Eugen
We'd love to get this HLS input working, any ideas on what this patch
would need for it to be applied?

Thanks in advance for any feedback!


Joe Koberg
***@koberg.us
Senior Video Platform Developer
PlayOn! Sports / NFHSNetwork
wm4
2018-03-16 18:52:06 UTC
Permalink
On Fri, 16 Mar 2018 09:58:17 -0400
Post by Joe Koberg
Hello,
This patch adds code to track and correct timestamp discontinuities, fixing
"non-monotonous dts" errors and timing issues with HLS playlists.
For some time, FFmpeg has not properly handled discontinuous timestamps in
the MPEG-TS stream from Apple HLS playlists. Symptoms include inability to
remux to containers like MP4 or stream to RTMP endpoints; choppy, broken,
or sped-up playback; and voluminous "Non-monotonous DTS", "went backwards",
and "out of order" messages logged. This is particularly troublesome in
the context of HLS, as the format allows insertion of independently-sourced
fragments for ad-insertion or to concatenate ongoing video to a stream.
Over the years, many users have encountered and reported this issue in
different ways.
* https://trac.ffmpeg.org/ticket/5419 "HLS EXT-X-DISCONTINUITY tag is not
supported"
* https://trac.ffmpeg.org/ticket/6613
* https://trac.ffmpeg.org/ticket/6810
* https://trac.ffmpeg.org/ticket/6709
* https://trac.ffmpeg.org/ticket/6365
* https://trac.ffmpeg.org/ticket/5236
* https://github.com/Bilibili/ijkplayer/issues/2565
* https://stackoverflow.com/questions/49289394/downloading-ts-stream-with-
ext-x-discontinuity-sequence-ffmpeg
* etc...
It appears that the -dts_delta_threshold checking code is quite similar in
intent, and adjacent to this code, but the HLS demuxer lacks the
AVFMT_TS_DISCONT flag, effectively disabling that option. Even when
AVFMT_TS_DISCONT is set on the input format (e.g. MPEG-TS),
"-dts_delta_threshold 0" produces out-of-sync audio in our tests. That
code has been left as-is for now. The flag probably should be set on the
HLS format, though.
In this fix, any backwards DTS jump is considered an error, as is any
forward jump of DTS or PTS by more than `dts_monotonicity_threshold`
microseconds (really, AV_TIME_BASE units, defaulting to 1000000 = 1
second). The delta from the predicted value is updated and applied to all
subsequent packets, with the assumption that video was stitched together
from continuous runs.
With this patch in place and the -force_dts_monotonicity flag set, we can
successfuly transcode arbitrary HLS playlists to MP4, FLV, or other
timestamped containers when we could not before. The playback presentation
of the result is smooth, properly timed, and in-sync in Quicktime Player,
VLC, and ffplay with correctly reported and the file seekable.
Interaction with flags such as -copyts, -copytb, and others has not been
fully analyzed.
We have prepared a test HLS playlist that reveals the issue.
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
ffmpeg -force_dts_monotonicity -i https://s3.amazonaws.com/
playon-test-videos/discont_test_new/discont_test.m3u8 -c copy -y output.mp4
Please let me know if I've missed anything!
So I haven't looked too closely at this. It's possible that ffmpeg.c
should handle discontinuities better, or has a regression with handling
it. But besides that, shouldn't the HLS demuxer just be changed to
parse the discontinuity markers, and add them as packet flags? Then
these "clever" heuristics (which are just really hard to maintain)
wouldn't be needed, and also applications using libavformat as library
would benefit from it.

Your patch may or may not have a benefit on top of it (for normal .ts
files etc.), and the HLS demuxer may or may not have additional issues,
but I think that's orthogonal.

There isn't such a discontinuity packet flag - that would have to be
added.
Joe Koberg
2018-03-16 19:04:36 UTC
Permalink
While HLS has the discontinuity flag, I'm not sure it would add
anything that wouldn't be needed anyway without knowledge of it.
I think it's valid for MPEG-TS streams to have discontinuous timestamps.

It turns out the MPEGTS demuxer deals with the underlying MPEGTS
stream just fine in this case, indicating that this patch wouldn't be
needed if that logic would trigger for HLS. We would really like to
not have to manually reconstruct the input from HLS playlists to make
FFMPEG work with HLS.

Joe
Post by wm4
On Fri, 16 Mar 2018 09:58:17 -0400
Post by Joe Koberg
Hello,
This patch adds code to track and correct timestamp discontinuities, fixing
"non-monotonous dts" errors and timing issues with HLS playlists.
For some time, FFmpeg has not properly handled discontinuous timestamps in
the MPEG-TS stream from Apple HLS playlists. Symptoms include inability to
remux to containers like MP4 or stream to RTMP endpoints; choppy, broken,
or sped-up playback; and voluminous "Non-monotonous DTS", "went backwards",
and "out of order" messages logged. This is particularly troublesome in
the context of HLS, as the format allows insertion of independently-sourced
fragments for ad-insertion or to concatenate ongoing video to a stream.
Over the years, many users have encountered and reported this issue in
different ways.
* https://trac.ffmpeg.org/ticket/5419 "HLS EXT-X-DISCONTINUITY tag is not
supported"
* https://trac.ffmpeg.org/ticket/6613
* https://trac.ffmpeg.org/ticket/6810
* https://trac.ffmpeg.org/ticket/6709
* https://trac.ffmpeg.org/ticket/6365
* https://trac.ffmpeg.org/ticket/5236
* https://github.com/Bilibili/ijkplayer/issues/2565
* https://stackoverflow.com/questions/49289394/downloading-ts-stream-with-
ext-x-discontinuity-sequence-ffmpeg
* etc...
It appears that the -dts_delta_threshold checking code is quite similar in
intent, and adjacent to this code, but the HLS demuxer lacks the
AVFMT_TS_DISCONT flag, effectively disabling that option. Even when
AVFMT_TS_DISCONT is set on the input format (e.g. MPEG-TS),
"-dts_delta_threshold 0" produces out-of-sync audio in our tests. That
code has been left as-is for now. The flag probably should be set on the
HLS format, though.
In this fix, any backwards DTS jump is considered an error, as is any
forward jump of DTS or PTS by more than `dts_monotonicity_threshold`
microseconds (really, AV_TIME_BASE units, defaulting to 1000000 = 1
second). The delta from the predicted value is updated and applied to all
subsequent packets, with the assumption that video was stitched together
from continuous runs.
With this patch in place and the -force_dts_monotonicity flag set, we can
successfuly transcode arbitrary HLS playlists to MP4, FLV, or other
timestamped containers when we could not before. The playback presentation
of the result is smooth, properly timed, and in-sync in Quicktime Player,
VLC, and ffplay with correctly reported and the file seekable.
Interaction with flags such as -copyts, -copytb, and others has not been
fully analyzed.
We have prepared a test HLS playlist that reveals the issue.
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
ffmpeg -force_dts_monotonicity -i https://s3.amazonaws.com/
playon-test-videos/discont_test_new/discont_test.m3u8 -c copy -y output.mp4
Please let me know if I've missed anything!
So I haven't looked too closely at this. It's possible that ffmpeg.c
should handle discontinuities better, or has a regression with handling
it. But besides that, shouldn't the HLS demuxer just be changed to
parse the discontinuity markers, and add them as packet flags? Then
these "clever" heuristics (which are just really hard to maintain)
wouldn't be needed, and also applications using libavformat as library
would benefit from it.
Your patch may or may not have a benefit on top of it (for normal .ts
files etc.), and the HLS demuxer may or may not have additional issues,
but I think that's orthogonal.
There isn't such a discontinuity packet flag - that would have to be
added.
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Bodecs Bela
2018-03-16 19:36:14 UTC
Permalink
Post by wm4
On Fri, 16 Mar 2018 09:58:17 -0400
Post by Joe Koberg
Hello,
This patch adds code to track and correct timestamp discontinuities, fixing
"non-monotonous dts" errors and timing issues with HLS playlists.
For some time, FFmpeg has not properly handled discontinuous timestamps in
the MPEG-TS stream from Apple HLS playlists. Symptoms include inability to
remux to containers like MP4 or stream to RTMP endpoints; choppy, broken,
or sped-up playback; and voluminous "Non-monotonous DTS", "went backwards",
and "out of order" messages logged. This is particularly troublesome in
the context of HLS, as the format allows insertion of independently-sourced
fragments for ad-insertion or to concatenate ongoing video to a stream.
Over the years, many users have encountered and reported this issue in
different ways.
* https://trac.ffmpeg.org/ticket/5419 "HLS EXT-X-DISCONTINUITY tag is not
supported"
* https://trac.ffmpeg.org/ticket/6613
* https://trac.ffmpeg.org/ticket/6810
* https://trac.ffmpeg.org/ticket/6709
* https://trac.ffmpeg.org/ticket/6365
* https://trac.ffmpeg.org/ticket/5236
* https://github.com/Bilibili/ijkplayer/issues/2565
* https://stackoverflow.com/questions/49289394/downloading-ts-stream-with-
ext-x-discontinuity-sequence-ffmpeg
* etc...
It appears that the -dts_delta_threshold checking code is quite similar in
intent, and adjacent to this code, but the HLS demuxer lacks the
AVFMT_TS_DISCONT flag, effectively disabling that option. Even when
AVFMT_TS_DISCONT is set on the input format (e.g. MPEG-TS),
"-dts_delta_threshold 0" produces out-of-sync audio in our tests. That
code has been left as-is for now. The flag probably should be set on the
HLS format, though.
In this fix, any backwards DTS jump is considered an error, as is any
forward jump of DTS or PTS by more than `dts_monotonicity_threshold`
microseconds (really, AV_TIME_BASE units, defaulting to 1000000 = 1
second). The delta from the predicted value is updated and applied to all
subsequent packets, with the assumption that video was stitched together
from continuous runs.
With this patch in place and the -force_dts_monotonicity flag set, we can
successfuly transcode arbitrary HLS playlists to MP4, FLV, or other
timestamped containers when we could not before. The playback presentation
of the result is smooth, properly timed, and in-sync in Quicktime Player,
VLC, and ffplay with correctly reported and the file seekable.
Interaction with flags such as -copyts, -copytb, and others has not been
fully analyzed.
We have prepared a test HLS playlist that reveals the issue.
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
ffmpeg -force_dts_monotonicity -i https://s3.amazonaws.com/
playon-test-videos/discont_test_new/discont_test.m3u8 -c copy -y output.mp4
Please let me know if I've missed anything!
So I haven't looked too closely at this. It's possible that ffmpeg.c
should handle discontinuities better, or has a regression with handling
it. But besides that, shouldn't the HLS demuxer just be changed to
parse the discontinuity markers, and add them as packet flags? Then
these "clever" heuristics (which are just really hard to maintain)
wouldn't be needed, and also applications using libavformat as library
would benefit from it.
Your patch may or may not have a benefit on top of it (for normal .ts
files etc.), and the HLS demuxer may or may not have additional issues,
but I think that's orthogonal.
There isn't such a discontinuity packet flag - that would have to be
added.
maybe thisĀ  discontinuity packet flag could be used in concat demuxer also?
Post by wm4
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
bb
Joe Koberg
2018-03-16 19:51:26 UTC
Permalink
I'm saying I don't think it needs it - MPEGTS does seem to handle the
discontinuous input fine if I just concatenate it.

MPEG TS packets themselves do have a discontinuity flag in the
adaptation field, but again it looks like it might be working as-is
without explicit notification of them. (and probably should)

The question remains: what is missing to allow the HLS format to
behave as MPEG-TS does right now?
Post by wm4
On Fri, 16 Mar 2018 09:58:17 -0400
Post by Joe Koberg
Hello,
This patch adds code to track and correct timestamp discontinuities, fixing
"non-monotonous dts" errors and timing issues with HLS playlists.
For some time, FFmpeg has not properly handled discontinuous timestamps in
the MPEG-TS stream from Apple HLS playlists. Symptoms include inability to
remux to containers like MP4 or stream to RTMP endpoints; choppy, broken,
or sped-up playback; and voluminous "Non-monotonous DTS", "went backwards",
and "out of order" messages logged. This is particularly troublesome in
the context of HLS, as the format allows insertion of
independently-sourced
fragments for ad-insertion or to concatenate ongoing video to a stream.
Over the years, many users have encountered and reported this issue in
different ways.
* https://trac.ffmpeg.org/ticket/5419 "HLS EXT-X-DISCONTINUITY tag is not
supported"
* https://trac.ffmpeg.org/ticket/6613
* https://trac.ffmpeg.org/ticket/6810
* https://trac.ffmpeg.org/ticket/6709
* https://trac.ffmpeg.org/ticket/6365
* https://trac.ffmpeg.org/ticket/5236
* https://github.com/Bilibili/ijkplayer/issues/2565
*
https://stackoverflow.com/questions/49289394/downloading-ts-stream-with-
ext-x-discontinuity-sequence-ffmpeg
* etc...
It appears that the -dts_delta_threshold checking code is quite similar in
intent, and adjacent to this code, but the HLS demuxer lacks the
AVFMT_TS_DISCONT flag, effectively disabling that option. Even when
AVFMT_TS_DISCONT is set on the input format (e.g. MPEG-TS),
"-dts_delta_threshold 0" produces out-of-sync audio in our tests. That
code has been left as-is for now. The flag probably should be set on the
HLS format, though.
In this fix, any backwards DTS jump is considered an error, as is any
forward jump of DTS or PTS by more than `dts_monotonicity_threshold`
microseconds (really, AV_TIME_BASE units, defaulting to 1000000 = 1
second). The delta from the predicted value is updated and applied to all
subsequent packets, with the assumption that video was stitched together
from continuous runs.
With this patch in place and the -force_dts_monotonicity flag set, we can
successfuly transcode arbitrary HLS playlists to MP4, FLV, or other
timestamped containers when we could not before. The playback presentation
of the result is smooth, properly timed, and in-sync in Quicktime Player,
VLC, and ffplay with correctly reported and the file seekable.
Interaction with flags such as -copyts, -copytb, and others has not been
fully analyzed.
We have prepared a test HLS playlist that reveals the issue.
ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
test_new/discont_test.m3u8 -c copy -y output.mp4
ffmpeg -force_dts_monotonicity -i https://s3.amazonaws.com/
playon-test-videos/discont_test_new/discont_test.m3u8 -c copy -y output.mp4
Please let me know if I've missed anything!
So I haven't looked too closely at this. It's possible that ffmpeg.c
should handle discontinuities better, or has a regression with handling
it. But besides that, shouldn't the HLS demuxer just be changed to
parse the discontinuity markers, and add them as packet flags? Then
these "clever" heuristics (which are just really hard to maintain)
wouldn't be needed, and also applications using libavformat as library
would benefit from it.
Your patch may or may not have a benefit on top of it (for normal .ts
files etc.), and the HLS demuxer may or may not have additional issues,
but I think that's orthogonal.
There isn't such a discontinuity packet flag - that would have to be
added.
maybe this discontinuity packet flag could be used in concat demuxer also?
Post by wm4
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
bb
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
wm4
2018-03-17 15:25:52 UTC
Permalink
On Fri, 16 Mar 2018 15:51:26 -0400
Post by Joe Koberg
I'm saying I don't think it needs it - MPEGTS does seem to handle the
discontinuous input fine if I just concatenate it.
MPEG TS packets themselves do have a discontinuity flag in the
adaptation field, but again it looks like it might be working as-is
without explicit notification of them. (and probably should)
The question remains: what is missing to allow the HLS format to
behave as MPEG-TS does right now?
First off, from hearing past rants from some people trying to use the
ffmpeg mpegts demuxer, it doesn't seem to handle these things fully
correctly.

Second, I'm talking about libavformat as a library, not whatever
"apparently working" hacks ffmpeg.c applies on top of it.
Loading...