Discussion:
[FFmpeg-devel] [PATCH 00/21] Add timed metadata tracks, track references, channel layout
e***@nokia.com
2016-08-23 09:03:18 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Hello,

Here is a patch set with the goal of adding the following features to
the ISO base media file format support of FFmpeg (also available at
https://github.com/nokiatech/FFmpeg/tree/timed-metadata):

* Ability to mux and demux timed metadata tracks

Timed metadata tracks have a sample descriptor describing their
contents (ie. an URI and a configuration) in the headers. The tracks
are associated with media tracks by using track references and the
samples of the metadata track can have arbitrary length in size and
time.

The sample type urim is supported. The metadata track has
codec id AV_CODEC_ID_META and its type is AVMEDIA_TYPE_DATA.

* Ability to set alternate track groups (instead of using the
automatically assigned ones)

This functionality is accessed by adding side data
AV_PKT_DATA_TRACK_ALTERNATE_GROUP to the track.

* Ability to add and retrieve any types of track reference and
ability to refere multiple tracks instead of only one

This functionality is accessed by adding/accessing side data
AV_PKT_DATA_TRACK_REFERENCES to the track.

* Ability to set advanced channel layout to tracks (the chnl box)

This functionality is separated into the following three side data:

AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT for choosing a
predefined channel layout (with a list of omitted channels).

AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT for setting arbitrary speaker
positions.

AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED for indicating the
audio track has object structured audio.

* Some smaller funtionality has also been added:

av_arraydup for duplicating an array (the allocated memory block is
compatible with av_realloc).

Ability to use stream ids directly as track numbers (with option
use_stream_ids_as_track_ids).

Two examples have been added (based on their non-metadata versions)
to doc/examples on how to work with timed metadata.

-

Erkki Seppälä (21):
libavutil/mem: added av_arraydup to accompany av_realloc*_array
functions
libavformat/utils: added ability to probe AVMEDIA_TYPE_DATA format
libavformat/movenc: added ability to use original stream ids as track
ids instead of regenerating them
libavcodec: added a structure AVData for decoding timed metadata
libavformat/movenc: mov_write_ftyp_tag: write the major brand a
compatible brand
libavcodec/avcodec.h: AV_CODEC_ID_META added for timed meta data
libavcodec/metaenc: added an encoder/decoder for timed metadata
libavformat/movenc: deal with AVMEDIA_TYPE_DATA by using
AV_CODEC_ID_META
libavformat/movenc: support for multiple and client-provided track
references
libavformat/movenc: add the urim sample descriptor
libavformat/movenc, isom: support metadata in mp4 files
libavformat/mov, isom: read (multiple) track references (tag and
multiple ids)
libavformat/mov: basic support for identifying (and reading) timed
metadata
libavformat/mov: ff_mov_read_stsd_entries now deals with
AVMEDIA_TYPE_DATA
libavformat/mov: read urim metadata from meta
libavcodec/avcodec, libavformat/movenc: support embedding channel
layout to stream side data
libavcodec/avcodec, libavformat/movenc: introduced
AV_PKT_DATA_TRACK_ALTERNATE_GROUP side data for expressing alternate
groups
libavformat/movenc: mov_write_audio_tag writes the proper number of
channels, not the hardcoded 2
doc/examples/extract_timed_metadata: added a bare-bones metadata
extractor; find only the frames
doc/examples/muxing_with_metadata: example for dealing with timed meta
data
changelog: updated

Changelog | 21 +-
configure | 4 +-
doc/examples/Makefile | 3 +
doc/examples/extract_timed_metadata.c | 233 +++++++++
doc/examples/muxing_with_metadata.c | 882 ++++++++++++++++++++++++++++++++++
libavcodec/Makefile | 2 +-
libavcodec/allcodecs.c | 3 +
libavcodec/avcodec.h | 115 ++++-
libavcodec/codec_desc.c | 8 +
libavcodec/metacodec.c | 94 ++++
libavcodec/utils.c | 19 +
libavformat/isom.c | 7 +
libavformat/isom.h | 5 +
libavformat/mov.c | 238 ++++++++-
libavformat/movenc.c | 493 +++++++++++++++++--
libavformat/movenc.h | 64 ++-
libavformat/movenchint.c | 11 +-
libavformat/movmeta.h | 46 ++
libavformat/utils.c | 7 +-
libavutil/mem.c | 11 +
libavutil/mem.h | 11 +
21 files changed, 2219 insertions(+), 58 deletions(-)
create mode 100644 doc/examples/extract_timed_metadata.c
create mode 100644 doc/examples/muxing_with_metadata.c
create mode 100644 libavcodec/metacodec.c
create mode 100644 libavformat/movmeta.h
--
2.7.4
e***@nokia.com
2016-08-23 09:03:21 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Sometimes it's useful to be able to define the exact track numbers in
the generated track, instead of always beginning at track id 1. Using
the option use_stream_ids_as_track_ids now copies the use stream ids
to track ids. Dynamically generated tracks (ie. tmcd) have their track
numbers defined as continuing from the highest numbered stream id.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/movenc.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++------
libavformat/movenc.h | 2 ++
2 files changed, 50 insertions(+), 6 deletions(-)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 1f55333..8c4252d 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -89,6 +89,7 @@ static const AVOption options[] = {
{ "encryption_scheme", "Configures the encryption scheme, allowed values are none, cenc-aes-ctr", offsetof(MOVMuxContext, encryption_scheme_str), AV_OPT_TYPE_STRING, {.str = NULL}, .flags = AV_OPT_FLAG_ENCODING_PARAM },
{ "encryption_key", "The media encryption key (hex)", offsetof(MOVMuxContext, encryption_key), AV_OPT_TYPE_BINARY, .flags = AV_OPT_FLAG_ENCODING_PARAM },
{ "encryption_kid", "The media encryption key identifier (hex)", offsetof(MOVMuxContext, encryption_kid), AV_OPT_TYPE_BINARY, .flags = AV_OPT_FLAG_ENCODING_PARAM },
+ { "use_stream_ids_as_track_ids", "use stream ids as track ids", offsetof(MOVMuxContext, use_stream_ids_as_track_ids), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, AV_OPT_FLAG_ENCODING_PARAM},
{ NULL },
};

@@ -3435,6 +3436,45 @@ static void build_chunks(MOVTrack *trk)
}
}

+/** Assign track ids. If use_stream_ids_as_track_ids is set, the
+ stream ids are used as track ids special case is taken to generate
+ track ids for generated tracks, which don't have a 1:1 stream to
+ copy the stream id from. This the order of tracks is the same in
+ both mov->tracks and s->streams (and no holes). */
+static int mov_setup_track_ids(MOVMuxContext *mov, AVFormatContext *s)
+{
+ int i;
+
+ if (mov->track_ids_ok)
+ return 0;
+
+ if (mov->use_stream_ids_as_track_ids) {
+ int next_generated_track_id = 0;
+ for (i = 0; i < s->nb_streams; i++) {
+ if (s->streams[i]->id > next_generated_track_id)
+ next_generated_track_id = s->streams[i]->id;
+ }
+
+ for (i = 0; i < mov->nb_streams; i++) {
+ if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
+ continue;
+
+ mov->tracks[i].track_id = i >= s->nb_streams ? ++next_generated_track_id : s->streams[i]->id;
+ }
+ } else {
+ for (i = 0; i < mov->nb_streams; i++) {
+ if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
+ continue;
+
+ mov->tracks[i].track_id = i + 1;
+ }
+ }
+
+ mov->track_ids_ok = 1;
+
+ return 0;
+}
+
static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
AVFormatContext *s)
{
@@ -3443,12 +3483,13 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
avio_wb32(pb, 0); /* size placeholder*/
ffio_wfourcc(pb, "moov");

+ mov_setup_track_ids(mov, s);
+
for (i = 0; i < mov->nb_streams; i++) {
if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
continue;

mov->tracks[i].time = mov->time;
- mov->tracks[i].track_id = i + 1;

if (mov->tracks[i].entry)
build_chunks(&mov->tracks[i]);
@@ -3529,7 +3570,7 @@ static void param_write_hex(AVIOContext *pb, const char *name, const uint8_t *va
avio_printf(pb, "<param name=\"%s\" value=\"%s\" valuetype=\"data\"/>\n", name, buf);
}

-static int mov_write_isml_manifest(AVIOContext *pb, MOVMuxContext *mov)
+static int mov_write_isml_manifest(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s)
{
int64_t pos = avio_tell(pb);
int i;
@@ -3552,12 +3593,13 @@ static int mov_write_isml_manifest(AVIOContext *pb, MOVMuxContext *mov)
avio_printf(pb, "</head>\n");
avio_printf(pb, "<body>\n");
avio_printf(pb, "<switch>\n");
+
+ mov_setup_track_ids(mov, s);
+
for (i = 0; i < mov->nb_streams; i++) {
MOVTrack *track = &mov->tracks[i];
const char *type;
- /* track->track_id is initialized in write_moov, and thus isn't known
- * here yet */
- int track_id = i + 1;
+ int track_id = track->track_id;

if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) {
type = "video";
@@ -5773,7 +5815,7 @@ static int mov_write_header(AVFormatContext *s)
avio_flush(pb);

if (mov->flags & FF_MOV_FLAG_ISML)
- mov_write_isml_manifest(pb, mov);
+ mov_write_isml_manifest(pb, mov, s);

if (mov->flags & FF_MOV_FLAG_EMPTY_MOOV &&
!(mov->flags & FF_MOV_FLAG_DELAY_MOOV)) {
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index 894a1b0..ea76e39 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -217,6 +217,8 @@ typedef struct MOVMuxContext {

int need_rewrite_extradata;

+ int use_stream_ids_as_track_ids;
+ int track_ids_ok;
} MOVMuxContext;

#define FF_MOV_FLAG_RTP_HINT (1 << 0)
--
2.7.4
Carl Eugen Hoyos
2016-08-23 19:21:01 UTC
Permalink
Post by e***@nokia.com
+/** Assign track ids. If use_stream_ids_as_track_ids is set, the
+ stream ids are used as track ids special case is taken to generate
+ track ids for generated tracks, which don't have a 1:1 stream to
+ copy the stream id from. This the order of tracks is the same in
+ both mov->tracks and s->streams (and no holes). */
The last sentence is missing an important word, I think.

Carl Eugen
Erkki Seppälä
2016-08-24 07:41:58 UTC
Permalink
Post by Carl Eugen Hoyos
Post by e***@nokia.com
+/** Assign track ids. If use_stream_ids_as_track_ids is set, the
+ stream ids are used as track ids special case is taken to generate
+ track ids for generated tracks, which don't have a 1:1 stream to
+ copy the stream id from. This the order of tracks is the same in
+ both mov->tracks and s->streams (and no holes). */
The last sentence is missing an important word, I think.
In fact probably the whole paragraph needs some rewriting :). Thanks for
review!
e***@nokia.com
2016-08-23 09:03:20 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Now force_codec_ids supports AVMEDIA_TYPE_DATA and
avformat_query_codec accepts data codecs as well in addition to video,
audio and subtitle tracks.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/utils.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavformat/utils.c b/libavformat/utils.c
index dd9df92..d7f3c7a 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -612,6 +612,10 @@ static void force_codec_ids(AVFormatContext *s, AVStream *st)
if (s->subtitle_codec_id)
st->codecpar->codec_id = s->subtitle_codec_id;
break;
+ case AVMEDIA_TYPE_DATA:
+ if (s->data_codec_id)
+ st->codec->codec_id = s->data_codec_id;
+ break;
}
}

@@ -4601,7 +4605,8 @@ int avformat_query_codec(const AVOutputFormat *ofmt, enum AVCodecID codec_id,
return !!av_codec_get_tag2(ofmt->codec_tag, codec_id, &codec_tag);
else if (codec_id == ofmt->video_codec ||
codec_id == ofmt->audio_codec ||
- codec_id == ofmt->subtitle_codec)
+ codec_id == ofmt->subtitle_codec ||
+ codec_id == ofmt->data_codec)
return 1;
}
return AVERROR_PATCHWELCOME;
--
2.7.4
Michael Niedermayer
2016-08-23 11:51:59 UTC
Permalink
Post by e***@nokia.com
Now force_codec_ids supports AVMEDIA_TYPE_DATA and
avformat_query_codec accepts data codecs as well in addition to video,
audio and subtitle tracks.
---
libavformat/utils.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
applied

thx

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No human being will ever know the Truth, for even if they happen to say it
by chance, they would not even known they had done so. -- Xenophanes
e***@nokia.com
2016-08-23 09:03:37 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
doc/examples/Makefile | 1 +
doc/examples/extract_timed_metadata.c | 233 ++++++++++++++++++++++++++++++++++
2 files changed, 234 insertions(+)
create mode 100644 doc/examples/extract_timed_metadata.c

diff --git a/doc/examples/Makefile b/doc/examples/Makefile
index af38159..c10033e 100644
--- a/doc/examples/Makefile
+++ b/doc/examples/Makefile
@@ -16,6 +16,7 @@ EXAMPLES= avio_dir_cmd \
decoding_encoding \
demuxing_decoding \
extract_mvs \
+ extract_timed_metadata \
filtering_video \
filtering_audio \
http_multiclient \
diff --git a/doc/examples/extract_timed_metadata.c b/doc/examples/extract_timed_metadata.c
new file mode 100644
index 0000000..ef84ed9
--- /dev/null
+++ b/doc/examples/extract_timed_metadata.c
@@ -0,0 +1,233 @@
+/*
+ * Copyright (c) 2010 Nicolas George
+ * Copyright (c) 2011 Stefano Sabatini
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+/**
+ * @file
+ * API example for decoding and filtering
+ * @example filtering_video.c
+ */
+
+#define _XOPEN_SOURCE 600 /* for usleep */
+#include <unistd.h>
+#include <stdint.h>
+
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/opt.h>
+#include <assert.h>
+#include <ctype.h>
+
+#define MAX_METADATA_STREAMS 10
+
+static AVFormatContext *fmt_ctx;
+static AVCodecContext *dec_ctx[MAX_METADATA_STREAMS];
+static int metadata_stream_indices[MAX_METADATA_STREAMS + 1] = { -1 }; /* terminated with -1 */
+
+static int open_input_file(const char *filename)
+{
+ int ret;
+ AVCodec *dec = avcodec_find_decoder(AV_CODEC_ID_META);
+ int i;
+ int nb_metadata = 0;
+
+ if ((ret = avformat_open_input(&fmt_ctx, filename, NULL, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot open input file\n");
+ return ret;
+ }
+
+ if ((ret = avformat_find_stream_info(fmt_ctx, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot find stream information\n");
+ return ret;
+ }
+
+ for (i = 0; i < fmt_ctx->nb_streams; i++) {
+ if (fmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_DATA) {
+ assert(nb_metadata < MAX_METADATA_STREAMS);
+ metadata_stream_indices[nb_metadata] = i;
+ nb_metadata++;
+ }
+ }
+ metadata_stream_indices[nb_metadata] = -1;
+
+ if (nb_metadata == 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot find a video stream in the input file\n");
+ return ret;
+ }
+
+ for (i = 0; i < nb_metadata; i++) {
+ dec_ctx[i] = avcodec_alloc_context3(dec);
+ if ((ret = avcodec_open2(dec_ctx[i], dec, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot open metadata decoder for metadata stream on track %d\n", i);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void* memndup(void* ptr, int n)
+{
+ void* buffer = malloc(n);
+ memcpy(buffer, ptr, n);
+ return buffer;
+}
+
+static char* local_strndup(char* str, int n)
+{
+ int len = strlen(str);
+ if (len > n) {
+ len = n;
+ }
+ char* buffer = malloc(n + 1);
+ memcpy(buffer, str, n);
+ buffer[len] = 0;
+ return buffer;
+}
+
+int main(int argc, char **argv)
+{
+ int ret;
+ AVPacket packet;
+ AVData *metadata = avdata_alloc();
+ int got_frame;
+ int i;
+
+ if (!metadata) {
+ perror("Could not allocate metadata");
+ exit(1);
+ }
+ if (argc != 2) {
+ fprintf(stderr, "Usage: %s file\n", argv[0]);
+ exit(1);
+ }
+
+ av_register_all();
+
+ if ((ret = open_input_file(argv[1])) < 0)
+ goto end;
+
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ char *uri = NULL;
+ int nb_src_idxs;
+ int *src_idxs = NULL;
+ char *tref_tag = NULL;
+ int size;
+ int trefsSize;
+ AVTimedMetadata* metadata =
+ (AVTimedMetadata*) av_stream_get_side_data(fmt_ctx->streams[metadata_stream_indices[i]],
+ AV_PKT_DATA_TIMED_METADATA_INFO,
+ &size);
+ AVTrackReferences* trefs =
+ (AVTrackReferences*) av_stream_get_side_data(fmt_ctx->streams[metadata_stream_indices[i]],
+ AV_PKT_DATA_TRACK_REFERENCES,
+ &trefsSize);
+ if (metadata) {
+ tref_tag = local_strndup(metadata->meta_tag, 4);
+ uri = local_strndup((char*) (metadata + 1), metadata->meta_length);
+ int j;
+ int conf_len = metadata->conf_length;
+ void* conf = memndup((char*) (metadata + 1) + metadata->meta_length, metadata->conf_length);
+ char* conf_tag = local_strndup(metadata->conf_tag, 4);
+ nb_src_idxs = trefs ? trefs->nb_tracks : 0;
+ src_idxs = (int*) (trefs + 1);
+ printf("Track #%d describing track%s with tref %s",
+ metadata_stream_indices[i] + 1,
+ nb_src_idxs > 1 ? "s" : "",
+ tref_tag);
+ for (j = 0; j < nb_src_idxs; j++)
+ printf(" #%d", src_idxs[j]);
+ printf(" url: %s \n", uri);
+ printf(" configuration conf tag %s, %d bytes:", conf_tag, conf_len);
+ for (j = 0; j < conf_len; ++j) {
+ unsigned char ch = ((unsigned char*) conf)[j];
+ if (isprint(ch)) {
+ printf(" %c", ch);
+ } else {
+ printf(" %2x", ((unsigned char*) conf)[j]);
+ }
+ }
+ printf("\n");
+ free(conf);
+ free(conf_tag);
+ free(uri);
+ free(tref_tag);
+ } else {
+ printf("No meta data info for track %d?!\n", metadata_stream_indices[i]);
+ }
+ }
+
+ /* read all packets */
+ while (1) {
+ if ((ret = av_read_frame(fmt_ctx, &packet)) < 0)
+ break;
+
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ if (packet.stream_index == metadata_stream_indices[i]) {
+ got_frame = 0;
+
+ ret = dec_ctx[i]->codec->decode(dec_ctx[i], metadata, &got_frame, &packet);
+
+ if (ret < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Error decoding meta data\n");
+ break;
+ }
+
+ if (got_frame) {
+ int c;
+ printf("track #%d at %" PRId64 " %d, ",
+ packet.stream_index + 1,
+ metadata->pts,
+ metadata->data->size);
+ for (c = 0; c < metadata->data->size; ++c) {
+ char ch = ((char*) metadata->data->data)[c];
+ if (ch == '\\') {
+ printf("\\\\");
+ } else if (isprint(ch)) {
+ putchar(ch);
+ } else {
+ printf("\\0x%2x", (unsigned char) ch);
+ }
+ }
+ putchar('\n');
+ } else {
+ printf("no frame..\n");
+ }
+ }
+ }
+ av_packet_unref(&packet);
+ }
+end:
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ avcodec_close(dec_ctx[i]);
+ }
+ avformat_close_input(&fmt_ctx);
+ avdata_free(metadata);
+
+ if (ret < 0 && ret != AVERROR_EOF) {
+ fprintf(stderr, "Error occurred: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ exit(0);
+}
--
2.7.4
Michael Niedermayer
2016-08-23 14:29:13 UTC
Permalink
Post by e***@nokia.com
---
doc/examples/Makefile | 1 +
doc/examples/extract_timed_metadata.c | 233 ++++++++++++++++++++++++++++++++++
2 files changed, 234 insertions(+)
create mode 100644 doc/examples/extract_timed_metadata.c
diff --git a/doc/examples/Makefile b/doc/examples/Makefile
index af38159..c10033e 100644
--- a/doc/examples/Makefile
+++ b/doc/examples/Makefile
@@ -16,6 +16,7 @@ EXAMPLES= avio_dir_cmd \
decoding_encoding \
demuxing_decoding \
extract_mvs \
+ extract_timed_metadata \
filtering_video \
filtering_audio \
http_multiclient \
diff --git a/doc/examples/extract_timed_metadata.c b/doc/examples/extract_timed_metadata.c
new file mode 100644
index 0000000..ef84ed9
--- /dev/null
+++ b/doc/examples/extract_timed_metadata.c
@@ -0,0 +1,233 @@
+/*
+ * Copyright (c) 2010 Nicolas George
+ * Copyright (c) 2011 Stefano Sabatini
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+/**
+ * API example for decoding and filtering
+ */
+
+#define _XOPEN_SOURCE 600 /* for usleep */
+#include <unistd.h>
+#include <stdint.h>
+
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/opt.h>
+#include <assert.h>
+#include <ctype.h>
+
+#define MAX_METADATA_STREAMS 10
+
+static AVFormatContext *fmt_ctx;
+static AVCodecContext *dec_ctx[MAX_METADATA_STREAMS];
+static int metadata_stream_indices[MAX_METADATA_STREAMS + 1] = { -1 }; /* terminated with -1 */
+
+static int open_input_file(const char *filename)
+{
+ int ret;
+ AVCodec *dec = avcodec_find_decoder(AV_CODEC_ID_META);
+ int i;
+ int nb_metadata = 0;
+
+ if ((ret = avformat_open_input(&fmt_ctx, filename, NULL, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot open input file\n");
+ return ret;
+ }
+
+ if ((ret = avformat_find_stream_info(fmt_ctx, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot find stream information\n");
+ return ret;
+ }
+
+ for (i = 0; i < fmt_ctx->nb_streams; i++) {
+ if (fmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_DATA) {
+ assert(nb_metadata < MAX_METADATA_STREAMS);
+ metadata_stream_indices[nb_metadata] = i;
+ nb_metadata++;
+ }
+ }
+ metadata_stream_indices[nb_metadata] = -1;
+
+ if (nb_metadata == 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot find a video stream in the input file\n");
+ return ret;
+ }
+
+ for (i = 0; i < nb_metadata; i++) {
+ dec_ctx[i] = avcodec_alloc_context3(dec);
+ if ((ret = avcodec_open2(dec_ctx[i], dec, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot open metadata decoder for metadata stream on track %d\n", i);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void* memndup(void* ptr, int n)
+{
+ void* buffer = malloc(n);
+ memcpy(buffer, ptr, n);
+ return buffer;
+}
+
+static char* local_strndup(char* str, int n)
+{
+ int len = strlen(str);
+ if (len > n) {
+ len = n;
+ }
+ char* buffer = malloc(n + 1);
+ memcpy(buffer, str, n);
+ buffer[len] = 0;
+ return buffer;
+}
+
+int main(int argc, char **argv)
+{
+ int ret;
+ AVPacket packet;
+ AVData *metadata = avdata_alloc();
+ int got_frame;
+ int i;
+
+ if (!metadata) {
+ perror("Could not allocate metadata");
+ exit(1);
+ }
+ if (argc != 2) {
+ fprintf(stderr, "Usage: %s file\n", argv[0]);
+ exit(1);
+ }
+
+ av_register_all();
+
+ if ((ret = open_input_file(argv[1])) < 0)
+ goto end;
+
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ char *uri = NULL;
+ int nb_src_idxs;
+ int *src_idxs = NULL;
+ char *tref_tag = NULL;
+ int size;
+ int trefsSize;
+ AVTimedMetadata* metadata =
+ (AVTimedMetadata*) av_stream_get_side_data(fmt_ctx->streams[metadata_stream_indices[i]],
+ AV_PKT_DATA_TIMED_METADATA_INFO,
+ &size);
+ AVTrackReferences* trefs =
+ (AVTrackReferences*) av_stream_get_side_data(fmt_ctx->streams[metadata_stream_indices[i]],
+ AV_PKT_DATA_TRACK_REFERENCES,
+ &trefsSize);
+ if (metadata) {
+ tref_tag = local_strndup(metadata->meta_tag, 4);
+ uri = local_strndup((char*) (metadata + 1), metadata->meta_length);
+ int j;
+ int conf_len = metadata->conf_length;
+ void* conf = memndup((char*) (metadata + 1) + metadata->meta_length, metadata->conf_length);
+ char* conf_tag = local_strndup(metadata->conf_tag, 4);
+ nb_src_idxs = trefs ? trefs->nb_tracks : 0;
+ src_idxs = (int*) (trefs + 1);
+ printf("Track #%d describing track%s with tref %s",
+ metadata_stream_indices[i] + 1,
+ nb_src_idxs > 1 ? "s" : "",
+ tref_tag);
+ for (j = 0; j < nb_src_idxs; j++)
+ printf(" #%d", src_idxs[j]);
+ printf(" url: %s \n", uri);
+ printf(" configuration conf tag %s, %d bytes:", conf_tag, conf_len);
+ for (j = 0; j < conf_len; ++j) {
+ unsigned char ch = ((unsigned char*) conf)[j];
+ if (isprint(ch)) {
+ printf(" %c", ch);
+ } else {
+ printf(" %2x", ((unsigned char*) conf)[j]);
+ }
+ }
+ printf("\n");
+ free(conf);
+ free(conf_tag);
+ free(uri);
+ free(tref_tag);
+ } else {
+ printf("No meta data info for track %d?!\n", metadata_stream_indices[i]);
+ }
+ }
+
+ /* read all packets */
+ while (1) {
+ if ((ret = av_read_frame(fmt_ctx, &packet)) < 0)
+ break;
+
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ if (packet.stream_index == metadata_stream_indices[i]) {
+ got_frame = 0;
+
+ ret = dec_ctx[i]->codec->decode(dec_ctx[i], metadata, &got_frame, &packet);
use of private libavcodec API is not allowed by external applications
like an example would be

see avcodec.h:
/*****************************************************************
* No fields below this line are part of the public API. They
* may not be used outside of libavcodec and can be changed and
* removed at will.
* New public fields should be added right above.
*****************************************************************

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When you are offended at any man's fault, turn to yourself and study your
own failings. Then you will forget your anger. -- Epictetus
Erkki Seppälä
2016-08-25 07:31:57 UTC
Permalink
Thanks for pointing out the use of private API. It seemed that this
would have needed to add a new function for the API for decoding timed
metadata frames, but probably a better solution is to use the
avcodec_receive_packet framework for this as the old API is deprecated
anyway?

Below is a patch to introduce the functionality (example edited for
brevity). If this seems like the way to, I'll add this to v2 of the
patches. (I may squash the first patch with one of the other patches if
it fits in.)

commit ddfb745109768a169e93c221092161d39c8f8208
Author: Erkki Seppälä <***@nokia.com>
Date: Thu Aug 25 10:21:15 2016 +0300

libavcodec/utils: do_decode now supports AVMEDIA_TYPE_DATA

This allows using avcodec_send_packet with data frames (ie. timed
metadata)

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index 138125a..8b55464 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -2737,6 +2737,10 @@ static int do_decode(AVCodecContext *avctx,
AVPacket *pkt)
} else if (avctx->codec_type == AVMEDIA_TYPE_AUDIO) {
ret = avcodec_decode_audio4(avctx, avctx->internal->buffer_frame,
&got_frame, pkt);
+ } else if (avctx->codec_type == AVMEDIA_TYPE_DATA) {
+ ret = avctx->codec->decode(avctx,
avctx->internal->buffer_frame, &got_frame, pkt);
+ if (ret == 0 && got_frame)
+ ret = pkt->size;
} else {
ret = AVERROR(EINVAL);
}

commit ebfceb706d8c8d0dbfe64ebe06d218aaa8807e43
Author: Erkki Seppälä <***@nokia.com>
Date: Thu Aug 25 10:22:19 2016 +0300

fixup! doc/examples/extract_timed_metadata: added a bare-bones
metadata extractor; find only the frames

diff --git a/doc/examples/extract_timed_metadata.c
b/doc/examples/extract_timed_metadata.c
index 48fb877..03da2b7 100644
--- a/doc/examples/extract_timed_metadata.c
+++ b/doc/examples/extract_timed_metadata.c
@@ -184,16 +183,15 @@ int main(int argc, char **argv)

for (i = 0; metadata_stream_indices[i] >= 0; i++) {
if (packet.stream_index == metadata_stream_indices[i]) {
- got_frame = 0;
-
- ret = dec_ctx[i]->codec->decode(dec_ctx[i], metadata,
&got_frame, &packet);
+ ret = avcodec_send_packet(dec_ctx[i], &packet);

+ // We always empty decoded frames so we don't handle
AVERROR(EAGAIN) here
if (ret < 0) {
av_log(NULL, AV_LOG_ERROR, "Error decoding meta
data\n");
break;
}

- if (got_frame) {
+ while (avcodec_receive_frame(dec_ctx[i], metadata) == 0) {
int c;
printf("track #%d at %" PRId64 " %d, ",
packet.stream_index + 1,
Michael Niedermayer
2016-08-31 11:01:44 UTC
Permalink
Hi
Post by Erkki Seppälä
Thanks for pointing out the use of private API. It seemed that this
would have needed to add a new function for the API for decoding
timed metadata frames, but probably a better solution is to use the
avcodec_receive_packet framework for this as the old API is
deprecated anyway?
thats an option yes
Post by Erkki Seppälä
Below is a patch to introduce the functionality (example edited for
brevity). If this seems like the way to, I'll add this to v2 of the
patches. (I may squash the first patch with one of the other patches
if it fits in.)
it seems ok, assuming it works

thx

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Rewriting code that is poorly written but fully understood is good.
Rewriting code that one doesnt understand is a sign that one is less smart
then the original author, trying to rewrite it will not make it better.
e***@nokia.com
2016-08-23 09:03:19 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

This allows copying an array so that is compatible with the array
reallocation functions. av_memdup won't do, as it uses av_malloc
underneath, but this one uses av_realloc_array for the allocation.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavutil/mem.c | 11 +++++++++++
libavutil/mem.h | 11 +++++++++++
2 files changed, 22 insertions(+)

diff --git a/libavutil/mem.c b/libavutil/mem.c
index 1a8fc21..c74374e 100644
--- a/libavutil/mem.c
+++ b/libavutil/mem.c
@@ -307,6 +307,17 @@ void *av_memdup(const void *p, size_t size)
return ptr;
}

+void *av_arraydup(const void *p, size_t nmemb, size_t size)
+{
+ void *ptr = NULL;
+ if (p) {
+ ptr = av_realloc_array(NULL, nmemb, size);
+ if (ptr)
+ memcpy(ptr, p, nmemb * size);
+ }
+ return ptr;
+}
+
int av_dynarray_add_nofree(void *tab_ptr, int *nb_ptr, void *elem)
{
void **tab;
diff --git a/libavutil/mem.h b/libavutil/mem.h
index 7f0c610..08ed520 100644
--- a/libavutil/mem.h
+++ b/libavutil/mem.h
@@ -514,6 +514,17 @@ char *av_strndup(const char *s, size_t len) av_malloc_attrib;
void *av_memdup(const void *p, size_t size);

/**
+ * Duplicate the array p. This array is compatible with the av_realloc
+ * functions.
+ * @param p array to be duplicated
+ * @param nmemb number of elements in the array
+ * @param size size of an element in the array
+ * @return Pointer to a newly allocated array containing a
+ * copy of p or NULL if the buffer cannot be allocated.
+ */
+void *av_arraydup(const void *p, size_t nmemb, size_t size);
+
+/**
* Overlapping memcpy() implementation.
*
* @param dst Destination buffer
--
2.7.4
Michael Niedermayer
2016-08-23 11:05:21 UTC
Permalink
Post by e***@nokia.com
This allows copying an array so that is compatible with the array
reallocation functions. av_memdup won't do, as it uses av_malloc
underneath, but this one uses av_realloc_array for the allocation.
on which platform does av_memdup() not work with av_realloc_array() ?

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 3
"Rare item" - "Common item with rare defect or maybe just a lie"
"Professional" - "'Toy' made in china, not functional except as doorstop"
"Experts will know" - "The seller hopes you are not an expert"
Erkki Seppälä
2016-08-23 11:10:28 UTC
Permalink
Post by Michael Niedermayer
on which platform does av_memdup() not work with av_realloc_array() ?
I cannot indicate such a platform. However, the documentation for
av_malloc says:

"Pointers originating from the av_malloc() family of functions
must not be passed to av_realloc(). The former can be
implemented using memalign() (or other functions), and
there is no guarantee that pointers from such functions
can be passed to realloc() at all. The situation is
undefined according to POSIX and may crash with some
libc implementations."

And av_memdup uses av_malloc.
Michael Niedermayer
2016-08-23 11:18:54 UTC
Permalink
Post by Erkki Seppälä
Post by Michael Niedermayer
on which platform does av_memdup() not work with av_realloc_array() ?
I cannot indicate such a platform. However, the documentation for
"Pointers originating from the av_malloc() family of functions
must not be passed to av_realloc(). The former can be
implemented using memalign() (or other functions), and
there is no guarantee that pointers from such functions
can be passed to realloc() at all. The situation is
undefined according to POSIX and may crash with some
libc implementations."
And av_memdup uses av_malloc.
see:
0421 15:12 Michael Niederm (3.9K) [FFmpeg-devel] [PATCH] avutil/mem: remove av_realloc / av_malloc incompatibility warning
i think thats a better choice than maintaining all memory allocation
in 2 incompatible systems

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Many things microsoft did are stupid, but not doing something just because
microsoft did it is even more stupid. If everything ms did were stupid they
would be bankrupt already.
Erkki Seppälä
2016-08-23 11:36:19 UTC
Permalink
Post by Michael Niedermayer
0421 15:12 Michael Niederm (3.9K) [FFmpeg-devel] [PATCH] avutil/mem: remove av_realloc / av_malloc incompatibility warning
i think thats a better choice than maintaining all memory allocation
in 2 incompatible systems
Well, it's still not completely useless even if redundant, given that
the interface is more similar to av_malloc_array and av_realloc_array.
Code that resizes arrays needing to make a copy of the array would be
more uniform with a function such as av_arraydup (better called
av_dup_array?).

But that's only a philosophical reason. And now checking for my use
scenario I noticed that the code that made use of the function does not
exist anymore, so this patch can be just dropped.
e***@nokia.com
2016-08-23 09:03:34 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Added support for passing complex channel layout configuration as side
packet data (AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT,
AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT,
AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED) to ISO media files
as specified by ISO/IEC 14496-12.

This information isn't integrated into the existing channel layout
system though, which is much more restricted compared to what the
standard permits. However, the side packet data is structured so that
it does not require too much ISO base media file format knowledge in
client code.

This information ends up to an (optional) chnl box in the written file
in an isom track.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 51 ++++++++++++++++++++++++++++++++-
libavformat/movenc.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 130 insertions(+), 2 deletions(-)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 36c85e9..6c64e6a 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1365,6 +1365,35 @@ typedef struct AVTrackReferences {
/** followed by: int tracks[nb_tracks]; -- tracks this track refers to */
} AVTrackReferences;

+/** Describes the speaker position of a single audio channel of a single track */
+typedef struct AVAudioTrackChannelPosition {
+ int speaker_position; /** an OutputChannelPosition from ISO/IEC 23001-8 */
+
+ /** The following are used if speaker_position == 126 */
+ int azimuth;
+ int elevation;
+} AVAudioTrackChannelPosition;
+
+/** Describes the channel layout (ie. speaker position) of a single audio track */
+typedef struct AVAudioTrackChannelLayout {
+ int nb_positions;
+ /** followed by: AVAudioTrackChannelPosition positions[nb_positions]; */
+} AVAudioTrackChannelLayout;
+
+/** Describes the channel layout based on predefined layout of a single track
+ by providing the layout and the list of channels are are omitted */
+typedef struct AVAudioTrackChannelPredefinedLayout {
+ int layout; /** ChannelConfiguration from ISO/IEC 23001-8 */
+ int nb_omitted_channels;
+ /** followed by: char omitted_channels[nb_omitted_channels]; - non-zero indicates the channel is omitted */
+} AVAudioTrackChannelPredefinedLayout;
+
+/** Describes the channel layout to be object-sturctued with given
+ number of objects */
+typedef struct AVAudioTrackChannelLayoutObjectStructured {
+ int object_count;
+} AVAudioTrackChannelLayoutObjectStructured;
+
enum AVPacketSideDataType {
AV_PKT_DATA_PALETTE,

@@ -1556,7 +1585,27 @@ enum AVPacketSideDataType {
* Configured the timed metadata parameters, such as the uri and
* meta data configuration. The key is of type AVTimedMetadata.
*/
- AV_PKT_DATA_TIMED_METADATA_INFO
+ AV_PKT_DATA_TIMED_METADATA_INFO,
+
+ /**
+ * Channel layout, describing the position of spakers for the
+ * channels of a track, following the structure
+ * AVAudioTrackChannelLayout.
+ */
+ AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT,
+
+ /**
+ * Predefined channel layout, describing the position of spakers
+ * for the channels of a track, following the structure
+ * AVAudioTrackChannelPredefinedLayout.
+ */
+ AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT,
+
+ /**
+ * The channel layout is object structured with the number of objects in
+ * AVAudioTrackChannelLayoutObjectStructured
+ */
+ AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index ff4bf85..9606918 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -557,6 +557,83 @@ static unsigned compute_avg_bitrate(MOVTrack *track)
return size * 8 * track->timescale / track->track_duration;
}

+static int mov_write_chnl_tag(AVIOContext *pb, MOVTrack *track)
+{
+ AVAudioTrackChannelLayout *layout =
+ (void*) av_stream_get_side_data(track->st, AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT,
+ NULL);
+
+ AVAudioTrackChannelPredefinedLayout *predefLayout =
+ (void*) av_stream_get_side_data(track->st, AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT,
+ NULL);
+
+ AVAudioTrackChannelLayoutObjectStructured *objectStructured =
+ (void*) av_stream_get_side_data(track->st, AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED,
+ NULL);
+
+ if (!layout && !predefLayout && !objectStructured) {
+ return 0;
+ } else {
+ int64_t pos = avio_tell(pb);
+
+ int channel_structured = !!layout || !!predefLayout;
+
+ // ChannelConfiguration from ISO/IEC 23001-8
+ int defined_layout = predefLayout ? predefLayout->layout : 0;
+ int channel_count = track->par->channels;
+
+ int object_structured = !!objectStructured;
+ int object_count = objectStructured ? objectStructured->object_count : 0;
+
+ int stream_structure = (channel_structured << 0) | (object_structured << 1);
+
+ avio_wb32(pb, 0); // size
+ ffio_wfourcc(pb, "chnl");
+ avio_wb32(pb, 0); // Version
+
+ avio_w8(pb, stream_structure);
+
+ if (channel_structured) {
+ avio_w8(pb, defined_layout);
+ if (defined_layout == 0) {
+ AVAudioTrackChannelPosition* positions;
+ int i;
+ av_assert0(layout);
+ av_assert0(layout->nb_positions >= channel_count);
+
+ positions = (void*) (layout + 1);
+
+ for (i = 0; i < channel_count; ++i) {
+ AVAudioTrackChannelPosition *pos = &positions[i];
+ avio_w8(pb, pos->speaker_position);
+ if (pos->speaker_position == 126) {
+ avio_wb16(pb, pos->azimuth);
+ avio_w8(pb, pos->elevation);
+ }
+ }
+ } else {
+ int omitted_channels_map[64 / 8] = { 0 };
+ char* omitted_channels;
+ int i;
+ av_assert0(predefLayout);
+
+ omitted_channels = (void*) (predefLayout + 1);
+
+ for (i = 0; i < FFMIN(64, predefLayout->nb_omitted_channels); ++i) {
+ omitted_channels_map[i / 8] |=
+ ((!!omitted_channels[i]) << (i % 8));
+ }
+ for (i = 0; i < 64 / 8; ++i)
+ avio_w8(pb, omitted_channels_map[i]);
+ }
+ }
+ if (object_structured)
+ avio_w8(pb, object_count);
+
+ return update_size(pb, pos);
+ }
+}
+
static int mov_write_esds_tag(AVIOContext *pb, MOVTrack *track) // Basic
{
AVCPBProperties *props;
@@ -996,8 +1073,10 @@ static int mov_write_audio_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex
(mov_pcm_le_gt16(track->par->codec_id) && version==1) ||
(mov_pcm_be_gt16(track->par->codec_id) && version==1)))
mov_write_wave_tag(s, pb, track);
- else if (track->tag == MKTAG('m','p','4','a'))
+ else if (track->tag == MKTAG('m','p','4','a')) {
+ mov_write_chnl_tag(pb, track);
mov_write_esds_tag(pb, track);
+ }
else if (track->par->codec_id == AV_CODEC_ID_AMR_NB)
mov_write_amr_tag(pb, track);
else if (track->par->codec_id == AV_CODEC_ID_AC3)
--
2.7.4
Nicolas George
2016-08-23 17:49:35 UTC
Permalink
Post by e***@nokia.com
Added support for passing complex channel layout configuration as side
packet data (AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT,
AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT,
AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED) to ISO media files
as specified by ISO/IEC 14496-12.
This information isn't integrated into the existing channel layout
system though,
I think it needs to be.
Post by e***@nokia.com
which is much more restricted compared to what the
standard permits.
Certainly, but the bitfield channel layout system is enough for most cases
and well tested. Any new system must work with it, not around it.
Post by e***@nokia.com
However, the side packet data is structured so that
it does not require too much ISO base media file format knowledge in
client code.
This information ends up to an (optional) chnl box in the written file
in an isom track.
---
libavcodec/avcodec.h | 51 ++++++++++++++++++++++++++++++++-
libavformat/movenc.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 130 insertions(+), 2 deletions(-)
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 36c85e9..6c64e6a 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1365,6 +1365,35 @@ typedef struct AVTrackReferences {
/** followed by: int tracks[nb_tracks]; -- tracks this track refers to */
} AVTrackReferences;
+/** Describes the speaker position of a single audio channel of a single track */
Please follow the surrounding style for Doxygen comments: /** and */ on a
separate line, * at the beginning of each line.
Post by e***@nokia.com
+typedef struct AVAudioTrackChannelPosition {
+ int speaker_position; /** an OutputChannelPosition from ISO/IEC 23001-8 */
I think most our API users and even developers have better use for 158 Swiss
Francs than feeding the ISO. Please make the API self-sufficient; I suggest
an enum.
Post by e***@nokia.com
+
+ /** The following are used if speaker_position == 126 */
+ int azimuth;
+ int elevation;
Please document the units and references.

Also, since it is repeated and packed, using a small type would make sense
here if possible.
Post by e***@nokia.com
+} AVAudioTrackChannelPosition;
+
+/** Describes the channel layout (ie. speaker position) of a single audio track */
+typedef struct AVAudioTrackChannelLayout {
+ int nb_positions;
+ /** followed by: AVAudioTrackChannelPosition positions[nb_positions]; */
AVAudioTrackChannelPosition positions[0];

That way, computing the address is taken care of by the compiler without
tricky pointer arithmetic, including alignment requirements.

(We already use 0-sized final arrays once in vaapi_encode.h; since VA API is
for Linux and BSD, it does not probe the crappy proprietary compilers
support it. In that case, using 1 instead of 0 is fine.)
Post by e***@nokia.com
+} AVAudioTrackChannelLayout;
+
+/** Describes the channel layout based on predefined layout of a single track
+ by providing the layout and the list of channels are are omitted */
Could you make that clearer?
Post by e***@nokia.com
+typedef struct AVAudioTrackChannelPredefinedLayout {
+ int layout; /** ChannelConfiguration from ISO/IEC 23001-8 */
+ int nb_omitted_channels;
+ /** followed by: char omitted_channels[nb_omitted_channels]; - non-zero indicates the channel is omitted */
Is there a reasonable upper bound to nb_omitted_channels?

Also, I think the naming is inconsistent: if nb_omitted_channels = 3 and
omitted_channels[] = { 1, 0, 1 }, there are only two omitted channels, not
three.
Post by e***@nokia.com
+} AVAudioTrackChannelPredefinedLayout;
+
+/** Describes the channel layout to be object-sturctued with given
+ number of objects */
Please make that clearer.
Post by e***@nokia.com
+typedef struct AVAudioTrackChannelLayoutObjectStructured {
+ int object_count;
+} AVAudioTrackChannelLayoutObjectStructured;
+
enum AVPacketSideDataType {
AV_PKT_DATA_PALETTE,
@@ -1556,7 +1585,27 @@ enum AVPacketSideDataType {
* Configured the timed metadata parameters, such as the uri and
* meta data configuration. The key is of type AVTimedMetadata.
*/
- AV_PKT_DATA_TIMED_METADATA_INFO
+ AV_PKT_DATA_TIMED_METADATA_INFO,
+
+ /**
+ * Channel layout, describing the position of spakers for the
+ * channels of a track, following the structure
+ * AVAudioTrackChannelLayout.
+ */
+ AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT,
I think it would be a good idea if the name of the side data matches the
name of the structure as much as possible: Track -> _TRACK_. Ditto for the
others.

Would it make sense to have a single side data type and an integer field at
the beginning announcing the type of the rest:

typedef struct AVComplexChannelLayout {
enum AVComplexChannelLayoutType type;
union {
uint64_t bitmask;
AVAudioTrackChannelLayout complete;
AVAudioTrackChannelPredefinedLayout predefined;
AVAudioTrackChannelLayoutObjectStructured object;
} layout;
};
Post by e***@nokia.com
+
+ /**
+ * Predefined channel layout, describing the position of spakers
+ * for the channels of a track, following the structure
+ * AVAudioTrackChannelPredefinedLayout.
+ */
+ AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT,
+
+ /**
+ * The channel layout is object structured with the number of objects in
+ * AVAudioTrackChannelLayoutObjectStructured
+ */
+ AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED
Missing final comma (ditto in the patch adding
AV_PKT_DATA_TIMED_METADATA_INFO; blame to Neil for not adding it to
AV_PKT_DATA_MASTERING_DISPLAY_METADATA).
Post by e***@nokia.com
};
#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index ff4bf85..9606918 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
I can not comment on that file.

Regards,
--
Nicolas George
Erkki Seppälä
2016-08-24 14:39:53 UTC
Permalink
Post by Nicolas George
Post by e***@nokia.com
+ int speaker_position; /** an OutputChannelPosition from ISO/IEC 23001-8 */
I think most our API users and even developers have better use for 158 Swiss
Francs than feeding the ISO. Please make the API self-sufficient; I suggest
an enum.
I'll introduce an enumeration AVSpeakerPosition. However, the spec is
available publicly at
http://standards.iso.org/ittf/PubliclyAvailableStandards/c062088_ISO_IEC_23001-8_2013.zip
. (And the table of output channel positions starts at page 24.)
Post by Nicolas George
Post by e***@nokia.com
Added support for passing complex channel layout configuration as side
packet data (AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT,
AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT,
AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED) to ISO media files
as specified by ISO/IEC 14496-12.
This information isn't integrated into the existing channel layout
system though,
I think it needs to be.
Unfortunately properly integrating this information into FFmpeg sounds
larger than the scope of my project. I imagine this still would be very
helpful base work should such information be actually supported by
various parts of FFmpeg.

A special bit in the AV_CH_LAYOUT_* could be used for indicating that a
special layout exists, but that information by itself is nowhere enough
to work with this data with ie. in the functions available in
channel_layout.c. And if codec or client code is able to deal with the
advanced channel layout configurations, they also have access to this
side data.

I also believe is is easier to study more deeper integration once a
non-intrusive possibility to work with advanced ISOBMFF channel layouts
exists.

Here is how the mapping on speaker position level might be done:

/** Speaker positions according to ISO/IEC 23001-8 */
enum AVSpeakerPosition {
! AV_SPEAKER_POSITION_L = 0, /// Left front
! AV_SPEAKER_POSITION_R = 1, /// Right front
AV_SPEAKER_POSITION_C = 2, /// Centre front
AV_SPEAKER_POSITION_LFE = 3, /// Low frequency enhancement
AV_SPEAKER_POSITION_LS = 4, /// Left surround
AV_SPEAKER_POSITION_RS = 5, /// Right surround
! AV_SPEAKER_POSITION_LC = 6, /// Left front centre
! AV_SPEAKER_POSITION_RC = 7, /// Right front centre
AV_SPEAKER_POSITION_LSR = 8, /// Rear surround left
AV_SPEAKER_POSITION_RSR = 9, /// Rear surround right
AV_SPEAKER_POSITION_CS = 10, /// Rear centre
AV_SPEAKER_POSITION_LSD = 11, /// Left surround direct
AV_SPEAKER_POSITION_RSD = 12, /// Right surround direct
AV_SPEAKER_POSITION_LSS = 13, /// Left side surround
AV_SPEAKER_POSITION_RSS = 14, /// Right side surround
! AV_SPEAKER_POSITION_LW = 15, /// Left wide front
! AV_SPEAKER_POSITION_RW = 16, /// Right wide front
AV_SPEAKER_POSITION_LV = 17, /// Left front vertical height
AV_SPEAKER_POSITION_RV = 18, /// Right front vertical height
AV_SPEAKER_POSITION_CV = 19, /// Centre front vertical height
AV_SPEAKER_POSITION_LVR = 20, /// Left surround vertical height
rear
AV_SPEAKER_POSITION_RVR = 21, /// Right surround vertical
height rear
AV_SPEAKER_POSITION_CVR = 22, /// Centre vertical height rear
AV_SPEAKER_POSITION_LVSS = 23, /// Left vertical height side
surround
AV_SPEAKER_POSITION_RVSS = 24, /// Right vertical height side
surround
AV_SPEAKER_POSITION_TS = 25, /// Top centre surround
AV_SPEAKER_POSITION_LFE2 = 26, /// E2 Low frequency enhancement 2
AV_SPEAKER_POSITION_LB = 27, /// Left front vertical bottom
AV_SPEAKER_POSITION_RB = 28, /// Right front vertical bottom
AV_SPEAKER_POSITION_CB = 29, /// Centre front vertical bottom
AV_SPEAKER_POSITION_LVS = 30, /// Left vertical height surround
AV_SPEAKER_POSITION_RVS = 31, /// Right vertical height surround
/// 32-45 Reserved
AV_SPEAKER_POSITION_LFE3 = 36, /// E3 Low frequency enhancement 3
AV_SPEAKER_POSITION_LEOS = 37, /// Left edge of screen
AV_SPEAKER_POSITION_REOS = 38, /// Right edge of screen
AV_SPEAKER_POSITION_HWBCAL = 39, /// half-way between centre of
screen and left edge of screen
AV_SPEAKER_POSITION_HWBCAR = 40, /// half-way between centre of
screen and right edge of screen
AV_SPEAKER_POSITION_LBS = 41, /// Left back surround
AV_SPEAKER_POSITION_RBS = 42, /// Right back surround
/// 43–125 Reserved
AV_SPEAKER_POSITION_EXPL = 126, /// Explicit position (see text)
/// 127 Unknown / undefined
};

I suppose if this is preferred, then one for 23001-8 channel layouts
would be preferred as well..

It does seem confusing though that similar #defines are available in
channel_layout.h and I am a bit uncertain of a 1:1 mapping between the
two. I've marked in above with ! the ones that I found a match from the
defines:

AV_CH_FRONT_LEFT AV_SPEAKER_POSITION_L
AV_CH_FRONT_RIGHT AV_SPEAKER_POSITION_R
AV_CH_FRONT_CENTER AV_SPEAKER_POSITION_LC
AV_CH_LOW_FREQUENCY AV_SPEAKER_POSITION_LFE
AV_CH_BACK_LEFT AV_SPEAKER_POSITION_LBS
AV_CH_BACK_RIGHT AV_SPEAKER_POSITION_RBS
AV_CH_FRONT_LEFT_OF_CENTER AV_SPEAKER_POSITION_LC
AV_CH_FRONT_RIGHT_OF_CENTER AV_SPEAKER_POSITION_RC
AV_CH_BACK_CENTER
AV_CH_SIDE_LEFT AV_SPEAKER_POSITION_LSS
AV_CH_SIDE_RIGHT AV_SPEAKER_POSITION_RSS
AV_CH_TOP_CENTER AV_SPEAKER_POSITION_TS
AV_CH_TOP_FRONT_LEFT
AV_CH_TOP_FRONT_CENTER
AV_CH_TOP_FRONT_RIGHT
AV_CH_TOP_BACK_LEFT
AV_CH_TOP_BACK_CENTER
AV_CH_TOP_BACK_RIGHT
AV_CH_STEREO_LEFT
AV_CH_STEREO_RIGHT
AV_CH_WIDE_LEFT AV_SPEAKER_POSITION_LW
AV_CH_WIDE_RIGHT AV_SPEAKER_POSITION_RW
AV_CH_SURROUND_DIRECT_LEFT AV_SPEAKER_POSITION_LSD
AV_CH_SURROUND_DIRECT_RIGHT AV_SPEAKER_POSITION_RSD
AV_CH_LOW_FREQUENCY_2 AV_SPEAKER_POSITION_LFE2

If this approach were taken, then the missing speakers positions
(channels?) would need to be added, conversion maps for 23001-8 would
need to be implemented (and probably refusing if there are non-mappable
positions). Then layout bitmaps would be mapped into 23001-8 ones or if
the layout cannot be found, a custom one generated.. Personally I don't
mind how separate&simple this system still is :).
Post by Nicolas George
Post by e***@nokia.com
which is much more restricted compared to what the
standard permits.
Certainly, but the bitfield channel layout system is enough for most cases
and well tested. Any new system must work with it, not around it.
But as you may agree, such a modification would not be small. Certainly
not one I would try as my first feature to FFmpeg.
Post by Nicolas George
Please follow the surrounding style for Doxygen comments: /** and */ on a
separate line, * at the beginning of each line.
Oops. I'll fix those.
Post by Nicolas George
Post by e***@nokia.com
+ /** The following are used if speaker_position == 126 */
+ int azimuth;
+ int elevation;
Please document the units and references.
Done.
Post by Nicolas George
Also, since it is repeated and packed, using a small type would make sense
here if possible.
I'll use the same types as in the headers: int16_t and int8_t.
Post by Nicolas George
Post by e***@nokia.com
+} AVAudioTrackChannelPosition;
+
+/** Describes the channel layout (ie. speaker position) of a single audio track */
+typedef struct AVAudioTrackChannelLayout {
+ int nb_positions;
+ /** followed by: AVAudioTrackChannelPosition positions[nb_positions]; */
AVAudioTrackChannelPosition positions[0];
That way, computing the address is taken care of by the compiler without
tricky pointer arithmetic, including alignment requirements.
Actually this is the way I had it in the first place, but compiler
warnings convinced me not to do it. If I'll use positions[1] the
allocations will end up slightly too large unless without slightly
convoluted code.

Is there a preferred way? It doesn't seem like foo[1] is used anywhere.
Post by Nicolas George
(We already use 0-sized final arrays once in vaapi_encode.h; since VA API is
for Linux and BSD, it does not probe the crappy proprietary compilers
support it. In that case, using 1 instead of 0 is fine.)
In particular I don't want to do it conditionally :).
Post by Nicolas George
Post by e***@nokia.com
+} AVAudioTrackChannelLayout;
+
+/** Describes the channel layout based on predefined layout of a single track
+ by providing the layout and the list of channels are are omitted */
Could you make that clearer?
Post by e***@nokia.com
+typedef struct AVAudioTrackChannelPredefinedLayout {
+ int layout; /** ChannelConfiguration from ISO/IEC 23001-8 */
+ int nb_omitted_channels;
+ /** followed by: char omitted_channels[nb_omitted_channels]; - non-zero indicates the channel is omitted */
Is there a reasonable upper bound to nb_omitted_channels?
Yes, ISOBMFF supports only 64 channels for this mask. But perhaps even a
higher number is usable for some special case scenarios, not involving
ISOBMFF?
Post by Nicolas George
Also, I think the naming is inconsistent: if nb_omitted_channels = 3 and
omitted_channels[] = { 1, 0, 1 }, there are only two omitted channels, not
three.
I agree that it seems a bit inconsistent.. Would an integer array
enumerating the omitted channels be better? Of course this whole problem
goes away if the structure is fixed-size (ie. uint8_t
omitted_channels_bitmask[64 / 8] or the uint64_t you suggested).
Post by Nicolas George
Post by e***@nokia.com
enum AVPacketSideDataType {
AV_PKT_DATA_PALETTE,
@@ -1556,7 +1585,27 @@ enum AVPacketSideDataType {
* Configured the timed metadata parameters, such as the uri and
* meta data configuration. The key is of type AVTimedMetadata.
*/
- AV_PKT_DATA_TIMED_METADATA_INFO
+ AV_PKT_DATA_TIMED_METADATA_INFO,
+
+ /**
+ * Channel layout, describing the position of spakers for the
+ * channels of a track, following the structure
+ * AVAudioTrackChannelLayout.
+ */
+ AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT,
I think it would be a good idea if the name of the side data matches the
name of the structure as much as possible: Track -> _TRACK_. Ditto for the
others.
Would it make sense to have a single side data type and an integer field at
typedef struct AVComplexChannelLayout {
enum AVComplexChannelLayoutType type;
union {
uint64_t bitmask;
AVAudioTrackChannelLayout complete;
AVAudioTrackChannelPredefinedLayout predefined;
AVAudioTrackChannelLayoutObjectStructured object;
} layout;
};
It does seem nice. The actual structs/enums might look like this:

typedef struct AVAudioTrackChannelLayout {
int nb_positions;

/** Contains nb_positions entries*/
AVAudioTrackChannelPosition positions[1];
} AVAudioTrackChannelLayout;

typedef struct AVAudioTrackChannelPredefinedLayout {
int layout; /** ChannelConfiguration from ISO/IEC 23001-8 */
uint64_t omitted_channels;
} AVAudioTrackChannelPredefinedLayout;

typedef enum AVComplexAudioTrackChannelLayoutType {
AV_COMPLEX_CHANNEL_LAYOUT_PREDEFINED,
AV_COMPLEX_CHANNEL_LAYOUT_CUSTOM
} AVComplexAudioTrackChannelLayoutType;

typedef struct AVComplexAudioChannelLayout {
enum AVComplexAudioTrackChannelLayoutType type;
union {
AVAudioTrackChannelPredefinedLayout predefined;
AVAudioTrackChannelLayout complete;
};
} AVComplexAudioChannelLayout;

enum AVPacketSideDataType {
...

/**
* Channel layout, describing the position of speakers for the
* channels of a track, following the structure
* AVComplexAudioChannelLayout.
*/
AV_PKT_DATA_COMPLEX_AUDIO_CHANNEL_LAYOUT,

/**
* The channel layout is object structured with the number of
objects in an int (may accompany AV_PKT_DATA_COMPLEX_AUDIO_CHANNEL_LAYOUT)
*/
AV_PKT_DATA_COMPLEX_AUDIO_TRACK_CHANNEL_LAYOUT_OBJECT_STRUCTURED,
} AVPacketSideDataType;

I'm not super happy about the long names (though tempted to
s/AVAudio/AVComplexAudio/), but what is the alternative :).

Thanks for reviewing and feedback!
e***@nokia.com
2016-08-23 09:03:31 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

mov_codec_id is now able to set AVMEDIA_TYPE_DATA to the
st->codec->codec_type field when the input is of type
AVMEDIA_TYPE_DATA; previously it used AVMEDIA_TYPE_SUBTITLE as the
value to set in that case.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/mov.c | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/libavformat/mov.c b/libavformat/mov.c
index 9cd915d..f18c0a3 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -1789,12 +1789,20 @@ static int mov_codec_id(AVStream *st, uint32_t format)
id = ff_codec_get_id(ff_codec_bmp_tags, format);
if (id > 0)
st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
- else if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA ||
- (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE &&
- st->codecpar->codec_id == AV_CODEC_ID_NONE)) {
- id = ff_codec_get_id(ff_codec_movsubtitle_tags, format);
- if (id > 0)
- st->codecpar->codec_type = AVMEDIA_TYPE_SUBTITLE;
+ else {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA ||
+ (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE &&
+ st->codecpar->codec_id == AV_CODEC_ID_NONE)) {
+ id = ff_codec_get_id(ff_codec_movsubtitle_tags, format);
+ if (id > 0)
+ st->codecpar->codec_type = AVMEDIA_TYPE_SUBTITLE;
+ }
+ if (id <= 0 &&
+ st->codecpar->codec_type == AVMEDIA_TYPE_DATA) {
+ id = ff_codec_get_id(ff_codec_metadata_tags, format);
+ if (id > 0)
+ st->codecpar->codec_type = AVMEDIA_TYPE_DATA;
+ }
}
}
--
2.7.4
Carl Eugen Hoyos
2016-08-23 19:18:23 UTC
Permalink
Post by e***@nokia.com
+ if (id <= 0 &&
+ st->codecpar->codec_type == AVMEDIA_TYPE_DATA) {
+ id = ff_codec_get_id(ff_codec_metadata_tags, format);
+ if (id > 0)
+ st->codecpar->codec_type = AVMEDIA_TYPE_DATA;
Is it just me or is this a no-op?

Carl Eugen
Erkki Seppälä
2016-08-24 07:47:13 UTC
Permalink
Post by Carl Eugen Hoyos
Post by e***@nokia.com
+ if (id <= 0 &&
+ st->codecpar->codec_type == AVMEDIA_TYPE_DATA) {
+ id = ff_codec_get_id(ff_codec_metadata_tags, format);
+ if (id > 0)
+ st->codecpar->codec_type = AVMEDIA_TYPE_DATA;
Is it just me or is this a no-op?
You would be correct. Not only it is never executed, but even if it
were, it would do nothing :). (I imagine I
copy-paste-search-and-replaced from the one above it and the editing
missed this.)

Thanks for review!
e***@nokia.com
2016-08-23 09:03:22 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Also added avdata_alloc and avdata_free for dealing with it. AVData
can contain arbitrary binary data and comes with a format-field so far
unused.

The purpose is that AVMEDIA_TYPE_DATA -kind codecs can store frames in
this format.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 20 ++++++++++++++++++++
libavcodec/utils.c | 19 +++++++++++++++++++
2 files changed, 39 insertions(+)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 6ac6646..fb8f363 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -3934,6 +3934,14 @@ typedef struct AVSubtitle {
int64_t pts; ///< Same as packet pts, in AV_TIME_BASE
} AVSubtitle;

+typedef struct AVData {
+ uint16_t format; /** 0 = timed metadata */
+ int64_t pts;
+ int64_t dts;
+
+ AVBufferRef *data;
+} AVData;
+
/**
* This struct describes the properties of an encoded stream.
*
@@ -4317,6 +4325,18 @@ int avcodec_close(AVCodecContext *avctx);
void avsubtitle_free(AVSubtitle *sub);

/**
+ * Allocate an empty data (typically use with timed metadata)
+ */
+AVData *avdata_alloc(void);
+
+/**
+ * Free all allocated data in the given data struct.
+ *
+ * @param sub AVData to free.
+ */
+void avdata_free(AVData *sub);
+
+/**
* @}
*/

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index 138125a..dabe97e 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -2955,6 +2955,25 @@ int attribute_align_arg avcodec_receive_packet(AVCodecContext *avctx, AVPacket *
return 0;
}

+AVData *avdata_alloc(void)
+{
+ AVData *data = av_mallocz(sizeof(*data));
+
+ if (!data)
+ return NULL;
+
+ return data;
+}
+
+void avdata_free(AVData *data)
+{
+ av_buffer_unref(&data->data);
+
+ memset(data, 0, sizeof(AVData));
+
+ av_free(data);
+}
+
av_cold int avcodec_close(AVCodecContext *avctx)
{
int i;
--
2.7.4
Nicolas George
2016-08-23 15:36:02 UTC
Permalink
Post by e***@nokia.com
Also added avdata_alloc and avdata_free for dealing with it. AVData
can contain arbitrary binary data and comes with a format-field so far
unused.
The purpose is that AVMEDIA_TYPE_DATA -kind codecs can store frames in
this format.
Please consider using AVFrame itself instead. It has all the necessary
fields. It has many more fields, but is it really an issue?

The strongest case for using AVFrame is uniformity. Thanks to wm4, we now
have the same API for audio and video decoding. With that, code can be
written to manipulate frames in a generic manner. Timed metadata would be
handled the same way without extra code.

Integration in libavfilter would be easier too.
Post by e***@nokia.com
---
libavcodec/avcodec.h | 20 ++++++++++++++++++++
libavcodec/utils.c | 19 +++++++++++++++++++
2 files changed, 39 insertions(+)
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 6ac6646..fb8f363 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -3934,6 +3934,14 @@ typedef struct AVSubtitle {
int64_t pts; ///< Same as packet pts, in AV_TIME_BASE
} AVSubtitle;
+typedef struct AVData {
+ uint16_t format;
Using a small type seems like a micro-optimization. Not even a real one:
because of alignment requirements, this field will actually use 8 octets
(remember to put smaller fields after bigger ones), and type promotions will
cost time.

I suggest to use a normal int field. That has the possibility of leaving
gaps in the values or using 4ccs.
Post by e***@nokia.com
/** 0 = timed metadata */
An enum would be better IMHO.
Post by e***@nokia.com
+ int64_t pts;
+ int64_t dts;
+
+ AVBufferRef *data;
Since it uses refcounted buffers, the API to create a new reference to the
same buffer seems missing.
Post by e***@nokia.com
+} AVData;
+
/**
* This struct describes the properties of an encoded stream.
*
@@ -4317,6 +4325,18 @@ int avcodec_close(AVCodecContext *avctx);
void avsubtitle_free(AVSubtitle *sub);
/**
+ * Allocate an empty data (typically use with timed metadata)
+ */
+AVData *avdata_alloc(void);
I say it again: I would prefer if these APIs returned an error code in case
of failure. Even if the error code can only be ENOMEM, it is better than
hardcoding it at every call site.
Post by e***@nokia.com
+
+/**
+ * Free all allocated data in the given data struct.
+ *
Well said, Captain Obvious ;-)
Post by e***@nokia.com
+ */
+void avdata_free(AVData *sub);
+
+/**
*/
diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index 138125a..dabe97e 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -2955,6 +2955,25 @@ int attribute_align_arg avcodec_receive_packet(AVCodecContext *avctx, AVPacket *
return 0;
}
+AVData *avdata_alloc(void)
+{
+ AVData *data = av_mallocz(sizeof(*data));
+
+ if (!data)
+ return NULL;
Seems redundant.
Post by e***@nokia.com
+
+ return data;
+}
+
+void avdata_free(AVData *data)
+{
+ av_buffer_unref(&data->data);
+
+ memset(data, 0, sizeof(AVData));
Seems useless.
Post by e***@nokia.com
+
+ av_free(data);
+}
+
av_cold int avcodec_close(AVCodecContext *avctx)
{
int i;
Regards,
--
Nicolas George
Paul B Mahol
2016-08-23 16:07:44 UTC
Permalink
Post by Nicolas George
Post by e***@nokia.com
Also added avdata_alloc and avdata_free for dealing with it. AVData
can contain arbitrary binary data and comes with a format-field so far
unused.
The purpose is that AVMEDIA_TYPE_DATA -kind codecs can store frames in
this format.
Please consider using AVFrame itself instead. It has all the necessary
fields. It has many more fields, but is it really an issue?
Yes, please use AVFrame.
Post by Nicolas George
The strongest case for using AVFrame is uniformity. Thanks to wm4, we now
have the same API for audio and video decoding. With that, code can be
written to manipulate frames in a generic manner. Timed metadata would be
handled the same way without extra code.
Integration in libavfilter would be easier too.
Michael Niedermayer
2016-08-23 16:13:25 UTC
Permalink
Post by Paul B Mahol
Post by Nicolas George
Post by e***@nokia.com
Also added avdata_alloc and avdata_free for dealing with it. AVData
can contain arbitrary binary data and comes with a format-field so far
unused.
The purpose is that AVMEDIA_TYPE_DATA -kind codecs can store frames in
this format.
Please consider using AVFrame itself instead. It has all the necessary
fields. It has many more fields, but is it really an issue?
Yes, please use AVFrame.
+1

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have never wished to cater to the crowd; for what I know they do not
approve, and what they approve I do not know. -- Epicurus
Erkki Seppälä
2016-08-24 08:30:36 UTC
Permalink
AVFrame certainly sounds a good solution. For AVData I followed the lead
from AVSubtitle as they are quite similar concepts. And AVFrame did
really look quite a big struct ;).

I'll rework the code to replace AVData with AVFrame.

Thanks for review!
e***@nokia.com
2016-08-23 09:03:24 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

This basically passes the data forward and is used for referring timed
meta data tracks by a codec.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 2 +-
libavcodec/metacodec.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 95 insertions(+), 1 deletion(-)
create mode 100644 libavcodec/metacodec.c

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index fb8f363..756eda5 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -639,7 +639,7 @@ enum AVCodecID {
AV_CODEC_ID_DVD_NAV,
AV_CODEC_ID_TIMED_ID3,
AV_CODEC_ID_BIN_DATA,
-
+ AV_CODEC_ID_META,

AV_CODEC_ID_PROBE = 0x19000, ///< codec_id is not known (like AV_CODEC_ID_NONE) but lavf should attempt to identify it

diff --git a/libavcodec/metacodec.c b/libavcodec/metacodec.c
new file mode 100644
index 0000000..f26a0d0
--- /dev/null
+++ b/libavcodec/metacodec.c
@@ -0,0 +1,94 @@
+/*
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/intreadwrite.h"
+#include "internal.h"
+#include "libavutil/opt.h"
+#include "libavformat/avio.h"
+
+static av_cold int meta_init(AVCodecContext *avctx)
+{
+ return 0;
+}
+
+static av_cold int meta_encode(AVCodecContext *avctx, AVPacket *avpkt, const AVFrame *frame,
+ int *got_packet_ptr)
+{
+ int ret;
+
+ *got_packet_ptr = 0;
+
+ if ((ret = ff_alloc_packet2(avctx, avpkt, frame->nb_samples, 0)) < 0)
+ return ret;
+ memcpy(avpkt->data, frame->data[0], frame->nb_samples);
+ avpkt->size = frame->nb_samples;
+ avpkt->pts = frame->pts;
+ avpkt->dts = frame->pts;
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+static int meta_decode(AVCodecContext *avctx, void *data, int *got_packet_ptr, AVPacket *avpkt)
+{
+ AVData *metadata = data;
+
+ *got_packet_ptr = 0;
+
+ av_buffer_unref(&metadata->data);
+ metadata->data = av_buffer_alloc(avpkt->size);
+ if (!metadata->data)
+ return AVERROR(ENOMEM);
+
+ metadata->dts = avpkt->dts;
+ metadata->pts = avpkt->pts;
+ memcpy(((char*) metadata->data->data), avpkt->data, avpkt->size);
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+static const AVClass metacodec_class = {
+ .class_name = "Meta data codec",
+ .item_name = av_default_item_name,
+ .version = LIBAVUTIL_VERSION_INT,
+};
+
+AVCodec ff_meta_encoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Meta Data Encoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .init = meta_init,
+ .encode2 = meta_encode,
+ .priv_class = &metacodec_class,
+ .priv_data_size = 0,
+};
+
+AVCodec ff_meta_decoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Meta Data Decoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .init = meta_init,
+ .decode = meta_decode,
+ .priv_class = &metacodec_class,
+ .priv_data_size = 0,
+};
--
2.7.4
e***@nokia.com
2016-08-23 09:03:26 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

This includes creating an AVCodecTag table ff_codec_metadata_tags as
there are for video, audio and subtitles. The tag table is used for
mov-compatiblity.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.c | 5 +++++
libavformat/isom.h | 1 +
libavformat/movenc.c | 16 ++++++++++++++++
3 files changed, 22 insertions(+)

diff --git a/libavformat/isom.c b/libavformat/isom.c
index cb457dd..1a90d00 100644
--- a/libavformat/isom.c
+++ b/libavformat/isom.c
@@ -355,6 +355,11 @@ const AVCodecTag ff_codec_movsubtitle_tags[] = {
{ AV_CODEC_ID_NONE, 0 },
};

+const AVCodecTag ff_codec_metadata_tags[] = {
+ { AV_CODEC_ID_META, MKTAG('m', 'e', 't', 'a') },
+ { AV_CODEC_ID_NONE, 0 },
+};
+
/* map numeric codes from mdhd atom to ISO 639 */
/* cf. QTFileFormat.pdf p253, qtff.pdf p205 */
/* http://developer.apple.com/documentation/mac/Text/Text-368.html */
diff --git a/libavformat/isom.h b/libavformat/isom.h
index df6c15a..49c8996 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -33,6 +33,7 @@ extern const AVCodecTag ff_mp4_obj_type[];
extern const AVCodecTag ff_codec_movvideo_tags[];
extern const AVCodecTag ff_codec_movaudio_tags[];
extern const AVCodecTag ff_codec_movsubtitle_tags[];
+extern const AVCodecTag ff_codec_metadata_tags[];

int ff_mov_iso639_to_lang(const char lang[4], int mp4);
int ff_mov_lang_to_iso639(unsigned code, char to[4]);
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 34bc235..c63fdc4 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1470,6 +1470,8 @@ static int mov_get_codec_tag(AVFormatContext *s, MOVTrack *track)
}
} else if (track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)
tag = ff_codec_get_tag(ff_codec_movsubtitle_tags, track->par->codec_id);
+ else if (track->par->codec_type == AVMEDIA_TYPE_DATA)
+ tag = ff_codec_get_tag(ff_codec_metadata_tags, track->par->codec_id);
}

return tag;
@@ -2222,6 +2224,9 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra
} else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) {
hdlr_type = "soun";
descr = "SoundHandler";
+ } else if (track->par->codec_type == AVMEDIA_TYPE_DATA) {
+ hdlr_type = "meta";
+ descr = "DataHandler";
} else if (track->par->codec_type == AVMEDIA_TYPE_SUBTITLE) {
if (is_clcp_track(track)) {
hdlr_type = "clcp";
@@ -4787,6 +4792,8 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt)
goto end;
avio_write(pb, pkt->data, size);
#endif
+ } else if (par->codec_id == AV_CODEC_ID_META) {
+ avio_write(pb, pkt->data, size);
} else {
if (mov->encryption_scheme == MOV_ENC_CENC_AES_CTR) {
if (par->codec_id == AV_CODEC_ID_H264 && par->extradata_size > 4) {
@@ -5301,6 +5308,7 @@ static void enable_tracks(AVFormatContext *s)
case AVMEDIA_TYPE_VIDEO:
case AVMEDIA_TYPE_AUDIO:
case AVMEDIA_TYPE_SUBTITLE:
+ case AVMEDIA_TYPE_DATA:
if (enabled[i] > 1)
mov->per_stream_grouping = 1;
if (!enabled[i] && first[i] >= 0)
@@ -6103,6 +6111,7 @@ AVOutputFormat ff_mov_muxer = {
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = CONFIG_LIBX264_ENCODER ?
AV_CODEC_ID_H264 : AV_CODEC_ID_MPEG4,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6122,6 +6131,7 @@ AVOutputFormat ff_tgp_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AMR_NB,
.video_codec = AV_CODEC_ID_H263,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6141,6 +6151,7 @@ AVOutputFormat ff_mp4_muxer = {
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = CONFIG_LIBX264_ENCODER ?
AV_CODEC_ID_H264 : AV_CODEC_ID_MPEG4,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6159,6 +6170,7 @@ AVOutputFormat ff_psp_muxer = {
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = CONFIG_LIBX264_ENCODER ?
AV_CODEC_ID_H264 : AV_CODEC_ID_MPEG4,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6176,6 +6188,7 @@ AVOutputFormat ff_tg2_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AMR_NB,
.video_codec = AV_CODEC_ID_H263,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6194,6 +6207,7 @@ AVOutputFormat ff_ipod_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = AV_CODEC_ID_H264,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6212,6 +6226,7 @@ AVOutputFormat ff_ismv_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = AV_CODEC_ID_H264,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6230,6 +6245,7 @@ AVOutputFormat ff_f4v_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = AV_CODEC_ID_H264,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
--
2.7.4
e***@nokia.com
2016-08-23 09:03:33 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

The data is read into side packet AV_PKT_DATA_TIMED_METADATA_INFO of format AVTimedMetadata.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.h | 1 +
libavformat/mov.c | 163 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 164 insertions(+)

diff --git a/libavformat/isom.h b/libavformat/isom.h
index 9ad898a..b4220c0 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -27,6 +27,7 @@
#include "avio.h"
#include "internal.h"
#include "dv.h"
+#include "movmeta.h"

/* isom.c */
extern const AVCodecTag ff_mp4_obj_type[];
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 275b532..fb233b6 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -2041,6 +2041,145 @@ static int mov_rewrite_dvd_sub_extradata(AVStream *st)
return 0;
}

+static int mov_parse_urim_uri_data(AVIOContext *pb, AVStream *st, MOVMeta *meta)
+{
+ int64_t size = avio_rb32(pb);
+ uint32_t uri = avio_rb32(pb); /* "uri " */
+ avio_r8(pb); /* version */
+ avio_rb24(pb); /* flags */
+
+ if (size < 12 || size >= 1024)
+ return AVERROR(ENOMEM);
+
+ if (uri == AV_RB32("uri ")) {
+ int remaining = size - 12;
+ int64_t pos = avio_tell(pb);
+ uint32_t string_len = 0;
+
+ while (string_len < remaining && avio_r8(pb) != 0) {
+ ++string_len;
+ }
+ avio_seek(pb, pos, SEEK_SET);
+
+ if (string_len >= remaining)
+ return AVERROR_INVALIDDATA;
+
+ meta->tag = MKTAG('u', 'r', 'i', 'm');
+ meta->data = av_malloc(string_len + 1);
+ if (!meta->data)
+ return AVERROR(ENOMEM);
+
+ avio_read(pb, meta->data, string_len);
+ ((char*) meta->data)[string_len] = 0;
+ meta->length = string_len;
+ remaining -= string_len;
+ avio_r8(pb); /* read the null terminator */
+ remaining--;
+
+ avio_skip(pb, remaining);
+ } else {
+ avio_skip(pb, size - 12);
+ }
+
+ return 0;
+}
+
+static int mov_parse_urim_conf_data(AVIOContext *pb, AVStream *st, int64_t remaining, MOVMeta *meta)
+{
+ int64_t size;
+ uint32_t tag;
+ int64_t data_size;
+
+ if (remaining < 12)
+ return 0;
+
+ size = avio_rb32(pb);
+ tag = avio_rl32(pb);
+ avio_r8(pb); /* version */
+ avio_rb24(pb); /* flags */
+ remaining -= 12;
+ data_size = size - 12;
+
+ if (data_size < 0 || data_size > remaining) {
+ avio_skip(pb, remaining);
+ return 0;
+ }
+
+ meta->conf.data = av_malloc(data_size);
+ if (!meta->conf.data)
+ return AVERROR(ENOMEM);
+
+ meta->conf.tag = tag;
+ meta->conf.length = data_size;
+ avio_read(pb, meta->conf.data, data_size);
+ remaining -= data_size;
+ avio_skip(pb, remaining);
+
+ return 0;
+}
+
+static int mov_parse_urim_data(AVIOContext *pb, AVStream *st, int64_t remaining)
+{
+ int64_t pos = avio_tell(pb);
+ int ret;
+ MOVMeta meta;
+
+ memset(&meta, 0, sizeof(meta));
+
+ ret = mov_parse_urim_uri_data(pb, st, &meta);
+ if (ret)
+ return ret;
+
+ remaining -= avio_tell(pb) - pos;
+ ret = mov_parse_urim_conf_data(pb, st, remaining, &meta);
+
+ // all data has been collected; now build the actual side channel
+ // object from the collected data
+ if (ret == 0) {
+ int tmdLength = sizeof(AVTimedMetadata) + meta.length + meta.conf.length;
+ AVTimedMetadata *tmd = av_malloc(tmdLength);
+ if (!tmd) {
+ ret = -1;
+ } else {
+ AVPacketSideData *sd;
+ ret = av_reallocp_array(&st->side_data,
+ st->nb_side_data + 1, sizeof(*sd));
+ if (ret >= 0) {
+ char* data;
+
+ sd = st->side_data + st->nb_side_data;
+ st->nb_side_data++;
+
+ sd->type = AV_PKT_DATA_TIMED_METADATA_INFO;
+ sd->size = sizeof(*tmd) + meta.length + meta.conf.length;
+ sd->data = (uint8_t*) tmd;
+
+ tmd->meta_tag[0] = (meta.tag >> 0) & 0xff;
+ tmd->meta_tag[1] = (meta.tag >> 8) & 0xff;
+ tmd->meta_tag[2] = (meta.tag >> 16) & 0xff;
+ tmd->meta_tag[3] = (meta.tag >> 24) & 0xff;
+ tmd->meta_length = meta.length;
+ tmd->conf_tag[0] = (meta.conf.tag >> 0) & 0xff;
+ tmd->conf_tag[1] = (meta.conf.tag >> 8) & 0xff;
+ tmd->conf_tag[2] = (meta.conf.tag >> 16) & 0xff;
+ tmd->conf_tag[3] = (meta.conf.tag >> 24) & 0xff;
+ tmd->conf_length = meta.conf.length;
+ data = (char*) (tmd + 1);
+
+ memcpy(data, meta.data, meta.length);
+ data += meta.length;
+ memcpy(data, meta.conf.data, meta.conf.length);
+ } else {
+ av_freep(&tmd);
+ }
+ }
+ }
+ av_freep(&meta.data);
+ av_freep(&meta.conf.data);
+
+ return ret;
+}
+
static int mov_parse_stsd_data(MOVContext *c, AVIOContext *pb,
AVStream *st, MOVStreamContext *sc,
int64_t size)
@@ -2100,6 +2239,8 @@ FF_ENABLE_DEPRECATION_WARNINGS
}
}
}
+ } else if (st->codecpar->codec_tag == MKTAG('u', 'r', 'i', 'm')) {
+ ret = mov_parse_urim_data(pb, st, size);
} else {
/* other codec type, just skip (rtp, mp4s ...) */
avio_skip(pb, size);
@@ -3207,6 +3348,28 @@ static int mov_read_tref(MOVContext *c, AVIOContext *pb, MOVAtom atom)
sc->nb_tref_ids++;
remaining -= 4;
}
+
+ if (sc->nb_tref_ids) {
+ int i;
+ AVTrackReferences *trefs_side;
+ int *trefs_tracks;
+
+ trefs_side = (void*) av_stream_new_side_data(st,
+ AV_PKT_DATA_TRACK_REFERENCES,
+ sizeof(AVTrackReferences) + sizeof(int) * sc->nb_tref_ids);
+ if (!trefs_side)
+ return AVERROR(ENOMEM);
+ trefs_side->nb_tracks = sc->nb_tref_ids;
+ trefs_side->tag[0] = (sc->tref_tag >> 0) & 0xff;
+ trefs_side->tag[1] = (sc->tref_tag >> 8) & 0xff;
+ trefs_side->tag[2] = (sc->tref_tag >> 16) & 0xff;
+ trefs_side->tag[3] = (sc->tref_tag >> 24) & 0xff;
+ trefs_tracks = (void*) (trefs_side + 1);
+ for (i = 0; i < sc->nb_tref_ids; ++i) {
+ trefs_tracks[i] = sc->tref_ids[i];
+ }
+ }
+
avio_seek(pb, remaining, SEEK_CUR);
return ret;
}
--
2.7.4
e***@nokia.com
2016-08-23 09:03:23 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/movenc.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 8c4252d..34bc235 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -4203,6 +4203,8 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s)

avio_wb32(pb, minor);

+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a compatible brand */
if (mov->mode == MODE_MOV)
ffio_wfourcc(pb, "qt ");
else if (mov->mode == MODE_ISM) {
--
2.7.4
Yusuke Nakamura
2016-08-23 15:35:51 UTC
Permalink
Post by e***@nokia.com
---
libavformat/movenc.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 8c4252d..34bc235 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -4203,6 +4203,8 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s)
avio_wb32(pb, minor);
+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a compatible brand */
WTF. libavformat has not listed all compatible brands? This is a wrong
approach if what David Singer (Apple) says is correct. The major_brand
always be written into compatible_brands in ISOBMFF.
Post by e***@nokia.com
if (mov->mode == MODE_MOV)
ffio_wfourcc(pb, "qt ");
else if (mov->mode == MODE_ISM) {
--
2.7.4
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Erkki Seppälä
2016-08-31 10:02:10 UTC
Permalink
Post by Yusuke Nakamura
Post by e***@nokia.com
+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a
compatible brand */
WTF. libavformat has not listed all compatible brands? This is a wrong
approach if what David Singer (Apple) says is correct. The major_brand
always be written into compatible_brands in ISOBMFF.
Hmm, so you're saying it should be written unconditionally?

It seems that in many cases it does happen so that the 4cc (ie. mp41)
ends up being written into the compatible brands as well. But this does
not happen if the major_brand is explicitly passed as an option.

I have a new patch with a better commit message (but same content).
Carl Eugen Hoyos
2016-08-23 19:00:49 UTC
Permalink
Hi!
Post by e***@nokia.com
+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a compatible brand */
How can I reproduce the issue this is trying to fix?

Carl Eugen
Erkki Seppälä
2016-08-24 14:53:27 UTC
Permalink
Hello,
Post by Carl Eugen Hoyos
Post by e***@nokia.com
+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a compatible brand */
How can I reproduce the issue this is trying to fix?
The issue we were fixing was that in the presence of custom major brand
(the option "brand" is set) the custom major brand did not end up in the
compatible brands. (In retrospect, this would have been a great commit
message..)

Prompted by your comment, we reviewed the specification, and it does not
seem like that the standard requires this functionality - but it doesn't
outright prohibit it either.

An alternative for our use case would be adding the option
"compatible_brands" for setting custom compatible brands from the client
code. It would probably either replace all custom brands with the ones
provided, or it would need to collect the custom brands in a list in
order to remove duplicates (though I imagine allowing duplicates would
only be a esthetic flaw).

Thanks for the review and input!
Carl Eugen Hoyos
2016-08-24 18:04:40 UTC
Permalink
Hi!
Post by Carl Eugen Hoyos
Post by e***@nokia.com
+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a
compatible brand */
How can I reproduce the issue this is trying to fix?
The issue we were fixing was that in the presence of custom major brand (the
option "brand" is set) the custom major brand did not end up in the
compatible brands.
Thanks for explaining!
(I couldn't find the option yesterday.)
(In retrospect, this would have been a great commit
message..)
Yes;-)
An alternative for our use case would be adding the option
"compatible_brands" for setting custom compatible brands from the client
code.
The advantage is that I believe users have asked for this.

No more comments from me, Carl Eugen
e***@nokia.com
2016-08-23 09:03:29 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

This involves adding a new tag to the ff_mp4_obj_type table as well as
modifying mp4_get_codec_tag to return 'meta' for AV_CODEC_ID_META.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.c | 1 +
libavformat/movenc.c | 1 +
2 files changed, 2 insertions(+)

diff --git a/libavformat/isom.c b/libavformat/isom.c
index 473700f..9fb96ef 100644
--- a/libavformat/isom.c
+++ b/libavformat/isom.c
@@ -66,6 +66,7 @@ const AVCodecTag ff_mp4_obj_type[] = {
{ AV_CODEC_ID_QCELP , 0xE1 },
{ AV_CODEC_ID_MPEG4SYSTEMS, 0x01 },
{ AV_CODEC_ID_MPEG4SYSTEMS, 0x02 },
+ { AV_CODEC_ID_META , 0x03 },
{ AV_CODEC_ID_NONE , 0 },
};

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 141f1a5..ff4bf85 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1175,6 +1175,7 @@ static int mp4_get_codec_tag(AVFormatContext *s, MOVTrack *track)
else if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) tag = MKTAG('m','p','4','v');
else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) tag = MKTAG('m','p','4','a');
else if (track->par->codec_id == AV_CODEC_ID_DVD_SUBTITLE) tag = MKTAG('m','p','4','s');
+ else if (track->par->codec_id == AV_CODEC_ID_META) tag = MKTAG('m','e','t','a');

return tag;
}
--
2.7.4
e***@nokia.com
2016-08-23 09:03:38 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
doc/examples/Makefile | 2 +
doc/examples/muxing_with_metadata.c | 882 ++++++++++++++++++++++++++++++++++++
2 files changed, 884 insertions(+)
create mode 100644 doc/examples/muxing_with_metadata.c

diff --git a/doc/examples/Makefile b/doc/examples/Makefile
index c10033e..d1f4e1f 100644
--- a/doc/examples/Makefile
+++ b/doc/examples/Makefile
@@ -22,6 +22,7 @@ EXAMPLES= avio_dir_cmd \
http_multiclient \
metadata \
muxing \
+ muxing_with_metadata \
remuxing \
resampling_audio \
scaling_video \
@@ -35,6 +36,7 @@ avcodec: LDLIBS += -lm
decoding_encoding: LDLIBS += -lm
muxing: LDLIBS += -lm
resampling_audio: LDLIBS += -lm
+muxing_with_metadata: LDLIBS += -lm

.phony: all clean-test clean

diff --git a/doc/examples/muxing_with_metadata.c b/doc/examples/muxing_with_metadata.c
new file mode 100644
index 0000000..507c6a1
--- /dev/null
+++ b/doc/examples/muxing_with_metadata.c
@@ -0,0 +1,882 @@
+/*
+ * Copyright (c) 2003 Fabrice Bellard
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+/**
+ * @file
+ * libavformat API example.
+ *
+ * Output a media file in any supported libavformat format. The default
+ * codecs are used.
+ * @example muxing.c
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <math.h>
+
+#include <libavutil/avassert.h>
+#include <libavutil/channel_layout.h>
+#include <libavutil/opt.h>
+#include <libavutil/mathematics.h>
+#include <libavutil/timestamp.h>
+#include <libavformat/avformat.h>
+#include <libswscale/swscale.h>
+#include <libswresample/swresample.h>
+
+#define STREAM_DURATION 10.0
+#define STREAM_FRAME_RATE 25 /* 25 images/s */
+#define STREAM_PIX_FMT AV_PIX_FMT_YUV420P /* default pix_fmt */
+
+#define SCALE_FLAGS SWS_BICUBIC
+
+// a wrapper around a single output AVStream
+typedef struct OutputStream {
+ AVStream *st;
+ AVCodecContext *enc;
+
+ /* pts of the next frame that will be generated */
+ int64_t next_pts;
+ int samples_count;
+
+ AVFrame *frame;
+ AVFrame *tmp_frame;
+
+ float t, tincr, tincr2;
+
+ struct SwsContext *sws_ctx;
+ struct SwrContext *swr_ctx;
+} OutputStream;
+
+typedef struct StreamState {
+ int (*writer)(AVFormatContext *oc, OutputStream *ost);
+ int *flag;
+ OutputStream *stream;
+} StreamState;
+
+static void log_packet(const AVFormatContext *fmt_ctx, const AVPacket *pkt)
+{
+ AVRational *time_base = &fmt_ctx->streams[pkt->stream_index]->time_base;
+
+ printf("pts:%s pts_time:%s dts:%s dts_time:%s duration:%s duration_time:%s stream_index:%d\n",
+ av_ts2str(pkt->pts), av_ts2timestr(pkt->pts, time_base),
+ av_ts2str(pkt->dts), av_ts2timestr(pkt->dts, time_base),
+ av_ts2str(pkt->duration), av_ts2timestr(pkt->duration, time_base),
+ pkt->stream_index);
+}
+
+static int write_frame(AVFormatContext *fmt_ctx, const AVRational *time_base, AVStream *st, AVPacket *pkt)
+{
+ /* rescale output packet timestamp values from codec to stream timebase */
+ av_packet_rescale_ts(pkt, *time_base, st->time_base);
+ pkt->stream_index = st->index;
+
+ /* Write the compressed frame to the media file. */
+ log_packet(fmt_ctx, pkt);
+ return av_interleaved_write_frame(fmt_ctx, pkt);
+}
+
+/* Add an output stream. */
+static void add_stream(OutputStream *ost, AVFormatContext *oc,
+ AVCodec **codec,
+ enum AVCodecID codec_id)
+{
+ AVCodecContext *c;
+ int i;
+
+ /* find the encoder */
+ *codec = avcodec_find_encoder(codec_id);
+ if (!(*codec)) {
+ fprintf(stderr, "Could not find encoder for '%s'\n",
+ avcodec_get_name(codec_id));
+ exit(1);
+ }
+
+ ost->st = avformat_new_stream(oc, NULL);
+ if (!ost->st) {
+ fprintf(stderr, "Could not allocate stream\n");
+ exit(1);
+ }
+ ost->st->id = oc->nb_streams-1;
+ c = avcodec_alloc_context3(*codec);
+ if (!c) {
+ fprintf(stderr, "Could not alloc an encoding context\n");
+ exit(1);
+ }
+ ost->enc = c;
+
+ switch ((*codec)->type) {
+ case AVMEDIA_TYPE_AUDIO:
+ c->sample_fmt = (*codec)->sample_fmts ?
+ (*codec)->sample_fmts[0] : AV_SAMPLE_FMT_FLTP;
+ c->bit_rate = 64000;
+ c->sample_rate = 44100;
+ if ((*codec)->supported_samplerates) {
+ c->sample_rate = (*codec)->supported_samplerates[0];
+ for (i = 0; (*codec)->supported_samplerates[i]; i++) {
+ if ((*codec)->supported_samplerates[i] == 44100)
+ c->sample_rate = 44100;
+ }
+ }
+ c->channels = av_get_channel_layout_nb_channels(c->channel_layout);
+ c->channel_layout = AV_CH_LAYOUT_STEREO;
+ if ((*codec)->channel_layouts) {
+ c->channel_layout = (*codec)->channel_layouts[0];
+ for (i = 0; (*codec)->channel_layouts[i]; i++) {
+ if ((*codec)->channel_layouts[i] == AV_CH_LAYOUT_STEREO)
+ c->channel_layout = AV_CH_LAYOUT_STEREO;
+ }
+ }
+ c->channels = av_get_channel_layout_nb_channels(c->channel_layout);
+ ost->st->time_base = (AVRational){ 1, c->sample_rate };
+ break;
+
+ case AVMEDIA_TYPE_VIDEO:
+ c->codec_id = codec_id;
+
+ c->bit_rate = 400000;
+ /* Resolution must be a multiple of two. */
+ c->width = 352;
+ c->height = 288;
+ /* timebase: This is the fundamental unit of time (in seconds) in terms
+ * of which frame timestamps are represented. For fixed-fps content,
+ * timebase should be 1/framerate and timestamp increments should be
+ * identical to 1. */
+ ost->st->time_base = (AVRational){ 1, STREAM_FRAME_RATE };
+ c->time_base = ost->st->time_base;
+
+ c->gop_size = 12; /* emit one intra frame every twelve frames at most */
+ c->pix_fmt = STREAM_PIX_FMT;
+ if (c->codec_id == AV_CODEC_ID_MPEG2VIDEO) {
+ /* just for testing, we also add B frames */
+ c->max_b_frames = 2;
+ }
+ if (c->codec_id == AV_CODEC_ID_MPEG1VIDEO) {
+ /* Needed to avoid using macroblocks in which some coeffs overflow.
+ * This does not happen with normal video, it just happens here as
+ * the motion of the chroma plane does not match the luma plane. */
+ c->mb_decision = 2;
+ }
+ break;
+
+ case AVMEDIA_TYPE_DATA:
+ c->codec_id = codec_id;
+
+ c->bit_rate = 10;
+ /* timebase: This is the fundamental unit of time (in seconds) in terms
+ * of which frame timestamps are represented. For fixed-fps content,
+ * timebase should be 1/framerate and timestamp increments should be
+ * identical to 1. */
+ ost->st->time_base = (AVRational){ 1, STREAM_FRAME_RATE };
+ c->time_base = ost->st->time_base;
+ break;
+
+ default:
+ break;
+ }
+
+ /* Some formats want stream headers to be separate. */
+ if (oc->oformat->flags & AVFMT_GLOBALHEADER)
+ c->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
+}
+
+/**************************************************************/
+/* audio output */
+
+static AVFrame *alloc_audio_frame(enum AVSampleFormat sample_fmt,
+ uint64_t channel_layout,
+ int sample_rate, int nb_samples)
+{
+ AVFrame *frame = av_frame_alloc();
+ int ret;
+
+ if (!frame) {
+ fprintf(stderr, "Error allocating an audio frame\n");
+ exit(1);
+ }
+
+ frame->format = sample_fmt;
+ frame->channel_layout = channel_layout;
+ frame->sample_rate = sample_rate;
+ frame->nb_samples = nb_samples;
+
+ if (nb_samples) {
+ ret = av_frame_get_buffer(frame, 0);
+ if (ret < 0) {
+ fprintf(stderr, "Error allocating an audio buffer\n");
+ exit(1);
+ }
+ }
+
+ return frame;
+}
+
+static void open_audio(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
+{
+ AVCodecContext *c;
+ int nb_samples;
+ int ret;
+ AVDictionary *opt = NULL;
+
+ c = ost->enc;
+
+ /* open it */
+ av_dict_copy(&opt, opt_arg, 0);
+ ret = avcodec_open2(c, codec, &opt);
+ av_dict_free(&opt);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open audio codec: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ /* init signal generator */
+ ost->t = 0;
+ ost->tincr = 2 * M_PI * 110.0 / c->sample_rate;
+ /* increment frequency by 110 Hz per second */
+ ost->tincr2 = 2 * M_PI * 110.0 / c->sample_rate / c->sample_rate;
+
+ if (c->codec->capabilities & AV_CODEC_CAP_VARIABLE_FRAME_SIZE)
+ nb_samples = 10000;
+ else
+ nb_samples = c->frame_size;
+
+ ost->frame = alloc_audio_frame(c->sample_fmt, c->channel_layout,
+ c->sample_rate, nb_samples);
+ ost->tmp_frame = alloc_audio_frame(AV_SAMPLE_FMT_S16, c->channel_layout,
+ c->sample_rate, nb_samples);
+
+ /* copy the stream parameters to the muxer */
+ ret = avcodec_parameters_from_context(ost->st->codecpar, c);
+ if (ret < 0) {
+ fprintf(stderr, "Could not copy the stream parameters\n");
+ exit(1);
+ }
+
+ /* create resampler context */
+ ost->swr_ctx = swr_alloc();
+ if (!ost->swr_ctx) {
+ fprintf(stderr, "Could not allocate resampler context\n");
+ exit(1);
+ }
+
+ /* set options */
+ av_opt_set_int (ost->swr_ctx, "in_channel_count", c->channels, 0);
+ av_opt_set_int (ost->swr_ctx, "in_sample_rate", c->sample_rate, 0);
+ av_opt_set_sample_fmt(ost->swr_ctx, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
+ av_opt_set_int (ost->swr_ctx, "out_channel_count", c->channels, 0);
+ av_opt_set_int (ost->swr_ctx, "out_sample_rate", c->sample_rate, 0);
+ av_opt_set_sample_fmt(ost->swr_ctx, "out_sample_fmt", c->sample_fmt, 0);
+
+ /* initialize the resampling context */
+ if ((ret = swr_init(ost->swr_ctx)) < 0) {
+ fprintf(stderr, "Failed to initialize the resampling context\n");
+ exit(1);
+ }
+}
+
+static AVFrame *alloc_meta_frame(int nb_samples)
+{
+ AVFrame *frame = av_frame_alloc();
+
+ if (!frame) {
+ fprintf(stderr, "Error allocating a meta frame\n");
+ exit(1);
+ }
+
+ frame->nb_samples = nb_samples;
+
+ if (nb_samples) {
+ frame->buf[0] = av_buffer_alloc(nb_samples);
+
+ av_assert0(frame->buf[0]);
+ av_assert0(frame->buf[0]->data);
+
+ frame->data[0] = frame->buf[0]->data;
+ }
+
+ return frame;
+}
+
+
+static void open_data(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
+{
+ AVCodecContext *c;
+ int nb_samples = 100;
+ int ret;
+ AVDictionary *opt = NULL;
+
+ c = ost->enc;
+
+ /* open it */
+ av_dict_copy(&opt, opt_arg, 0);
+ ret = avcodec_open2(c, codec, &opt);
+ av_dict_free(&opt);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open meta codec: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ ost->frame = alloc_meta_frame(nb_samples);
+ ost->tmp_frame = alloc_meta_frame(nb_samples);
+
+ /* copy the stream parameters to the muxer */
+ ret = avcodec_parameters_from_context(ost->st->codecpar, c);
+ if (ret < 0) {
+ fprintf(stderr, "Could not copy the stream parameters\n");
+ exit(1);
+ }
+}
+
+
+/* Prepare a 16 bit dummy audio frame of 'frame_size' samples and
+ * 'nb_channels' channels. */
+static AVFrame *get_audio_frame(OutputStream *ost)
+{
+ AVFrame *frame = ost->tmp_frame;
+ int j, i, v;
+ int16_t *q = (int16_t*)frame->data[0];
+
+ /* check if we want to generate more frames */
+ if (av_compare_ts(ost->next_pts, ost->enc->time_base,
+ STREAM_DURATION, (AVRational){ 1, 1 }) >= 0)
+ return NULL;
+
+ for (j = 0; j <frame->nb_samples; j++) {
+ v = (int)(sin(ost->t) * 10000);
+ for (i = 0; i < ost->enc->channels; i++)
+ *q++ = v;
+ ost->t += ost->tincr;
+ ost->tincr += ost->tincr2;
+ }
+
+ frame->pts = ost->next_pts;
+ ost->next_pts += frame->nb_samples;
+
+ return frame;
+}
+
+static AVFrame *get_meta_frame(OutputStream *ost)
+{
+ AVFrame *frame = ost->tmp_frame;
+ unsigned char *buffer = (unsigned char*) frame->data[0];
+
+ static int n = 0;
+ ++n;
+
+ frame->pts = ost->next_pts;
+ ost->next_pts += 1;
+
+ snprintf((char*) buffer, 42, "urn:example.com:%d", n);
+ frame->data[0] = buffer;
+ frame->nb_samples = strlen((char*) buffer);
+ frame->pts = ost->next_pts++;
+
+ return frame;
+}
+
+static int write_timed_meta_frame(AVFormatContext *oc, OutputStream *ost)
+{
+ AVCodecContext *c;
+ AVPacket pkt = { 0 }; // data and size must be 0;
+ AVFrame *frame;
+ int ret;
+ int got_packet;
+
+ av_init_packet(&pkt);
+ c = ost->enc;
+
+ frame = get_meta_frame(ost);
+
+ if (frame) {
+ ost->samples_count += 1;
+ }
+
+ if ((ret = c->codec->encode2(c, &pkt, frame, &got_packet))) {
+ fprintf(stderr, "Error while encoding meta frame: %s\n",
+ av_err2str(ret));
+ exit(1);
+ } else {
+ c->frame_number++;
+
+ if (got_packet) {
+ ret = write_frame(oc, &c->time_base, ost->st, &pkt);
+ if (ret < 0) {
+ fprintf(stderr, "Error while writing meta frame: %s\n",
+ av_err2str(ret));
+ exit(1);
+ }
+ }
+ }
+
+ return (frame || got_packet) ? 0 : 1;
+}
+
+
+/*
+ * encode one audio frame and send it to the muxer
+ * return 1 when encoding is finished, 0 otherwise
+ */
+static int write_audio_frame(AVFormatContext *oc, OutputStream *ost)
+{
+ AVCodecContext *c;
+ AVPacket pkt = { 0 }; // data and size must be 0;
+ AVFrame *frame;
+ int ret;
+ int got_packet;
+ int dst_nb_samples;
+
+ av_init_packet(&pkt);
+ c = ost->enc;
+
+ frame = get_audio_frame(ost);
+
+ if (frame) {
+ /* convert samples from native format to destination codec format, using the resampler */
+ /* compute destination number of samples */
+ dst_nb_samples = av_rescale_rnd(swr_get_delay(ost->swr_ctx, c->sample_rate) + frame->nb_samples,
+ c->sample_rate, c->sample_rate, AV_ROUND_UP);
+ av_assert0(dst_nb_samples == frame->nb_samples);
+
+ /* when we pass a frame to the encoder, it may keep a reference to it
+ * internally;
+ * make sure we do not overwrite it here
+ */
+ ret = av_frame_make_writable(ost->frame);
+ if (ret < 0)
+ exit(1);
+
+ /* convert to destination format */
+ ret = swr_convert(ost->swr_ctx,
+ ost->frame->data, dst_nb_samples,
+ (const uint8_t **)frame->data, frame->nb_samples);
+ if (ret < 0) {
+ fprintf(stderr, "Error while converting\n");
+ exit(1);
+ }
+ frame = ost->frame;
+
+ frame->pts = av_rescale_q(ost->samples_count, (AVRational){1, c->sample_rate}, c->time_base);
+ ost->samples_count += dst_nb_samples;
+ }
+
+ ret = avcodec_encode_audio2(c, &pkt, frame, &got_packet);
+ if (ret < 0) {
+ fprintf(stderr, "Error encoding audio frame: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ if (got_packet) {
+ ret = write_frame(oc, &c->time_base, ost->st, &pkt);
+ if (ret < 0) {
+ fprintf(stderr, "Error while writing audio frame: %s\n",
+ av_err2str(ret));
+ exit(1);
+ }
+ }
+
+ return (frame || got_packet) ? 0 : 1;
+}
+
+/**************************************************************/
+/* video output */
+
+static AVFrame *alloc_picture(enum AVPixelFormat pix_fmt, int width, int height)
+{
+ AVFrame *picture;
+ int ret;
+
+ picture = av_frame_alloc();
+ if (!picture)
+ return NULL;
+
+ picture->format = pix_fmt;
+ picture->width = width;
+ picture->height = height;
+
+ /* allocate the buffers for the frame data */
+ ret = av_frame_get_buffer(picture, 32);
+ if (ret < 0) {
+ fprintf(stderr, "Could not allocate frame data.\n");
+ exit(1);
+ }
+
+ return picture;
+}
+
+static void open_video(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
+{
+ int ret;
+ AVCodecContext *c = ost->enc;
+ AVDictionary *opt = NULL;
+
+ av_dict_copy(&opt, opt_arg, 0);
+
+ /* open the codec */
+ ret = avcodec_open2(c, codec, &opt);
+ av_dict_free(&opt);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open video codec: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ /* allocate and init a re-usable frame */
+ ost->frame = alloc_picture(c->pix_fmt, c->width, c->height);
+ if (!ost->frame) {
+ fprintf(stderr, "Could not allocate video frame\n");
+ exit(1);
+ }
+
+ /* If the output format is not YUV420P, then a temporary YUV420P
+ * picture is needed too. It is then converted to the required
+ * output format. */
+ ost->tmp_frame = NULL;
+ if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
+ ost->tmp_frame = alloc_picture(AV_PIX_FMT_YUV420P, c->width, c->height);
+ if (!ost->tmp_frame) {
+ fprintf(stderr, "Could not allocate temporary picture\n");
+ exit(1);
+ }
+ }
+
+ /* copy the stream parameters to the muxer */
+ ret = avcodec_parameters_from_context(ost->st->codecpar, c);
+ if (ret < 0) {
+ fprintf(stderr, "Could not copy the stream parameters\n");
+ exit(1);
+ }
+}
+
+/* Prepare a dummy image. */
+static void fill_yuv_image(AVFrame *pict, int frame_index,
+ int width, int height)
+{
+ int x, y, i, ret;
+
+ /* when we pass a frame to the encoder, it may keep a reference to it
+ * internally;
+ * make sure we do not overwrite it here
+ */
+ ret = av_frame_make_writable(pict);
+ if (ret < 0)
+ exit(1);
+
+ i = frame_index;
+
+ /* Y */
+ for (y = 0; y < height; y++)
+ for (x = 0; x < width; x++)
+ pict->data[0][y * pict->linesize[0] + x] = x + y + i * 3;
+
+ /* Cb and Cr */
+ for (y = 0; y < height / 2; y++) {
+ for (x = 0; x < width / 2; x++) {
+ pict->data[1][y * pict->linesize[1] + x] = 128 + y + i * 2;
+ pict->data[2][y * pict->linesize[2] + x] = 64 + x + i * 5;
+ }
+ }
+}
+
+static AVFrame *get_video_frame(OutputStream *ost)
+{
+ AVCodecContext *c = ost->enc;
+
+ /* check if we want to generate more frames */
+ if (av_compare_ts(ost->next_pts, ost->enc->time_base,
+ STREAM_DURATION, (AVRational){ 1, 1 }) >= 0)
+ return NULL;
+
+ if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
+ /* as we only generate a YUV420P picture, we must convert it
+ * to the codec pixel format if needed */
+ if (!ost->sws_ctx) {
+ ost->sws_ctx = sws_getContext(c->width, c->height,
+ AV_PIX_FMT_YUV420P,
+ c->width, c->height,
+ c->pix_fmt,
+ SCALE_FLAGS, NULL, NULL, NULL);
+ if (!ost->sws_ctx) {
+ fprintf(stderr,
+ "Could not initialize the conversion context\n");
+ exit(1);
+ }
+ }
+ fill_yuv_image(ost->tmp_frame, ost->next_pts, c->width, c->height);
+ sws_scale(ost->sws_ctx,
+ (const uint8_t * const *)ost->tmp_frame->data, ost->tmp_frame->linesize,
+ 0, c->height, ost->frame->data, ost->frame->linesize);
+ } else {
+ fill_yuv_image(ost->frame, ost->next_pts, c->width, c->height);
+ }
+
+ ost->frame->pts = ost->next_pts++;
+
+ return ost->frame;
+}
+
+/*
+ * encode one video frame and send it to the muxer
+ * return 1 when encoding is finished, 0 otherwise
+ */
+static int write_video_frame(AVFormatContext *oc, OutputStream *ost)
+{
+ int ret;
+ AVCodecContext *c;
+ AVFrame *frame;
+ int got_packet = 0;
+
+ c = ost->enc;
+
+ frame = get_video_frame(ost);
+
+ if (oc->oformat->flags & AVFMT_RAWPICTURE) {
+ /* a hack to avoid data copy with some raw video muxers */
+ AVPacket pkt;
+ av_init_packet(&pkt);
+
+ if (!frame)
+ return 1;
+
+ pkt.flags |= AV_PKT_FLAG_KEY;
+ pkt.stream_index = ost->st->index;
+ pkt.data = (uint8_t *)frame;
+ pkt.size = sizeof(AVPicture);
+
+ pkt.pts = pkt.dts = frame->pts;
+ av_packet_rescale_ts(&pkt, c->time_base, ost->st->time_base);
+
+ ret = av_interleaved_write_frame(oc, &pkt);
+ } else {
+ AVPacket pkt = { 0 };
+ av_init_packet(&pkt);
+
+ /* encode the image */
+ ret = avcodec_encode_video2(c, &pkt, frame, &got_packet);
+ if (ret < 0) {
+ fprintf(stderr, "Error encoding video frame: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ if (got_packet) {
+ ret = write_frame(oc, &c->time_base, ost->st, &pkt);
+ } else {
+ ret = 0;
+ }
+ }
+
+ if (ret < 0) {
+ fprintf(stderr, "Error while writing video frame: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ return (frame || got_packet) ? 0 : 1;
+}
+
+static void close_stream(AVFormatContext *oc, OutputStream *ost)
+{
+ avcodec_close(ost->enc);
+ av_frame_free(&ost->frame);
+ av_frame_free(&ost->tmp_frame);
+ sws_freeContext(ost->sws_ctx);
+ swr_free(&ost->swr_ctx);
+}
+
+/**************************************************************/
+/* media file output */
+
+int main(int argc, char **argv)
+{
+ OutputStream video_st = { 0 }, audio_st = { 0 }, data_st = { 0 };
+ const char *filename;
+ AVOutputFormat *fmt;
+ AVFormatContext *oc;
+ AVCodec *audio_codec, *video_codec, *data_codec;
+ int ret;
+ int have_video = 0, have_audio = 0, have_data = 0;
+ int encode_video = 0, encode_audio = 0, mux_meta = 0;
+ AVDictionary *opt = NULL;
+
+ StreamState streams[3] = {
+ { write_video_frame,
+ &encode_video,
+ &video_st },
+ { write_audio_frame,
+ &encode_audio,
+ &audio_st },
+ { write_timed_meta_frame,
+ &mux_meta,
+ &data_st }
+ };
+
+ /* Initialize libavcodec, and register all codecs and formats. */
+ av_register_all();
+
+ if (argc < 2) {
+ printf("usage: %s output_file\n"
+ "API example program to output a media file with libavformat.\n"
+ "This program generates a synthetic audio and video stream, encodes and\n"
+ "muxes them into a file named output_file.\n"
+ "The output format is automatically guessed according to the file extension.\n"
+ "Raw images can also be output by using '%%d' in the filename.\n"
+ "\n", argv[0]);
+ return 1;
+ }
+
+ filename = argv[1];
+ if (argc > 3 && !strcmp(argv[2], "-flags")) {
+ av_dict_set(&opt, argv[2]+1, argv[3], 0);
+ }
+
+ /* allocate the output media context */
+ avformat_alloc_output_context2(&oc, NULL, NULL, filename);
+ if (!oc) {
+ printf("Could not deduce output format from file extension: using MPEG.\n");
+ avformat_alloc_output_context2(&oc, NULL, "mpeg", filename);
+ }
+ if (!oc)
+ return 1;
+
+ fmt = oc->oformat;
+
+ /* Add the audio and video streams using the default format codecs
+ * and initialize the codecs. */
+ if (fmt->video_codec != AV_CODEC_ID_NONE) {
+ add_stream(&video_st, oc, &video_codec, fmt->video_codec);
+ have_video = 1;
+ encode_video = 1;
+ }
+ if (fmt->audio_codec != AV_CODEC_ID_NONE) {
+ add_stream(&audio_st, oc, &audio_codec, fmt->audio_codec);
+ have_audio = 1;
+ encode_audio = 1;
+ }
+ if (fmt->data_codec != AV_CODEC_ID_NONE) {
+ add_stream(&data_st, oc, &data_codec, fmt->data_codec);
+ have_data = 1;
+ mux_meta = 1;
+ }
+
+ /* Now that all the parameters are set, we can open the audio and
+ * video codecs and allocate the necessary encode buffers. */
+ if (have_video)
+ open_video(oc, video_codec, &video_st, opt);
+
+ if (have_audio)
+ open_audio(oc, audio_codec, &audio_st, opt);
+
+ if (have_data)
+ open_data(oc, data_codec, &data_st, opt);
+
+ av_dump_format(oc, 0, filename, 1);
+
+ /* open the output file, if needed */
+ if (!(fmt->flags & AVFMT_NOFILE)) {
+ ret = avio_open(&oc->pb, filename, AVIO_FLAG_WRITE);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open '%s': %s\n", filename,
+ av_err2str(ret));
+ return 1;
+ }
+ }
+
+ /* Write the stream header, if any. */
+ ret = avformat_write_header(oc, &opt);
+ if (ret < 0) {
+ fprintf(stderr, "Error occurred when opening output file: %s\n",
+ av_err2str(ret));
+ return 1;
+ }
+
+ if (fmt->data_codec != AV_CODEC_ID_NONE) {
+ AVTimedMetadata* metadata;
+ char* uri = "http://example.com/";
+ void* ptr;
+
+ AVTrackReferences* trefs;
+ char* msg = "test config data";
+ int nb_tref_tracks = 2;
+
+ metadata = (AVTimedMetadata*) av_stream_new_side_data(oc->streams[2], AV_PKT_DATA_TIMED_METADATA_INFO,
+ sizeof(AVTimedMetadata) + strlen(uri) + strlen(msg));
+
+ memcpy(metadata->meta_tag, "urim", 4);
+ metadata->meta_length = strlen(uri);
+
+ memset(metadata->conf_tag, 0, sizeof(metadata->conf_tag));
+ memcpy(metadata->conf_tag, "cfg ", 4);
+ metadata->conf_length = strlen(msg);
+
+ ptr = (char*) (metadata + 1);
+ memcpy(ptr, uri, strlen(uri));
+ ptr += strlen(uri);
+ memcpy(ptr, msg, strlen(msg));
+
+ trefs = (AVTrackReferences*) av_stream_new_side_data(oc->streams[2], AV_PKT_DATA_TRACK_REFERENCES,
+ sizeof(AVTrackReferences) + nb_tref_tracks * sizeof(int));
+
+ memcpy(trefs->tag, "cdsc", 4);
+ trefs->nb_tracks = nb_tref_tracks;
+ ((int*) (trefs + 1))[0] = 0;
+ ((int*) (trefs + 1))[1] = 1;
+ }
+
+ while (encode_video || encode_audio) {
+ StreamState *stream = NULL;
+ int idx;
+
+ for (idx = 0; idx < 3; ++idx) {
+ if (*streams[idx].flag &&
+ (stream == NULL ||
+ av_compare_ts(streams[idx].stream->next_pts, streams[idx].stream->enc->time_base,
+ stream->stream->next_pts, stream->stream->enc->time_base) <= 0)) {
+ stream = &streams[idx];
+ }
+ }
+
+ if (stream) {
+ *stream->flag = !stream->writer(oc, stream->stream);
+ }
+ }
+
+ /* Write the trailer, if any. The trailer must be written before you
+ * close the CodecContexts open when you wrote the header; otherwise
+ * av_write_trailer() may try to use memory that was freed on
+ * av_codec_close(). */
+ av_write_trailer(oc);
+
+ /* Close each codec. */
+ if (have_video)
+ close_stream(oc, &video_st);
+ if (have_audio)
+ close_stream(oc, &audio_st);
+ if (mux_meta)
+ close_stream(oc, &data_st);
+
+ if (!(fmt->flags & AVFMT_NOFILE))
+ /* Close the output file. */
+ avio_closep(&oc->pb);
+
+ /* free the stream */
+ avformat_free_context(oc);
+
+ return 0;
+}
--
2.7.4
e***@nokia.com
2016-08-23 09:03:39 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
Changelog | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/Changelog b/Changelog
index 71abe8c..e45ab1e 100644
--- a/Changelog
+++ b/Changelog
@@ -17,7 +17,26 @@ version <next>:
- acrusher audio filter
- bitplanenoise video filter
- floating point support in als decoder
-
+- libavutil/mem: added av_arraydup to accompany av_realloc*_array functions
+- libavformat/utils: added ability to probe AVMEDIA_TYPE_DATA format
+- libavformat/movenc: added ability to use original stream ids as track ids instead of regenerating them
+- libavcodec: added a structure AVData for decoding timed metadata
+- libavformat/movenc: mov_write_ftyp_tag: write the major brand a compatible brand
+- libavcodec/avcodec.h: AV_CODEC_ID_META added for timed meta data
+- libavcodec/metaenc: added an encoder/decoder for timed metadata
+- libavformat/movenc: deal with AVMEDIA_TYPE_DATA by using AV_CODEC_ID_META
+- libavformat/movenc: support for embedding timed metadata
+- libavformat/movenc: add the urim sample descriptor
+- libavformat/movenc, isom: support metadata in mp4 files
+- libavformat/mov, isom: read (multiple) track references (tag and multiple ids)
+- libavformat/mov: support for identifying (and reading) timed metadata (urim)
+- libavformat/mov: added support for AVMEDIA_TYPE_DATA
+- libavcodec/avcodec, libavformat/movenc: support embedding channel layout to stream side data
+- libavcodec: introduced AV_PKT_DATA_TRACK_ALTERNATE_GROUP side data for expressing alternate groups
+- libavformat/movenc: support for setting explicit alternate group for a track
+- libavformat/movenc: mov_write_audio_tag writes the proper number of channels, not the hardcoded 2
+- doc/examples/extract_timed_metadata: added a bare-bones metadata extractor
+- doc/examples/muxing_with_metadata: example for dealing with timed meta data

version 3.1:
- DXVA2-accelerated HEVC Main10 decoding
--
2.7.4
Paul B Mahol
2016-08-23 09:08:33 UTC
Permalink
Post by e***@nokia.com
---
Changelog | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)
diff --git a/Changelog b/Changelog
index 71abe8c..e45ab1e 100644
--- a/Changelog
+++ b/Changelog
- acrusher audio filter
- bitplanenoise video filter
- floating point support in als decoder
-
+- libavutil/mem: added av_arraydup to accompany av_realloc*_array functions
+- libavformat/utils: added ability to probe AVMEDIA_TYPE_DATA format
+- libavformat/movenc: added ability to use original stream ids as track ids
instead of regenerating them
+- libavcodec: added a structure AVData for decoding timed metadata
+- libavformat/movenc: mov_write_ftyp_tag: write the major brand a compatible brand
+- libavcodec/avcodec.h: AV_CODEC_ID_META added for timed meta data
+- libavcodec/metaenc: added an encoder/decoder for timed metadata
+- libavformat/movenc: deal with AVMEDIA_TYPE_DATA by using AV_CODEC_ID_META
+- libavformat/movenc: support for embedding timed metadata
+- libavformat/movenc: add the urim sample descriptor
+- libavformat/movenc, isom: support metadata in mp4 files
+- libavformat/mov, isom: read (multiple) track references (tag and multiple ids)
+- libavformat/mov: support for identifying (and reading) timed metadata (urim)
+- libavformat/mov: added support for AVMEDIA_TYPE_DATA
+- libavcodec/avcodec, libavformat/movenc: support embedding channel layout
to stream side data
+- libavcodec: introduced AV_PKT_DATA_TRACK_ALTERNATE_GROUP side data for
expressing alternate groups
+- libavformat/movenc: support for setting explicit alternate group for a track
+- libavformat/movenc: mov_write_audio_tag writes the proper number of
channels, not the hardcoded 2
+- doc/examples/extract_timed_metadata: added a bare-bones metadata extractor
+- doc/examples/muxing_with_metadata: example for dealing with timed meta data
Commit logs does not belong to Changelog
Erkki Seppälä
2016-08-23 09:12:49 UTC
Permalink
Post by Paul B Mahol
Commit logs does not belong to Changelog
That is a good point, though it was (ever-so-slightly) revised from the
raw change log. I can write a version following the points mentioned in
the cover letter for a more high-level view.
e***@nokia.com
2016-08-23 09:03:28 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

This also adds libavformat/movmeta that contains the meta data
information. Later commits will add public interfaces for accessing
the data.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 18 ++++++++++-
libavformat/isom.c | 1 +
libavformat/movenc.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++++---
libavformat/movenc.h | 1 +
libavformat/movmeta.h | 46 ++++++++++++++++++++++++++++
5 files changed, 144 insertions(+), 5 deletions(-)
create mode 100644 libavformat/movmeta.h

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 893b89b..36c85e9 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1349,6 +1349,16 @@ typedef struct AVCPBProperties {
* @{
*/

+typedef struct AVTimedMetadata {
+ char meta_tag[4]; /** 4cc describing this metadata box type */
+ int meta_length; /** length of data for the metadata type information */
+
+ char conf_tag[4]; /** configurationg box type 4cc, ie. 'conf' */
+ int conf_length; /** length of the data for the configuration box */
+
+ /** followed by meta_length bytes of meta data followed by conf_length bytes of conf data */
+} AVTimedMetadata;
+
typedef struct AVTrackReferences {
char tag[4]; /** 4cc used for describing this */
int nb_tracks; /** number of tracks */
@@ -1540,7 +1550,13 @@ enum AVPacketSideDataType {
* (including the track list that follows it), for as long as
* indicated by the key's length.
*/
- AV_PKT_DATA_TRACK_REFERENCES
+ AV_PKT_DATA_TRACK_REFERENCES,
+
+ /**
+ * Configured the timed metadata parameters, such as the uri and
+ * meta data configuration. The key is of type AVTimedMetadata.
+ */
+ AV_PKT_DATA_TIMED_METADATA_INFO
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/isom.c b/libavformat/isom.c
index 1a90d00..473700f 100644
--- a/libavformat/isom.c
+++ b/libavformat/isom.c
@@ -357,6 +357,7 @@ const AVCodecTag ff_codec_movsubtitle_tags[] = {

const AVCodecTag ff_codec_metadata_tags[] = {
{ AV_CODEC_ID_META, MKTAG('m', 'e', 't', 'a') },
+ { AV_CODEC_ID_META, MKTAG('u', 'r', 'i', 'm') },
{ AV_CODEC_ID_NONE, 0 },
};

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 072e660..141f1a5 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1552,6 +1552,69 @@ static int mov_write_fiel_tag(AVIOContext *pb, MOVTrack *track, int field_order)
return 10;
}

+static int mov_write_urim_uri_box(AVIOContext *pb, const char *uri, int length)
+{
+ int64_t pos = avio_tell(pb);
+ avio_wb32(pb, 0); /* size */
+ ffio_wfourcc(pb, "uri ");
+ avio_w8(pb, 0); /* version */
+ avio_wb24(pb, 0); /* flags */
+
+ avio_write(pb, uri, length); /* uri */
+ avio_w8(pb, 0); /* null-terminated */
+
+ return update_size(pb, pos);
+}
+
+static int mov_write_conf_box(AVIOContext *pb, const uint8_t *data, int length, char* tag)
+{
+ int64_t pos = avio_tell(pb);
+ avio_wb32(pb, 0); /* size */
+ ffio_wfourcc(pb, tag);
+
+ avio_w8(pb, 0); /* version */
+ avio_wb24(pb, 0); /* flags */
+
+ avio_write(pb, data, length); /* data */
+
+ return update_size(pb, pos);
+}
+
+static int mov_write_meta_codec(AVIOContext *pb, MOVTrack *track)
+{
+ int64_t pos = avio_tell(pb);
+
+ char *data;
+ char *tag;
+
+ AVTimedMetadata *meta =
+ (void*) av_stream_get_side_data(track->st, AV_PKT_DATA_TIMED_METADATA_INFO,
+ NULL);
+
+ av_assert0(meta);
+
+ avio_wb32(pb, 0); /* size */
+ ffio_wfourcc(pb, meta->meta_tag);
+
+ avio_wb32(pb, 0); /* Reserved */
+ avio_wb16(pb, 0); /* Reserved */
+ avio_wb16(pb, 1); /* Data-reference index */
+
+ data = (char*) (meta + 1);
+ tag = meta->meta_tag;
+ if (tag[0] == 'u' && tag[1] == 'r' && tag[2] == 'i' && tag[3] == 'm') {
+ mov_write_urim_uri_box(pb, data, meta->meta_length);
+ data += meta->meta_length;
+ }
+
+ if (meta->conf_length) {
+ mov_write_conf_box(pb, data, meta->conf_length, meta->conf_tag);
+ data += meta->conf_length;
+ }
+
+ return update_size(pb, pos);
+}
+
static int mov_write_subtitle_tag(AVIOContext *pb, MOVTrack *track)
{
int64_t pos = avio_tell(pb);
@@ -1951,6 +2014,18 @@ static int mov_write_tmcd_tag(AVIOContext *pb, MOVTrack *track)
return update_size(pb, pos);
}

+static int mov_write_data_tag(AVIOContext *pb, MOVTrack *track)
+{
+ if (track->par->codec_id == AV_CODEC_ID_META)
+ return mov_write_meta_codec(pb, track);
+ else if (track->par->codec_tag == MKTAG('t','m','c','d'))
+ return mov_write_tmcd_tag(pb, track);
+ else if (track->par->codec_tag == MKTAG('r','t','p',' '))
+ return mov_write_rtp_tag(pb, track);
+ else
+ return 0;
+}
+
static int mov_write_stsd_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track)
{
int64_t pos = avio_tell(pb);
@@ -1962,12 +2037,10 @@ static int mov_write_stsd_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
mov_write_video_tag(pb, mov, track);
else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO)
mov_write_audio_tag(s, pb, mov, track);
+ else if (track->par->codec_type == AVMEDIA_TYPE_DATA)
+ mov_write_data_tag(pb, track);
else if (track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)
mov_write_subtitle_tag(pb, track);
- else if (track->par->codec_tag == MKTAG('r','t','p',' '))
- mov_write_rtp_tag(pb, track);
- else if (track->par->codec_tag == MKTAG('t','m','c','d'))
- mov_write_tmcd_tag(pb, track);
return update_size(pb, pos);
}

@@ -2309,6 +2382,8 @@ static int mov_write_minf_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
mov_write_vmhd_tag(pb);
else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO)
mov_write_smhd_tag(pb);
+ else if (track->par->codec_type == AVMEDIA_TYPE_DATA)
+ mov_write_nmhd_tag(pb);
else if (track->par->codec_type == AVMEDIA_TYPE_SUBTITLE) {
if (track->tag == MKTAG('t','e','x','t') || is_clcp_track(track)) {
mov_write_gmhd_tag(pb, track);
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index 5bf9469..15aa4b3 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -26,6 +26,7 @@

#include "avformat.h"
#include "movenccenc.h"
+#include "movmeta.h"

#define MOV_FRAG_INFO_ALLOC_INCREMENT 64
#define MOV_INDEX_CLUSTER_SIZE 1024
diff --git a/libavformat/movmeta.h b/libavformat/movmeta.h
new file mode 100644
index 0000000..e57b381
--- /dev/null
+++ b/libavformat/movmeta.h
@@ -0,0 +1,46 @@
+/*
+ * MOV, 3GP, MP4 muxer
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVFORMAT_MOVMETA_H
+#define AVFORMAT_MOVMETA_H
+
+#include "avformat.h"
+
+typedef enum MOVMetaType {
+ MOV_META_NONE,
+ MOV_META_URIM
+} MOVMetaType;
+
+typedef struct MOVConfMeta {
+ uint32_t tag;
+ int length;
+ void *data;
+} MOVConfMeta;
+
+typedef struct MOVMeta {
+ uint32_t tag;
+ int length;
+ void *data;
+
+ MOVConfMeta conf;
+} MOVMeta;
+
+#endif /* AVFORMAT_MOVMETA_H */
--
2.7.4
e***@nokia.com
2016-08-23 09:03:30 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

This can be useful in particular with timed meta data tracks related
to multiple tracks.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.h | 3 +++
libavformat/mov.c | 46 +++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 48 insertions(+), 1 deletion(-)

diff --git a/libavformat/isom.h b/libavformat/isom.h
index 49c8996..9ad898a 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -168,6 +168,9 @@ typedef struct MOVStreamContext {
int start_pad; ///< amount of samples to skip due to enc-dec delay
unsigned int rap_group_count;
MOVSbgp *rap_group;
+ uint32_t tref_tag;
+ int nb_tref_ids;
+ int *tref_ids; ///< trackIDs of the referenced tracks

int nb_frames_for_fps;
int64_t duration_for_fps;
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 1bc3800..9cd915d 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -3157,6 +3157,48 @@ static void fix_timescale(MOVContext *c, MOVStreamContext *sc)
}
}

+static int mov_read_tref(MOVContext *c, AVIOContext *pb, MOVAtom atom)
+{
+ uint32_t size = avio_rb32(pb);
+ AVStream *st;
+ MOVStreamContext *sc;
+ int ret;
+
+ if (c->fc->nb_streams < 1)
+ return 0;
+
+ st = c->fc->streams[c->fc->nb_streams-1];
+ sc = st->priv_data;
+
+ if (size < 12 || size > atom.size) {
+ avio_seek(pb, -4, SEEK_CUR);
+ return mov_read_default(c, pb, atom);
+ } else {
+ int remaining = size - 4;
+
+ if (remaining % 4 != 0)
+ return AVERROR_INVALIDDATA;
+
+ /* an arbitrary upper limit to prevent wasting all memory to this */
+ if (remaining > 4 * 256)
+ remaining = 4 * 256;
+
+ ret = av_reallocp_array(&sc->tref_ids, remaining / 4, sizeof(*sc->tref_ids));
+ if (ret != 0)
+ return ret;
+
+ sc->tref_tag = avio_rl32(pb);
+ remaining -= 4;
+ while (remaining > 0) {
+ sc->tref_ids[sc->nb_tref_ids] = avio_rb32(pb);
+ sc->nb_tref_ids++;
+ remaining -= 4;
+ }
+ avio_seek(pb, remaining, SEEK_CUR);
+ return ret;
+ }
+}
+
static int mov_read_trak(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
AVStream *st;
@@ -4414,7 +4456,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = {
{ MKTAG('t','f','h','d'), mov_read_tfhd }, /* track fragment header */
{ MKTAG('t','r','a','k'), mov_read_trak },
{ MKTAG('t','r','a','f'), mov_read_default },
-{ MKTAG('t','r','e','f'), mov_read_default },
+{ MKTAG('t','r','e','f'), mov_read_tref },
{ MKTAG('t','m','c','d'), mov_read_tmcd },
{ MKTAG('c','h','a','p'), mov_read_chap },
{ MKTAG('t','r','e','x'), mov_read_trex },
@@ -4832,6 +4874,8 @@ static int mov_read_close(AVFormatContext *s)
av_freep(&sc->cenc.auxiliary_info);
av_freep(&sc->cenc.auxiliary_info_sizes);
av_aes_ctr_free(sc->cenc.aes_ctr);
+
+ av_freep(&sc->tref_ids);
}

if (mov->dv_demux) {
--
2.7.4
e***@nokia.com
2016-08-23 09:03:35 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Alternate groups previously always generated for ISO media files. With
this addition client code can define track groups arbitrarily.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 13 ++++++++++++-
libavformat/movenc.c | 9 +++++++++
2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 6c64e6a..9d69911 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1605,7 +1605,18 @@ enum AVPacketSideDataType {
* The channel layout is object structured with the number of objects in
* AVAudioTrackChannelLayoutObjectStructured
*/
- AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED
+ AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED,
+
+ /**
+ * Assign alternate groups for tracks. An example of alternate
+ * groups would be audio tracks (or video tracks) that are
+ * alternative to each other. Each alternative track shares the
+ * same non-zero alternate group.
+ *
+ * The content is:
+ * uint: The alternate group of this track
+ */
+ AV_PKT_DATA_TRACK_ALTERNATE_GROUP
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 9606918..94d978b 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2567,6 +2567,8 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov,
int flags = MOV_TKHD_FLAG_IN_MOVIE;
int rotation = 0;
int group = 0;
+ int *alternate_group = NULL;
+ int alternate_group_size;

uint32_t *display_matrix = NULL;
int display_matrix_size, i;
@@ -2583,6 +2585,13 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov,
display_matrix = NULL;
}

+ if (st) {
+ alternate_group = (int*) av_stream_get_side_data(st, AV_PKT_DATA_TRACK_ALTERNATE_GROUP,
+ &alternate_group_size);
+ if (alternate_group && alternate_group_size >= sizeof(int))
+ group = *alternate_group;
+ }
+
if (track->flags & MOV_TRACK_ENABLED)
flags |= MOV_TKHD_FLAG_ENABLED;
--
2.7.4
e***@nokia.com
2016-08-23 09:03:36 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/movenc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 94d978b..020d13d 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1039,7 +1039,7 @@ static int mov_write_audio_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex
avio_wb16(pb, 16);
avio_wb16(pb, track->audio_vbr ? -2 : 0); /* compression ID */
} else { /* reserved for mp4/3gp */
- avio_wb16(pb, 2);
+ avio_wb16(pb, track->par->channels);
avio_wb16(pb, 16);
avio_wb16(pb, 0);
}
--
2.7.4
Yusuke Nakamura
2016-08-23 15:20:42 UTC
Permalink
Post by e***@nokia.com
---
libavformat/movenc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 94d978b..020d13d 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1039,7 +1039,7 @@ static int mov_write_audio_tag(AVFormatContext *s,
AVIOContext *pb, MOVMuxContex
avio_wb16(pb, 16);
avio_wb16(pb, track->audio_vbr ? -2 : 0); /* compression ID */
} else { /* reserved for mp4/3gp */
- avio_wb16(pb, 2);
+ avio_wb16(pb, track->par->channels);
No. the ChannelCount field is templated. It may be fixed to 2 by derived
specs or the specs of codec encapsulations. I mean the current
implemetation is wrong, and your implementation is also wrong. For
instance, ChannelCount of AC-3 in ISOBMFF is always hardcoded to 2 even if
5.1ch. This field is basically not useful for ISOBMFF and the actual
channels shall be referred to Codec specific info.

avio_wb16(pb, 16);
Post by e***@nokia.com
avio_wb16(pb, 0);
}
--
2.7.4
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Erkki Seppälä
2016-08-24 07:14:30 UTC
Permalink
Post by Yusuke Nakamura
Post by e***@nokia.com
- avio_wb16(pb, 2);
+ avio_wb16(pb, track->par->channels);
No. the ChannelCount field is templated. It may be fixed to 2 by derived
specs or the specs of codec encapsulations. I mean the current
implemetation is wrong, and your implementation is also wrong. For
instance, ChannelCount of AC-3 in ISOBMFF is always hardcoded to 2 even if
5.1ch. This field is basically not useful for ISOBMFF and the actual
channels shall be referred to Codec specific info.
It seems the part of the standard didn't give the full view on the
matter. This patch can be dropped.

Thanks for the review!
e***@nokia.com
2016-08-23 09:03:25 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

This encoder is used primarily with ISO media files and basically
copies the data, while allowing the rest of the FFMPEG to refer
to the meta data with its identifiers.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
configure | 4 ++--
libavcodec/Makefile | 2 +-
libavcodec/allcodecs.c | 3 +++
libavcodec/codec_desc.c | 8 ++++++++
4 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 9b92426..ded5452 100755
--- a/configure
+++ b/configure
@@ -2848,11 +2848,11 @@ matroska_demuxer_select="iso_media riffdec"
matroska_demuxer_suggest="bzlib lzo zlib"
matroska_muxer_select="iso_media riffenc"
mmf_muxer_select="riffenc"
-mov_demuxer_select="iso_media riffdec"
+mov_demuxer_select="iso_media riffdec meta_decoder"
mov_demuxer_suggest="zlib"
mov_muxer_select="iso_media riffenc rtpenc_chain"
mp3_demuxer_select="mpegaudio_parser"
-mp4_muxer_select="mov_muxer"
+mp4_muxer_select="mov_muxer meta_encoder"
mpegts_demuxer_select="iso_media"
mpegts_muxer_select="adts_muxer latm_muxer"
mpegtsraw_demuxer_select="mpegts_demuxer"
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index a6e79ce..98a7a8d 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -799,7 +799,7 @@ OBJS-$(CONFIG_VP9_DXVA2_HWACCEL) += dxva2_vp9.o
OBJS-$(CONFIG_VP9_VAAPI_HWACCEL) += vaapi_vp9.o

# libavformat dependencies
-OBJS-$(CONFIG_ISO_MEDIA) += mpeg4audio.o mpegaudiodata.o
+OBJS-$(CONFIG_ISO_MEDIA) += mpeg4audio.o mpegaudiodata.o metacodec.o

OBJS-$(CONFIG_ADTS_MUXER) += mpeg4audio.o
OBJS-$(CONFIG_CAF_DEMUXER) += ac3tab.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 4c6b94e..30d0243 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -687,4 +687,7 @@ void avcodec_register_all(void)
REGISTER_PARSER(VP3, vp3);
REGISTER_PARSER(VP8, vp8);
REGISTER_PARSER(VP9, vp9);
+
+ /* data, meta data */
+ REGISTER_ENCDEC(META, meta);
}
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 24948ca..e85b51d 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -2965,6 +2965,14 @@ static const AVCodecDescriptor codec_descriptors[] = {
.mime_types= MT("application/octet-stream"),
},

+ {
+ .id = AV_CODEC_ID_META,
+ .type = AVMEDIA_TYPE_DATA,
+ .name = "meta_data",
+ .long_name = NULL_IF_CONFIG_SMALL("binary data"),
+ .mime_types= MT("application/octet-stream"),
+ },
+
/* deprecated codec ids */
};
--
2.7.4
Michael Niedermayer
2016-08-23 14:46:02 UTC
Permalink
Post by e***@nokia.com
This encoder is used primarily with ISO media files and basically
copies the data, while allowing the rest of the FFMPEG to refer
to the meta data with its identifiers.
---
configure | 4 ++--
libavcodec/Makefile | 2 +-
libavcodec/allcodecs.c | 3 +++
libavcodec/codec_desc.c | 8 ++++++++
4 files changed, 14 insertions(+), 3 deletions(-)
this breaks fate
make distclean ; ./configure && make -j12 fate
deadlocks and never finishes

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

It is what and why we do it that matters, not just one of them.
Erkki Seppälä
2016-08-25 15:58:41 UTC
Permalink
Post by Michael Niedermayer
this breaks fate
make distclean ; ./configure && make -j12 fate
deadlocks and never finishes
Well that was embarrassing, thanks for pointing it out.

The issue was that I had introduced metacodec_class that was shared
between ff_meta_encoder and ff_meta_decoder, and due to this sharing
av_opt_child_class_next was never able to get pass them as it
automatically returned to the same metacodec_class on every iteration.

As it seems the metacodec_class was useless in the first place, I opted
to remove it. make fate now doesn't hang, but I discovered uhm issues
with some of the other patches that will be fixed in v2.

I'll be sure to ensure make fate passes all my next patches. Thanks.
e***@nokia.com
2016-08-23 09:03:32 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/mov.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/libavformat/mov.c b/libavformat/mov.c
index f18c0a3..275b532 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -2259,6 +2259,7 @@ int ff_mov_read_stsd_entries(MOVContext *c, AVIOContext *pb, int entries)
(format >> 0) & 0xff, (format >> 8) & 0xff, (format >> 16) & 0xff,
(format >> 24) & 0xff, format, st->codecpar->codec_type);

+ ret = 0;
if (st->codecpar->codec_type==AVMEDIA_TYPE_VIDEO) {
st->codecpar->codec_id = id;
mov_parse_stsd_video(c, pb, st, sc);
@@ -2269,12 +2270,16 @@ int ff_mov_read_stsd_entries(MOVContext *c, AVIOContext *pb, int entries)
st->codecpar->codec_id = id;
mov_parse_stsd_subtitle(c, pb, st, sc,
size - (avio_tell(pb) - start_pos));
+ } else if (st->codecpar->codec_type==AVMEDIA_TYPE_DATA){
+ st->codecpar->codec_id = id;
+ ret = mov_parse_stsd_data(c, pb, st, sc,
+ size - (avio_tell(pb) - start_pos));
} else {
ret = mov_parse_stsd_data(c, pb, st, sc,
size - (avio_tell(pb) - start_pos));
- if (ret < 0)
- return ret;
}
+ if (ret < 0)
+ return ret;
/* this will read extra atoms at the end (wave, alac, damr, avcC, hvcC, SMI ...) */
a.size = size - (avio_tell(pb) - start_pos);
if (a.size > 8) {
--
2.7.4
e***@nokia.com
2016-08-23 09:03:27 UTC
Permalink
From: Erkki Seppälä <***@nokia.com>

Instead of one track reference, allow multiple. In addition, allow
client to explicitly add track references with side packet
AV_PKG_DATA_TRACK_REFERENCES containing AVTrackReferences. MOVTrack's
track references can be manipulated with helper functions
ff_mov_*tref*.

Multiple track references can be useful in particular with timed meta
data tracks, indicating the track is related to multiple other tracks.

This information ends up in ISO media file box 'tref' as specified by
ISO/IEC 14496-12.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 17 +++-
libavformat/movenc.c | 245 ++++++++++++++++++++++++++++++++++++++++++-----
libavformat/movenc.h | 61 +++++++++++-
libavformat/movenchint.c | 11 ++-
4 files changed, 303 insertions(+), 31 deletions(-)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 756eda5..893b89b 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1348,6 +1348,13 @@ typedef struct AVCPBProperties {
* Types and functions for working with AVPacket.
* @{
*/
+
+typedef struct AVTrackReferences {
+ char tag[4]; /** 4cc used for describing this */
+ int nb_tracks; /** number of tracks */
+ /** followed by: int tracks[nb_tracks]; -- tracks this track refers to */
+} AVTrackReferences;
+
enum AVPacketSideDataType {
AV_PKT_DATA_PALETTE,

@@ -1525,7 +1532,15 @@ enum AVPacketSideDataType {
* should be associated with a video stream and containts data in the form
* of the AVMasteringDisplayMetadata struct.
*/
- AV_PKT_DATA_MASTERING_DISPLAY_METADATA
+ AV_PKT_DATA_MASTERING_DISPLAY_METADATA,
+
+ /**
+ * Define track references (in particular applicaple for ISO MP4
+ * files). The data is a sequence of type AVTrackReferences
+ * (including the track list that follows it), for as long as
+ * indicated by the key's length.
+ */
+ AV_PKT_DATA_TRACK_REFERENCES
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index c63fdc4..072e660 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2631,14 +2631,30 @@ static int mov_write_edts_tag(AVIOContext *pb, MOVMuxContext *mov,
return size;
}

-static int mov_write_tref_tag(AVIOContext *pb, MOVTrack *track)
+static int mov_write_tref_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track)
{
- avio_wb32(pb, 20); // size
+ int64_t pos = avio_tell(pb);
+ int64_t pos_sub;
+ int i;
+ int tref_idx;
+ avio_wb32(pb, 0); // size
ffio_wfourcc(pb, "tref");
- avio_wb32(pb, 12); // size (subatom)
- avio_wl32(pb, track->tref_tag);
- avio_wb32(pb, track->tref_id);
- return 20;
+ for (tref_idx = 0; tref_idx < track->nb_trefs; tref_idx++) {
+ pos_sub = avio_tell(pb);
+ avio_wb32(pb, 0); // size (subatom)
+ avio_wl32(pb, track->trefs[tref_idx].tag);
+ for (i = 0; i < track->trefs[tref_idx].nb_ids; i++) {
+ int stream_idx;
+ int tref_stream_id = track->trefs[tref_idx].ids[i];
+ for (stream_idx = 0; stream_idx < mov->nb_streams; ++stream_idx)
+ if (mov->tracks[stream_idx].st->id == tref_stream_id) {
+ avio_wb32(pb, mov->tracks[stream_idx].track_id);
+ break;
+ }
+ }
+ update_size(pb, pos_sub);
+ }
+ return update_size(pb, pos);
}

// goes at the end of each track! ... Critical for PSP playback ("Incompatible data" without it)
@@ -2666,7 +2682,8 @@ static int mov_write_udta_sdp(AVIOContext *pb, MOVTrack *track)
char buf[1000] = "";
int len;

- ff_sdp_write_media(buf, sizeof(buf), ctx->streams[0], track->src_track,
+ ff_sdp_write_media(buf, sizeof(buf), ctx->streams[0],
+ track->nb_src_tracks ? track->src_tracks[0] : 0,
NULL, NULL, 0, 0, ctx);
av_strlcatf(buf, sizeof(buf), "a=control:streamid=%d\r\n", track->track_id);
len = strlen(buf);
@@ -2749,8 +2766,8 @@ static int mov_write_trak_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
"Not writing any edit list even though one would have been required\n");
}

- if (track->tref_tag)
- mov_write_tref_tag(pb, track);
+ if (track->nb_trefs)
+ mov_write_tref_tag(pb, mov, track);

if ((ret = mov_write_mdia_tag(s, pb, mov, track)) < 0)
return ret;
@@ -3480,17 +3497,139 @@ static int mov_setup_track_ids(MOVMuxContext *mov, AVFormatContext *s)
return 0;
}

+MOVTRef *ff_mov_find_tref(MOVTrack *track, uint32_t tag)
+{
+ int i;
+ MOVTRef *tref = NULL;
+
+ for (i = 0; i < track->nb_trefs && !tref; ++i) {
+ if (track->trefs[i].tag == tag) {
+ tref = track->trefs + i;
+ }
+ }
+
+ return tref;
+}
+
+int ff_mov_find_or_add_tref(MOVTrack *track, uint32_t tag, MOVTRef **tref_ret)
+{
+ int ret;
+ int i;
+ MOVTRef *tref = ff_mov_find_tref(track, tag);
+ *tref_ret = NULL;
+
+ for (i = 0; i < track->nb_trefs && !tref; ++i) {
+ if (track->trefs[i].tag == tag) {
+ tref = track->trefs + i;
+ }
+ }
+
+ if (!tref) {
+ ret = av_reallocp_array(&track->trefs, track->nb_trefs + 1, sizeof(*track->trefs));
+ if (ret < 0)
+ return ret;
+ tref = track->trefs + track->nb_trefs;
+ track->nb_trefs++;
+ tref->tag = tag;
+ tref->ids = NULL;
+ tref->nb_ids = 0;
+ }
+
+ *tref_ret = tref;
+ return 0;
+}
+
+int ff_mov_add_tref_track(MOVTRef *tref, int id)
+{
+ int ret = av_reallocp_array(&tref->ids, tref->nb_ids + 1, sizeof(tref->ids));
+ if (ret >= 0) {
+ tref->ids[tref->nb_ids] = id;
+ tref->nb_ids++;
+ }
+ return ret;
+}
+
+// replaces all tref ids with ones that refer to the track's src_tracks
+int ff_mov_map_trefs_from_src_tracks(MOVMuxContext *mov, MOVTrack *track, MOVTRef *tref)
+{
+ int ret;
+ int i;
+
+ ret = av_reallocp_array(&tref->ids, track->nb_src_tracks, sizeof(tref->ids));
+ if (ret < 0) {
+ // so we may have added in the tag, but it has no ids.. but we
+ // probably cannot undo it either, because we couldn't
+ // allocate memory.
+ return ret;
+ }
+ tref->nb_ids = track->nb_src_tracks;
+
+ for (i = 0; i < track->nb_src_tracks; ++i) {
+ tref->ids[i] = mov->tracks[track->src_tracks[i]].st->id;
+ }
+ return ret;
+}
+
+/** sets a single tref id */
+static int mov_set_one_tref_track(MOVTRef *tref, int id)
+{
+ int ret;
+ ret = av_reallocp_array(&tref->ids, 1, sizeof(*tref->ids));
+ if (ret < 0)
+ return ret;
+ tref->nb_ids = 1;
+
+ tref->ids[0] = id;
+ return ret;
+}
+
+static int mov_copy_tref_side_data(MOVMuxContext *mov, MOVTrack *track, AVFormatContext *s)
+{
+ int size;
+ int ret = 0;
+ int i;
+ char *ptr = (void*) av_stream_get_side_data(track->st,
+ AV_PKT_DATA_TRACK_REFERENCES,
+ &size);
+
+ if (!ptr)
+ return 0;
+
+ while (ret == 0 && size > 0) {
+ AVTrackReferences *refs = (void*) ptr;
+ MOVTRef *tref;
+ int cur_size;
+ int* stream_ids = (void*) (refs + 1);
+ if (ff_mov_find_or_add_tref(track, MKTAG(refs->tag[0], refs->tag[1], refs->tag[2], refs->tag[3]),
+ &tref) < 0)
+ ret = -1;
+
+ for (i = 0; i < refs->nb_tracks; i++)
+ ff_mov_add_tref_track(tref, stream_ids[i]);
+
+ cur_size = sizeof(*refs) + refs->nb_tracks * sizeof(*stream_ids);
+ ptr += cur_size;
+ size -= cur_size;
+ }
+ return ret;
+}
+
static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
AVFormatContext *s)
{
int i;
int64_t pos = avio_tell(pb);
+ int ret;
avio_wb32(pb, 0); /* size placeholder*/
ffio_wfourcc(pb, "moov");

mov_setup_track_ids(mov, s);

for (i = 0; i < mov->nb_streams; i++) {
+ mov_copy_tref_side_data(mov, &mov->tracks[i], s);
+ }
+
+ for (i = 0; i < mov->nb_streams; i++) {
if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
continue;

@@ -3502,14 +3641,27 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,

if (mov->chapter_track)
for (i = 0; i < s->nb_streams; i++) {
- mov->tracks[i].tref_tag = MKTAG('c','h','a','p');
- mov->tracks[i].tref_id = mov->tracks[mov->chapter_track].track_id;
+ if (!ff_codec_get_id(ff_codec_metadata_tags, mov->tracks[i].tag)) {
+ MOVTrack *track = &mov->tracks[i];
+ MOVTRef *tref = NULL;
+ ret = ff_mov_find_or_add_tref(track, MKTAG('c','h','a','p'), &tref);
+ if (ret < 0)
+ return ret;
+ mov_set_one_tref_track(tref, mov->chapter_track);
+ track->nb_src_tracks = 1; /* this seems an odd hardcoded number.. */
+ }
}
for (i = 0; i < mov->nb_streams; i++) {
MOVTrack *track = &mov->tracks[i];
if (track->tag == MKTAG('r','t','p',' ')) {
- track->tref_tag = MKTAG('h','i','n','t');
- track->tref_id = mov->tracks[track->src_track].track_id;
+ MOVTrack *track = &mov->tracks[i];
+ MOVTRef *tref;
+ ret = ff_mov_find_or_add_tref(track, MKTAG('h','i','n','t'), &tref);
+ if (ret < 0)
+ return ret;
+ ret = ff_mov_map_trefs_from_src_tracks(mov, track, tref);
+ if (ret < 0)
+ return ret;
} else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) {
int * fallback, size;
fallback = (int*)av_stream_get_side_data(track->st,
@@ -3517,21 +3669,55 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
&size);
if (fallback != NULL && size == sizeof(int)) {
if (*fallback >= 0 && *fallback < mov->nb_streams) {
- track->tref_tag = MKTAG('f','a','l','l');
- track->tref_id = mov->tracks[*fallback].track_id;
+ MOVTRef *tref;
+ ret = ff_mov_find_or_add_tref(track, MKTAG('f','a','l','l'), &tref);
+ if (ret < 0)
+ return ret;
+ ret = mov_set_one_tref_track(tref, *fallback);
+ if (ret < 0)
+ return ret;
}
}
}
}
for (i = 0; i < mov->nb_streams; i++) {
- if (mov->tracks[i].tag == MKTAG('t','m','c','d')) {
- int src_trk = mov->tracks[i].src_track;
- mov->tracks[src_trk].tref_tag = mov->tracks[i].tag;
- mov->tracks[src_trk].tref_id = mov->tracks[i].track_id;
- //src_trk may have a different timescale than the tmcd track
- mov->tracks[i].track_duration = av_rescale(mov->tracks[src_trk].track_duration,
- mov->tracks[i].timescale,
- mov->tracks[src_trk].timescale);
+ MOVTrack *track = &mov->tracks[i];
+ if (ff_codec_get_id(ff_codec_metadata_tags, track->tag) &&
+ track->nb_src_tracks) {
+ MOVTRef *tref;
+ ret = ff_mov_find_or_add_tref(track, MKTAG('c','d','s','c'), &tref);
+ if (ret < 0)
+ return ret;
+ ret = ff_mov_map_trefs_from_src_tracks(mov, track, tref);
+ if (ret < 0)
+ return ret;
+ }
+ }
+
+ for (i = 0; i < mov->nb_streams; i++) {
+ MOVTRef *tref = ff_mov_find_tref(&mov->tracks[i], MKTAG('t','m','c','d'));
+ if (tref) {
+ /* This fragment only works if there is one source track */
+ if (mov->tracks[i].nb_src_tracks > 1) {
+ return -1;
+ } else {
+ int src_trk_idx = mov->tracks[i].src_tracks[0];
+ MOVTrack *src_trk = &mov->tracks[src_trk_idx];
+ MOVTRef *src_tref = NULL;
+
+ ret = ff_mov_find_or_add_tref(src_trk, MKTAG('t','m','c','d'), &src_tref);
+ if (ret < 0)
+ return ret;
+
+ ret = mov_set_one_tref_track(src_tref, mov->tracks[i].st->id);
+ if (ret < 0)
+ return ret;
+
+ //src_trk may have a different timescale than the tmcd track
+ mov->tracks[i].track_duration = av_rescale(src_trk->track_duration,
+ mov->tracks[i].timescale,
+ src_trk->timescale);
+ }
}
}

@@ -5239,7 +5425,11 @@ static int mov_create_timecode_track(AVFormatContext *s, int index, int src_inde
/* tmcd track based on video stream */
track->mode = mov->mode;
track->tag = MKTAG('t','m','c','d');
- track->src_track = src_index;
+ ret = av_reallocp_array(&track->src_tracks, 1, sizeof(*track->src_tracks));
+ if (ret < 0)
+ return ret;
+ track->nb_src_tracks = 1;
+ track->src_tracks[0] = src_index;
track->timescale = mov->tracks[src_index].timescale;
if (tc.flags & AV_TIMECODE_FLAG_DROPFRAME)
track->timecode_flags |= MOV_TIMECODE_FLAG_DROPFRAME;
@@ -5321,7 +5511,7 @@ static void enable_tracks(AVFormatContext *s)
static void mov_free(AVFormatContext *s)
{
MOVMuxContext *mov = s->priv_data;
- int i;
+ int i, j;

if (mov->chapter_track) {
if (mov->tracks[mov->chapter_track].par)
@@ -5341,6 +5531,11 @@ static void mov_free(AVFormatContext *s)
av_freep(&mov->tracks[i].vos_data);

ff_mov_cenc_free(&mov->tracks[i].cenc);
+ av_freep(&mov->tracks[i].src_tracks);
+ for (j = 0; j < mov->tracks[i].nb_trefs; j++) {
+ av_freep(&mov->tracks[i].trefs[j].ids);
+ }
+ av_freep(&mov->tracks[i].trefs);
}

av_freep(&mov->tracks);
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index ea76e39..5bf9469 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -78,6 +78,12 @@ typedef struct MOVFragmentInfo {
int size;
} MOVFragmentInfo;

+typedef struct MOVTRef {
+ uint32_t tag;
+ int nb_ids;
+ int* ids;
+} MOVTRef;
+
typedef struct MOVTrack {
int mode;
int entry;
@@ -110,15 +116,16 @@ typedef struct MOVTrack {
unsigned cluster_capacity;
int audio_vbr;
int height; ///< active picture (w/o VBI) height for D-10/IMX
- uint32_t tref_tag;
- int tref_id; ///< trackID of the referenced track
+ MOVTRef* trefs;
+ int nb_trefs;
int64_t start_dts;
int64_t start_cts;
int64_t end_pts;
int end_reliable;

int hint_track; ///< the track that hints this track, -1 if no hint track is set
- int src_track; ///< the track that this hint (or tmcd) track describes
+ int nb_src_tracks; ///< number of src tracks
+ int *src_tracks; ///< the tracks that this hint (or tmcd or cdsc) track describes
AVFormatContext *rtp_ctx; ///< the format context for the hinting rtp muxer
uint32_t prev_rtp_ts;
int64_t cur_rtp_ts_unwrapped;
@@ -248,4 +255,52 @@ int ff_mov_add_hinted_packet(AVFormatContext *s, AVPacket *pkt,
uint8_t *sample_data, int sample_size);
void ff_mov_close_hinting(MOVTrack *track);

+/**
+ * @brief Finds a track reference of certain tag from a track; if not
+ * found, return NULL
+ *
+ * @param track The track to search from
+ * @param tag The tag (4cc) to search for
+ * @return a MOVTRef describing the track reference or NULL if not found.
+ */
+MOVTRef *ff_mov_find_tref(MOVTrack *track, uint32_t tag);
+
+/**
+ * @brief Finds a track reference of certain tag from a track; if not
+ * found, create an empty track reference object for the track put it
+ * into the track.
+ *
+ * @param track The track to search from
+ * @param tag The tag (4cc) to search for
+ * @param tref_ret A pointer to the the found or created tref is
+ * returned here. Must not be NULL.
+ * @return Zero on success, an AVERROR error code on failure. On error
+ * no modifications have been performed.
+ */
+int ff_mov_find_or_add_tref(MOVTrack *track, uint32_t tag, MOVTRef **tref_ret);
+
+/**
+ * @brief Adds a a track to a track reference that may or may not be
+ * put into a track
+ *
+ * @param tref The track reference to add the track for
+ * @param tag The tag (4cc) of the track reference
+ * @return Zero on success, an AVERROR error code on failure. On error
+ * no modifications have been performed.
+ */
+int ff_mov_add_tref_track(MOVTRef *tref, int id);
+
+/**
+ * @brief Copies track->src_tracks from a track to the give tref
+ *
+ * @param mov The MOV muxer context
+ * @param track The track to take the src_tracks from
+ * @param tref The tref to translate the src_tracks into
+ * @return Zero on success, an AVERROR error code on failure. On error
+ * no modifications have been performed. However, probably
+ * prior to this call src_tracks has been manipulated and now
+ * it is out-of-sync, so this is pretty fatal.
+ */
+int ff_mov_map_trefs_from_src_tracks(MOVMuxContext *mov, MOVTrack *track, MOVTRef *tref);
+
#endif /* AVFORMAT_MOVENC_H */
diff --git a/libavformat/movenchint.c b/libavformat/movenchint.c
index 964026e..61ad09b 100644
--- a/libavformat/movenchint.c
+++ b/libavformat/movenchint.c
@@ -32,10 +32,17 @@ int ff_mov_init_hinting(AVFormatContext *s, int index, int src_index)
MOVTrack *track = &mov->tracks[index];
MOVTrack *src_track = &mov->tracks[src_index];
AVStream *src_st = s->streams[src_index];
- int ret = AVERROR(ENOMEM);
+ int ret;

track->tag = MKTAG('r','t','p',' ');
- track->src_track = src_index;
+ ret = av_reallocp_array(&track->src_tracks,
+ track->nb_src_tracks + 1,
+ sizeof(*track->src_tracks));
+ if (ret != 0)
+ goto fail;
+ ret = AVERROR(ENOMEM); /* default error for the rest of the errors */
+ track->src_tracks[track->nb_src_tracks] = src_index;
+ track->nb_src_tracks++;

track->par = avcodec_parameters_alloc();
if (!track->par)
--
2.7.4
Erkki Seppälä
2016-08-25 11:04:54 UTC
Permalink
Hello,

However long this patch is, it is also mostly useless: the standard does
NOT support multiple distinct tref tags. Instead it supports multiple
references for one or zero trefs, which is what FFmpeg already supports.

I will introduce a patch that instead just adds the ability to set the
track reference type and add track references. The side packet API does
not need to change as far as I can see.
Post by e***@nokia.com
Instead of one track reference, allow multiple. In addition, allow
client to explicitly add track references with side packet
AV_PKG_DATA_TRACK_REFERENCES containing AVTrackReferences. MOVTrack's
track references can be manipulated with helper functions
ff_mov_*tref*.
Multiple track references can be useful in particular with timed meta
data tracks, indicating the track is related to multiple other tracks.
This information ends up in ISO media file box 'tref' as specified by
ISO/IEC 14496-12.
Erkki Seppälä
2016-08-25 11:30:02 UTC
Permalink
..though on more precise look it FFmpeg doesn't in fact implement
multiple track references at all (MOVTrack has src_track while the patch
introduces src_tracks). But the patch can be greatly simplified regardless.
Erkki Seppälä
2016-08-31 11:35:46 UTC
Permalink
It has the codec id AV_CODEC_ID_META and type of AVMEDIA_TYPE_DATA.

This codec basically passes the data forward and is used for referring
timed meta data tracks by a codec. It is useful for dealing with the
metadata in a similar way as other kinds of codecs.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
configure | 4 +--
libavcodec/Makefile | 2 +-
libavcodec/allcodecs.c | 3 ++
libavcodec/avcodec.h | 2 +-
libavcodec/codec_desc.c | 8 +++++
libavcodec/metacodec.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 110 insertions(+), 4 deletions(-)
create mode 100644 libavcodec/metacodec.c

diff --git a/configure b/configure
index 9b92426..ded5452 100755
--- a/configure
+++ b/configure
@@ -2848,11 +2848,11 @@ matroska_demuxer_select="iso_media riffdec"
matroska_demuxer_suggest="bzlib lzo zlib"
matroska_muxer_select="iso_media riffenc"
mmf_muxer_select="riffenc"
-mov_demuxer_select="iso_media riffdec"
+mov_demuxer_select="iso_media riffdec meta_decoder"
mov_demuxer_suggest="zlib"
mov_muxer_select="iso_media riffenc rtpenc_chain"
mp3_demuxer_select="mpegaudio_parser"
-mp4_muxer_select="mov_muxer"
+mp4_muxer_select="mov_muxer meta_encoder"
mpegts_demuxer_select="iso_media"
mpegts_muxer_select="adts_muxer latm_muxer"
mpegtsraw_demuxer_select="mpegts_demuxer"
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index a6e79ce..98a7a8d 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -799,7 +799,7 @@ OBJS-$(CONFIG_VP9_DXVA2_HWACCEL) += dxva2_vp9.o
OBJS-$(CONFIG_VP9_VAAPI_HWACCEL) += vaapi_vp9.o

# libavformat dependencies
-OBJS-$(CONFIG_ISO_MEDIA) += mpeg4audio.o mpegaudiodata.o
+OBJS-$(CONFIG_ISO_MEDIA) += mpeg4audio.o mpegaudiodata.o metacodec.o

OBJS-$(CONFIG_ADTS_MUXER) += mpeg4audio.o
OBJS-$(CONFIG_CAF_DEMUXER) += ac3tab.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 4c6b94e..30d0243 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -687,4 +687,7 @@ void avcodec_register_all(void)
REGISTER_PARSER(VP3, vp3);
REGISTER_PARSER(VP8, vp8);
REGISTER_PARSER(VP9, vp9);
+
+ /* data, meta data */
+ REGISTER_ENCDEC(META, meta);
}
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 6ac6646..0d0447e6 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -639,7 +639,7 @@ enum AVCodecID {
AV_CODEC_ID_DVD_NAV,
AV_CODEC_ID_TIMED_ID3,
AV_CODEC_ID_BIN_DATA,
-
+ AV_CODEC_ID_META,

AV_CODEC_ID_PROBE = 0x19000, ///< codec_id is not known (like AV_CODEC_ID_NONE) but lavf should attempt to identify it

diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 24948ca..e85b51d 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -2965,6 +2965,14 @@ static const AVCodecDescriptor codec_descriptors[] = {
.mime_types= MT("application/octet-stream"),
},

+ {
+ .id = AV_CODEC_ID_META,
+ .type = AVMEDIA_TYPE_DATA,
+ .name = "meta_data",
+ .long_name = NULL_IF_CONFIG_SMALL("binary data"),
+ .mime_types= MT("application/octet-stream"),
+ },
+
/* deprecated codec ids */
};

diff --git a/libavcodec/metacodec.c b/libavcodec/metacodec.c
new file mode 100644
index 0000000..bd8013c
--- /dev/null
+++ b/libavcodec/metacodec.c
@@ -0,0 +1,95 @@
+/*
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/intreadwrite.h"
+#include "internal.h"
+#include "libavutil/opt.h"
+#include "libavformat/avio.h"
+
+static int meta_encode(AVCodecContext *avctx, AVPacket *avpkt, const AVFrame *frame,
+ int *got_packet_ptr)
+{
+ *got_packet_ptr = 0;
+
+ av_packet_unref(avpkt);
+ if (frame->buf) {
+ avpkt->buf = av_buffer_ref(frame->buf[0]);
+ if (!avpkt->buf)
+ return AVERROR(ENOMEM);
+ avpkt->data = frame->data[0];
+ } else {
+ avpkt->buf = av_buffer_alloc(frame->nb_samples);
+ if (!avpkt->buf)
+ return AVERROR(ENOMEM);
+ avpkt->data = avpkt->buf;
+ memcpy(avpkt->data, frame->data[0], frame->nb_samples);
+ }
+ avpkt->size = frame->nb_samples;
+ avpkt->pts = frame->pts;
+ avpkt->dts = frame->pts;
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+static int meta_decode(AVCodecContext *avctx, void *data, int *got_packet_ptr, AVPacket *avpkt)
+{
+ AVFrame *metadata = data;
+
+ *got_packet_ptr = 0;
+
+ av_frame_unref(metadata);
+ if (avpkt->buf) {
+ metadata->buf[0] = av_buffer_ref(avpkt->buf);
+ if (!metadata->buf[0])
+ return AVERROR(ENOMEM);
+ metadata->buf[0]->size = avpkt->size;
+ } else {
+ metadata->buf[0] = av_buffer_alloc(avpkt->size);
+ if (!metadata->buf[0])
+ return AVERROR(ENOMEM);
+ metadata->data[0] = metadata->buf[0]->data;
+ memcpy(metadata->data[0], avpkt->data, avpkt->size);
+ }
+
+ metadata->nb_samples = avpkt->size;
+ metadata->pts = avpkt->pts;
+ metadata->pkt_pts = avpkt->pts;
+ metadata->pkt_dts = avpkt->dts;
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+AVCodec ff_meta_encoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Metadata Encoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .encode2 = meta_encode,
+};
+
+AVCodec ff_meta_decoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Metadata Decoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .decode = meta_decode,
+};
--
2.7.4
Carl Eugen Hoyos
2016-08-31 13:53:24 UTC
Permalink
Hi!
Post by e***@nokia.com
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 6ac6646..0d0447e6 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -639,7 +639,7 @@ enum AVCodecID {
AV_CODEC_ID_DVD_NAV,
AV_CODEC_ID_TIMED_ID3,
AV_CODEC_ID_BIN_DATA,
-
+ AV_CODEC_ID_META,
Please do not remove the empty line.
Post by e***@nokia.com
diff --git a/libavcodec/metacodec.c b/libavcodec/metacodec.c
new file mode 100644
index 0000000..bd8013c
--- /dev/null
+++ b/libavcodec/metacodec.c
@@ -0,0 +1,95 @@
+/*
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/intreadwrite.h"
+#include "internal.h"
+#include "libavutil/opt.h"
+#include "libavformat/avio.h"
+
+static int meta_encode(AVCodecContext *avctx, AVPacket *avpkt, const AVFrame *frame,
+ int *got_packet_ptr)
+{
+ *got_packet_ptr = 0;
+
+ av_packet_unref(avpkt);
+ if (frame->buf) {
+ avpkt->buf = av_buffer_ref(frame->buf[0]);
+ if (!avpkt->buf)
+ return AVERROR(ENOMEM);
+ avpkt->data = frame->data[0];
+ } else {
+ avpkt->buf = av_buffer_alloc(frame->nb_samples);
+ if (!avpkt->buf)
+ return AVERROR(ENOMEM);
+ avpkt->data = avpkt->buf;
+ memcpy(avpkt->data, frame->data[0], frame->nb_samples);
+ }
+ avpkt->size = frame->nb_samples;
+ avpkt->pts = frame->pts;
+ avpkt->dts = frame->pts;
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+static int meta_decode(AVCodecContext *avctx, void *data, int *got_packet_ptr, AVPacket *avpkt)
+{
+ AVFrame *metadata = data;
+
+ *got_packet_ptr = 0;
+
+ av_frame_unref(metadata);
+ if (avpkt->buf) {
+ metadata->buf[0] = av_buffer_ref(avpkt->buf);
+ if (!metadata->buf[0])
+ return AVERROR(ENOMEM);
+ metadata->buf[0]->size = avpkt->size;
+ } else {
+ metadata->buf[0] = av_buffer_alloc(avpkt->size);
+ if (!metadata->buf[0])
+ return AVERROR(ENOMEM);
+ metadata->data[0] = metadata->buf[0]->data;
+ memcpy(metadata->data[0], avpkt->data, avpkt->size);
+ }
+
+ metadata->nb_samples = avpkt->size;
+ metadata->pts = avpkt->pts;
+ metadata->pkt_pts = avpkt->pts;
+ metadata->pkt_dts = avpkt->dts;
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+AVCodec ff_meta_encoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Metadata Encoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .encode2 = meta_encode,
+};
+
+AVCodec ff_meta_decoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Metadata Decoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .decode = meta_decode,
+};
You either have to add some "#if"'s around the code - configure allows to
enable / disable the encoders and decoders separately - or split the file.
Since the encoder and decoder do not share code, this seems simpler,

Carl Eugen
Michael Niedermayer
2016-09-01 17:01:18 UTC
Permalink
Post by Erkki Seppälä
It has the codec id AV_CODEC_ID_META and type of AVMEDIA_TYPE_DATA.
This codec basically passes the data forward and is used for referring
timed meta data tracks by a codec. It is useful for dealing with the
metadata in a similar way as other kinds of codecs.
---
configure | 4 +--
libavcodec/Makefile | 2 +-
libavcodec/allcodecs.c | 3 ++
libavcodec/avcodec.h | 2 +-
libavcodec/codec_desc.c | 8 +++++
libavcodec/metacodec.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 110 insertions(+), 4 deletions(-)
create mode 100644 libavcodec/metacodec.c
[...]
Post by Erkki Seppälä
diff --git a/libavcodec/metacodec.c b/libavcodec/metacodec.c
new file mode 100644
index 0000000..bd8013c
--- /dev/null
+++ b/libavcodec/metacodec.c
this causes a compiler warning:
libavcodec/metacodec.c: In function ‘meta_encode’:
libavcodec/metacodec.c:41:21: warning: assignment from incompatible pointer type [enabled by default]

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When the tyrant has disposed of foreign enemies by conquest or treaty, and
there is nothing more to fear from them, then he is always stirring up
some war or other, in order that the people may require a leader. -- Plato
Erkki Seppälä
2016-09-02 07:12:21 UTC
Permalink
Post by Michael Niedermayer
libavcodec/metacodec.c:41:21: warning: assignment from incompatible pointer type [enabled by default]
avpkt->buf = av_buffer_alloc(frame->nb_samples);
if (!avpkt->buf)
return AVERROR(ENOMEM);
avpkt->data = avpkt->buf; /* warning */
memcpy(avpkt->data, frame->data[0], frame->nb_samples);
Oops. Fixed. I now understand why FFmpeg prefers uint8_t* over void*..
Erkki Seppälä
2016-08-31 11:35:44 UTC
Permalink
Sometimes it's useful to be able to define the exact track numbers in
the generated track, instead of always beginning at track id 1. Using
the option use_stream_ids_as_track_ids now copies the use stream ids
to track ids. Dynamically generated tracks (ie. tmcd) have their track
numbers defined as continuing from the highest numbered stream id.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/movenc.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++------
libavformat/movenc.h | 2 ++
2 files changed, 57 insertions(+), 6 deletions(-)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 1f55333..525d103 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -89,6 +89,7 @@ static const AVOption options[] = {
{ "encryption_scheme", "Configures the encryption scheme, allowed values are none, cenc-aes-ctr", offsetof(MOVMuxContext, encryption_scheme_str), AV_OPT_TYPE_STRING, {.str = NULL}, .flags = AV_OPT_FLAG_ENCODING_PARAM },
{ "encryption_key", "The media encryption key (hex)", offsetof(MOVMuxContext, encryption_key), AV_OPT_TYPE_BINARY, .flags = AV_OPT_FLAG_ENCODING_PARAM },
{ "encryption_kid", "The media encryption key identifier (hex)", offsetof(MOVMuxContext, encryption_kid), AV_OPT_TYPE_BINARY, .flags = AV_OPT_FLAG_ENCODING_PARAM },
+ { "use_stream_ids_as_track_ids", "use stream ids as track ids", offsetof(MOVMuxContext, use_stream_ids_as_track_ids), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, AV_OPT_FLAG_ENCODING_PARAM},
{ NULL },
};

@@ -3435,6 +3436,52 @@ static void build_chunks(MOVTrack *trk)
}
}

+/**
+ * Assign track ids. If option "use_stream_ids_as_track_ids" is set,
+ * the stream ids are used as track ids.
+ *
+ * This assumes mov->tracks and s->streams are in the same order and
+ * there are no gaps in either of them (so mov->tracks[n] refers to
+ * s->streams[n]).
+ *
+ * As an exception, there can be more entries in
+ * s->streams than in mov->tracks, in which case new track ids are
+ * generated (starting after the largest found stream id).
+ */
+static int mov_setup_track_ids(MOVMuxContext *mov, AVFormatContext *s)
+{
+ int i;
+
+ if (mov->track_ids_ok)
+ return 0;
+
+ if (mov->use_stream_ids_as_track_ids) {
+ int next_generated_track_id = 0;
+ for (i = 0; i < s->nb_streams; i++) {
+ if (s->streams[i]->id > next_generated_track_id)
+ next_generated_track_id = s->streams[i]->id;
+ }
+
+ for (i = 0; i < mov->nb_streams; i++) {
+ if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
+ continue;
+
+ mov->tracks[i].track_id = i >= s->nb_streams ? ++next_generated_track_id : s->streams[i]->id;
+ }
+ } else {
+ for (i = 0; i < mov->nb_streams; i++) {
+ if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
+ continue;
+
+ mov->tracks[i].track_id = i + 1;
+ }
+ }
+
+ mov->track_ids_ok = 1;
+
+ return 0;
+}
+
static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
AVFormatContext *s)
{
@@ -3443,12 +3490,13 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
avio_wb32(pb, 0); /* size placeholder*/
ffio_wfourcc(pb, "moov");

+ mov_setup_track_ids(mov, s);
+
for (i = 0; i < mov->nb_streams; i++) {
if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
continue;

mov->tracks[i].time = mov->time;
- mov->tracks[i].track_id = i + 1;

if (mov->tracks[i].entry)
build_chunks(&mov->tracks[i]);
@@ -3529,7 +3577,7 @@ static void param_write_hex(AVIOContext *pb, const char *name, const uint8_t *va
avio_printf(pb, "<param name=\"%s\" value=\"%s\" valuetype=\"data\"/>\n", name, buf);
}

-static int mov_write_isml_manifest(AVIOContext *pb, MOVMuxContext *mov)
+static int mov_write_isml_manifest(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s)
{
int64_t pos = avio_tell(pb);
int i;
@@ -3552,12 +3600,13 @@ static int mov_write_isml_manifest(AVIOContext *pb, MOVMuxContext *mov)
avio_printf(pb, "</head>\n");
avio_printf(pb, "<body>\n");
avio_printf(pb, "<switch>\n");
+
+ mov_setup_track_ids(mov, s);
+
for (i = 0; i < mov->nb_streams; i++) {
MOVTrack *track = &mov->tracks[i];
const char *type;
- /* track->track_id is initialized in write_moov, and thus isn't known
- * here yet */
- int track_id = i + 1;
+ int track_id = track->track_id;

if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) {
type = "video";
@@ -5773,7 +5822,7 @@ static int mov_write_header(AVFormatContext *s)
avio_flush(pb);

if (mov->flags & FF_MOV_FLAG_ISML)
- mov_write_isml_manifest(pb, mov);
+ mov_write_isml_manifest(pb, mov, s);

if (mov->flags & FF_MOV_FLAG_EMPTY_MOOV &&
!(mov->flags & FF_MOV_FLAG_DELAY_MOOV)) {
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index 894a1b0..ea76e39 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -217,6 +217,8 @@ typedef struct MOVMuxContext {

int need_rewrite_extradata;

+ int use_stream_ids_as_track_ids;
+ int track_ids_ok;
} MOVMuxContext;

#define FF_MOV_FLAG_RTP_HINT (1 << 0)
--
2.7.4
Michael Niedermayer
2016-09-01 13:29:16 UTC
Permalink
Post by e***@nokia.com
Sometimes it's useful to be able to define the exact track numbers in
the generated track, instead of always beginning at track id 1. Using
the option use_stream_ids_as_track_ids now copies the use stream ids
to track ids. Dynamically generated tracks (ie. tmcd) have their track
numbers defined as continuing from the highest numbered stream id.
---
libavformat/movenc.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++------
libavformat/movenc.h | 2 ++
2 files changed, 57 insertions(+), 6 deletions(-)
applied

thx

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

What does censorship reveal? It reveals fear. -- Julian Assange
Erkki Seppälä
2016-08-31 11:35:45 UTC
Permalink
when the option "brand" is used. This allows custom brands to end up in
the compatible brands as well.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/movenc.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 525d103..571c2a7 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -4210,6 +4210,8 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s)

avio_wb32(pb, minor);

+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a compatible brand */
if (mov->mode == MODE_MOV)
ffio_wfourcc(pb, "qt ");
else if (mov->mode == MODE_ISM) {
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:58 UTC
Permalink
It does it only when the chnl box is to be written. The specification
for the chnl box (ISO 14496-12) requires proper number of channels to be
written here is it is used, and without the proper number of channels
available it becomes tricky to parse the data, as the number of channels
in this box comes from that value.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/movenc.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 6e179ef..bbd176f 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1031,7 +1031,13 @@ static int mov_write_audio_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex
avio_wb16(pb, 16);
avio_wb16(pb, track->audio_vbr ? -2 : 0); /* compression ID */
} else { /* reserved for mp4/3gp */
- avio_wb16(pb, 2);
+ /* parsing chln box requires the proper number of channels having been
+ written into the audio header */
+ if (av_stream_get_side_data(track->st, AV_PKT_DATA_AUDIO_TRACK_CHANNEL_LAYOUT,
+ NULL))
+ avio_wb16(pb, track->par->channels);
+ else
+ avio_wb16(pb, 2);
avio_wb16(pb, 16);
avio_wb16(pb, 0);
}
--
2.7.4
Erkki Seppälä
2016-08-31 11:36:00 UTC
Permalink
Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
doc/examples/Makefile | 1 +
doc/examples/extract_timed_metadata.c | 233 ++++++++++++++++++++++++++++++++++
2 files changed, 234 insertions(+)
create mode 100644 doc/examples/extract_timed_metadata.c

diff --git a/doc/examples/Makefile b/doc/examples/Makefile
index af38159..c10033e 100644
--- a/doc/examples/Makefile
+++ b/doc/examples/Makefile
@@ -16,6 +16,7 @@ EXAMPLES= avio_dir_cmd \
decoding_encoding \
demuxing_decoding \
extract_mvs \
+ extract_timed_metadata \
filtering_video \
filtering_audio \
http_multiclient \
diff --git a/doc/examples/extract_timed_metadata.c b/doc/examples/extract_timed_metadata.c
new file mode 100644
index 0000000..e72bd44
--- /dev/null
+++ b/doc/examples/extract_timed_metadata.c
@@ -0,0 +1,233 @@
+/*
+ * Copyright (c) 2010 Nicolas George
+ * Copyright (c) 2011 Stefano Sabatini
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+/**
+ * @file
+ * API example extracting timed metadata
+ * @example extract_timed_metadata.c
+ */
+
+#define _XOPEN_SOURCE 600 /* for usleep */
+#include <unistd.h>
+#include <stdint.h>
+
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/opt.h>
+#include <assert.h>
+#include <ctype.h>
+
+#define MAX_METADATA_STREAMS 10
+
+static AVFormatContext *fmt_ctx;
+static AVCodecContext *dec_ctx[MAX_METADATA_STREAMS];
+static int metadata_stream_indices[MAX_METADATA_STREAMS + 1] = { -1 }; /* terminated with -1 */
+
+static int open_input_file(const char *filename)
+{
+ int ret;
+ AVCodec *dec = avcodec_find_decoder(AV_CODEC_ID_META);
+ int i;
+ int nb_metadata = 0;
+
+ if ((ret = avformat_open_input(&fmt_ctx, filename, NULL, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot open input file\n");
+ return ret;
+ }
+
+ if ((ret = avformat_find_stream_info(fmt_ctx, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot find stream information\n");
+ return ret;
+ }
+
+ for (i = 0; i < fmt_ctx->nb_streams; i++) {
+ if (fmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_DATA) {
+ assert(nb_metadata < MAX_METADATA_STREAMS);
+ metadata_stream_indices[nb_metadata] = i;
+ nb_metadata++;
+ }
+ }
+ metadata_stream_indices[nb_metadata] = -1;
+
+ if (nb_metadata == 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot find a video stream in the input file\n");
+ return ret;
+ }
+
+ for (i = 0; i < nb_metadata; i++) {
+ dec_ctx[i] = avcodec_alloc_context3(dec);
+ if ((ret = avcodec_open2(dec_ctx[i], dec, NULL)) < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Cannot open metadata decoder for metadata stream on track %d\n", i);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void* memndup(void* ptr, int n)
+{
+ void* buffer = malloc(n);
+ memcpy(buffer, ptr, n);
+ return buffer;
+}
+
+static char* local_strndup(char* str, int n)
+{
+ int len = 0;
+ while (str[len] && len < n) {
+ ++len;
+ }
+ char* buffer = malloc(n + 1);
+ memcpy(buffer, str, n);
+ buffer[len] = 0;
+ return buffer;
+}
+
+int main(int argc, char **argv)
+{
+ int ret;
+ AVPacket packet;
+ AVFrame *metadata = av_frame_alloc();
+ int i;
+
+ if (!metadata) {
+ perror("Could not allocate metadata");
+ exit(1);
+ }
+ if (argc != 2) {
+ fprintf(stderr, "Usage: %s file\n", argv[0]);
+ exit(1);
+ }
+
+ av_register_all();
+
+ if ((ret = open_input_file(argv[1])) < 0)
+ goto end;
+
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ char *uri = NULL;
+ int nb_src_idxs;
+ int *src_idxs = NULL;
+ char *tref_tag = NULL;
+ int size;
+ int trefsSize;
+ AVTimedMetadata* metadata =
+ (AVTimedMetadata*) av_stream_get_side_data(fmt_ctx->streams[metadata_stream_indices[i]],
+ AV_PKT_DATA_TIMED_METADATA_INFO,
+ &size);
+ AVTrackReferences* trefs =
+ (AVTrackReferences*) av_stream_get_side_data(fmt_ctx->streams[metadata_stream_indices[i]],
+ AV_PKT_DATA_TRACK_REFERENCES,
+ &trefsSize);
+ if (!trefs)
+ printf("No track references!\n");
+ else if (!metadata)
+ printf("No metadata info!\n");
+ else if (metadata) {
+ tref_tag = local_strndup(metadata->meta_tag, 4);
+ uri = local_strndup((char*) (metadata + 1), metadata->meta_length);
+ int j;
+ int conf_len = metadata->conf_length;
+ void* conf = memndup((char*) (metadata + 1) + metadata->meta_length, metadata->conf_length);
+ char* conf_tag = local_strndup(metadata->conf_tag, 4);
+ src_idxs = trefs->tracks;
+ printf("Track #%d describing track%s ",
+ metadata_stream_indices[i] + 1,
+ nb_src_idxs > 1 ? "s" : "");
+ for (j = 0; j < trefs->nb_tracks; ++j)
+ printf("#%d ", trefs->tracks[j]);
+ printf("with tref %s",
+ tref_tag);
+ printf(" url: %s \n", uri);
+ printf(" configuration conf tag %s, %d bytes:", conf_tag, conf_len);
+ for (j = 0; j < conf_len; ++j) {
+ unsigned char ch = ((unsigned char*) conf)[j];
+ if (isprint(ch)) {
+ printf(" %c", ch);
+ } else {
+ printf(" %2x", ((unsigned char*) conf)[j]);
+ }
+ }
+ printf("\n");
+ free(conf);
+ free(conf_tag);
+ free(uri);
+ free(tref_tag);
+ } else {
+ printf("No meta data info for track %d?!\n", metadata_stream_indices[i]);
+ }
+ }
+
+ /* read all packets */
+ while (1) {
+ if ((ret = av_read_frame(fmt_ctx, &packet)) < 0)
+ break;
+
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ if (packet.stream_index == metadata_stream_indices[i]) {
+ ret = avcodec_send_packet(dec_ctx[i], &packet);
+
+ // We always empty decoded frames so we don't handle AVERROR(EAGAIN) here
+ if (ret < 0) {
+ av_log(NULL, AV_LOG_ERROR, "Error decoding meta data\n");
+ break;
+ }
+
+ while (avcodec_receive_frame(dec_ctx[i], metadata) == 0) {
+ int c;
+ printf("track #%d at %" PRId64 " (%d bytes), ",
+ packet.stream_index + 1,
+ metadata->pts,
+ metadata->buf[0]->size);
+ for (c = 0; c < metadata->buf[0]->size; ++c) {
+ char ch = ((char*) metadata->buf[0]->data)[c];
+ if (ch == '\\') {
+ printf("\\\\");
+ } else if (isprint(ch)) {
+ putchar(ch);
+ } else {
+ printf("\\0x%2x", (unsigned char) ch);
+ }
+ }
+ putchar('\n');
+ }
+ }
+ }
+ av_packet_unref(&packet);
+ }
+end:
+ for (i = 0; metadata_stream_indices[i] >= 0; i++) {
+ avcodec_close(dec_ctx[i]);
+ }
+ avformat_close_input(&fmt_ctx);
+ av_frame_free(&metadata);
+
+ if (ret < 0 && ret != AVERROR_EOF) {
+ fprintf(stderr, "Error occurred: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ exit(0);
+}
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:50 UTC
Permalink
This involves adding a new tag to the ff_mp4_obj_type table as well as
modifying mp4_get_codec_tag to return 'meta' for AV_CODEC_ID_META.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.c | 1 +
libavformat/movenc.c | 1 +
2 files changed, 2 insertions(+)

diff --git a/libavformat/isom.c b/libavformat/isom.c
index 473700f..9fb96ef 100644
--- a/libavformat/isom.c
+++ b/libavformat/isom.c
@@ -66,6 +66,7 @@ const AVCodecTag ff_mp4_obj_type[] = {
{ AV_CODEC_ID_QCELP , 0xE1 },
{ AV_CODEC_ID_MPEG4SYSTEMS, 0x01 },
{ AV_CODEC_ID_MPEG4SYSTEMS, 0x02 },
+ { AV_CODEC_ID_META , 0x03 },
{ AV_CODEC_ID_NONE , 0 },
};

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 282551b..883fa57 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1175,6 +1175,7 @@ static int mp4_get_codec_tag(AVFormatContext *s, MOVTrack *track)
else if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) tag = MKTAG('m','p','4','v');
else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) tag = MKTAG('m','p','4','a');
else if (track->par->codec_id == AV_CODEC_ID_DVD_SUBTITLE) tag = MKTAG('m','p','4','s');
+ else if (track->par->codec_id == AV_CODEC_ID_META) tag = MKTAG('m','e','t','a');

return tag;
}
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:54 UTC
Permalink
mov_codec_id is now able to set AVMEDIA_TYPE_DATA to the
st->codec->codec_type field when the input is of type
AVMEDIA_TYPE_DATA; previously it used AVMEDIA_TYPE_SUBTITLE as the
value to set in that case.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/mov.c | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/libavformat/mov.c b/libavformat/mov.c
index 97bbdb9..1012b3c 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -1789,12 +1789,18 @@ static int mov_codec_id(AVStream *st, uint32_t format)
id = ff_codec_get_id(ff_codec_bmp_tags, format);
if (id > 0)
st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
- else if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA ||
- (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE &&
- st->codecpar->codec_id == AV_CODEC_ID_NONE)) {
- id = ff_codec_get_id(ff_codec_movsubtitle_tags, format);
- if (id > 0)
- st->codecpar->codec_type = AVMEDIA_TYPE_SUBTITLE;
+ else {
+ if (st->codecpar->codec_type == AVMEDIA_TYPE_DATA ||
+ (st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE &&
+ st->codecpar->codec_id == AV_CODEC_ID_NONE)) {
+ id = ff_codec_get_id(ff_codec_movsubtitle_tags, format);
+ if (id > 0)
+ st->codecpar->codec_type = AVMEDIA_TYPE_SUBTITLE;
+ }
+ if (id <= 0 &&
+ st->codecpar->codec_type == AVMEDIA_TYPE_DATA) {
+ id = ff_codec_get_id(ff_codec_metadata_tags, format);
+ }
}
}
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:49 UTC
Permalink
This also adds libavformat/movmeta that contains the meta data
information. Actually the sample configuration data is performed with
side packet data AV_PKT_DATA_TIMED_METADATA_INFO where the value is
stored inside AVTimedMetadataInfo.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 16 ++++++++++
libavformat/isom.c | 1 +
libavformat/movenc.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++++---
libavformat/movenc.h | 1 +
libavformat/movmeta.h | 46 ++++++++++++++++++++++++++++
5 files changed, 143 insertions(+), 4 deletions(-)
create mode 100644 libavformat/movmeta.h

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index f45cdc2..8373bca 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1349,6 +1349,16 @@ typedef struct AVCPBProperties {
* @{
*/

+typedef struct AVTimedMetadataInfo {
+ char meta_tag[4]; /** 4cc describing this metadata box type */
+ int meta_length; /** length of data for the metadata type information */
+
+ char conf_tag[4]; /** configurationg box type 4cc, ie. 'conf' */
+ int conf_length; /** length of the data for the configuration box */
+
+ /** followed by meta_length bytes of meta data followed by conf_length bytes of conf data */
+} AVTimedMetadataInfo;
+
typedef struct AVTrackReferences {
int next_tref_ofs; /** offset in bytes to the next AVTrackReferences or 0 if this is the last one*/
char tag[4]; /** 4cc used for describing this */
@@ -1543,6 +1553,12 @@ enum AVPacketSideDataType {
* indicated by the key's length.
*/
AV_PKT_DATA_TRACK_REFERENCES,
+
+ /**
+ * Configured the timed metadata parameters, such as the uri and
+ * meta data configuration. The value is of type AVTimedMetadataInfo.
+ */
+ AV_PKT_DATA_TIMED_METADATA_INFO,
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/isom.c b/libavformat/isom.c
index 1a90d00..473700f 100644
--- a/libavformat/isom.c
+++ b/libavformat/isom.c
@@ -357,6 +357,7 @@ const AVCodecTag ff_codec_movsubtitle_tags[] = {

const AVCodecTag ff_codec_metadata_tags[] = {
{ AV_CODEC_ID_META, MKTAG('m', 'e', 't', 'a') },
+ { AV_CODEC_ID_META, MKTAG('u', 'r', 'i', 'm') },
{ AV_CODEC_ID_NONE, 0 },
};

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 6cc315a..282551b 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1552,6 +1552,69 @@ static int mov_write_fiel_tag(AVIOContext *pb, MOVTrack *track, int field_order)
return 10;
}

+static int mov_write_urim_uri_box(AVIOContext *pb, const char *uri, int length)
+{
+ int64_t pos = avio_tell(pb);
+ avio_wb32(pb, 0); /* size */
+ ffio_wfourcc(pb, "uri ");
+ avio_w8(pb, 0); /* version */
+ avio_wb24(pb, 0); /* flags */
+
+ avio_write(pb, uri, length); /* uri */
+ avio_w8(pb, 0); /* null-terminated */
+
+ return update_size(pb, pos);
+}
+
+static int mov_write_conf_box(AVIOContext *pb, const uint8_t *data, int length, char* tag)
+{
+ int64_t pos = avio_tell(pb);
+ avio_wb32(pb, 0); /* size */
+ ffio_wfourcc(pb, tag);
+
+ avio_w8(pb, 0); /* version */
+ avio_wb24(pb, 0); /* flags */
+
+ avio_write(pb, data, length); /* data */
+
+ return update_size(pb, pos);
+}
+
+static int mov_write_meta_codec(AVIOContext *pb, MOVTrack *track)
+{
+ int64_t pos = avio_tell(pb);
+
+ char *data;
+ char *tag;
+
+ AVTimedMetadataInfo *meta =
+ (void*) av_stream_get_side_data(track->st, AV_PKT_DATA_TIMED_METADATA_INFO,
+ NULL);
+
+ av_assert0(meta);
+
+ avio_wb32(pb, 0); /* size */
+ ffio_wfourcc(pb, meta->meta_tag);
+
+ avio_wb32(pb, 0); /* Reserved */
+ avio_wb16(pb, 0); /* Reserved */
+ avio_wb16(pb, 1); /* Data-reference index */
+
+ data = (char*) (meta + 1);
+ tag = meta->meta_tag;
+ if (tag[0] == 'u' && tag[1] == 'r' && tag[2] == 'i' && tag[3] == 'm') {
+ mov_write_urim_uri_box(pb, data, meta->meta_length);
+ data += meta->meta_length;
+ }
+
+ if (meta->conf_length) {
+ mov_write_conf_box(pb, data, meta->conf_length, meta->conf_tag);
+ data += meta->conf_length;
+ }
+
+ return update_size(pb, pos);
+}
+
static int mov_write_subtitle_tag(AVIOContext *pb, MOVTrack *track)
{
int64_t pos = avio_tell(pb);
@@ -1951,6 +2014,18 @@ static int mov_write_tmcd_tag(AVIOContext *pb, MOVTrack *track)
return update_size(pb, pos);
}

+static int mov_write_data_tag(AVIOContext *pb, MOVTrack *track)
+{
+ if (track->par->codec_id == AV_CODEC_ID_META)
+ return mov_write_meta_codec(pb, track);
+ else if (track->par->codec_tag == MKTAG('t','m','c','d'))
+ return mov_write_tmcd_tag(pb, track);
+ else if (track->par->codec_tag == MKTAG('r','t','p',' '))
+ return mov_write_rtp_tag(pb, track);
+ else
+ return 0;
+}
+
static int mov_write_stsd_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track)
{
int64_t pos = avio_tell(pb);
@@ -1962,12 +2037,10 @@ static int mov_write_stsd_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
mov_write_video_tag(pb, mov, track);
else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO)
mov_write_audio_tag(s, pb, mov, track);
+ else if (track->par->codec_type == AVMEDIA_TYPE_DATA)
+ mov_write_data_tag(pb, track);
else if (track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)
mov_write_subtitle_tag(pb, track);
- else if (track->par->codec_tag == MKTAG('r','t','p',' '))
- mov_write_rtp_tag(pb, track);
- else if (track->par->codec_tag == MKTAG('t','m','c','d'))
- mov_write_tmcd_tag(pb, track);
return update_size(pb, pos);
}

@@ -2323,6 +2396,8 @@ static int mov_write_minf_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
else
mov_write_gmhd_tag(pb, track);
}
+ else if (track->par->codec_type == AVMEDIA_TYPE_DATA)
+ mov_write_nmhd_tag(pb);
if (track->mode == MODE_MOV) /* FIXME: Why do it for MODE_MOV only ? */
mov_write_hdlr_tag(s, pb, NULL);
mov_write_dinf_tag(pb);
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index 700d8d5..0fdc70f 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -27,6 +27,7 @@
#include "avformat.h"
#include "movenccenc.h"
#include "movtref.h"
+#include "movmeta.h"

#define MOV_FRAG_INFO_ALLOC_INCREMENT 64
#define MOV_INDEX_CLUSTER_SIZE 1024
diff --git a/libavformat/movmeta.h b/libavformat/movmeta.h
new file mode 100644
index 0000000..e57b381
--- /dev/null
+++ b/libavformat/movmeta.h
@@ -0,0 +1,46 @@
+/*
+ * MOV, 3GP, MP4 muxer
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVFORMAT_MOVMETA_H
+#define AVFORMAT_MOVMETA_H
+
+#include "avformat.h"
+
+typedef enum MOVMetaType {
+ MOV_META_NONE,
+ MOV_META_URIM
+} MOVMetaType;
+
+typedef struct MOVConfMeta {
+ uint32_t tag;
+ int length;
+ void *data;
+} MOVConfMeta;
+
+typedef struct MOVMeta {
+ uint32_t tag;
+ int length;
+ void *data;
+
+ MOVConfMeta conf;
+} MOVMeta;
+
+#endif /* AVFORMAT_MOVMETA_H */
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:57 UTC
Permalink
Added support for passing complex channel layout configuration as side
packet data (AV_PKT_DATA_AUDIO_TRACK_CHANNEL_LAYOUT) to ISO media files
as specified by ISO/IEC 14496-12. AVAudioTrackChannelLayout has the
fields to setting both predefined audio layouts, completely configuring
the azimuth and elevation of each speaker as well as describing the
number of audio objects in the scene.

This information isn't integrated into the existing channel layout
system though, which is much more restricted compared to what the
standard permits. However, the side packet data is structured so that it
does not require too much ISO base media file format knowledge in client
code. In addition, it should be possible to extend the enumeration and
the record to allow for using the same side data for a more native
solution.

The names of the channels and layouts are available in
channel_layout_isoiec23001_8.h with slightly obtuse names such as
AV_SPEAKER_POSITION_ISOIEC23001_8_L and
AV_CH_LAYOUT_ISOIEC23001_8_1_0_0 to encourage path forward to a more
native solution for FFmpeg.

This channel layout information ends up to a chnl box in the written
file in an isom track.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 71 +++++++++++++++++++++++
libavformat/movenc.c | 73 +++++++++++++++++++++++-
libavutil/channel_layout_isoiec23001_8.h | 97 ++++++++++++++++++++++++++++++++
3 files changed, 240 insertions(+), 1 deletion(-)
create mode 100644 libavutil/channel_layout_isoiec23001_8.h

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 8373bca..7f14751 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -39,6 +39,7 @@
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
#include "libavutil/rational.h"
+#include "libavutil/channel_layout_isoiec23001_8.h"

#include "version.h"

@@ -1367,6 +1368,69 @@ typedef struct AVTrackReferences {
/** followed by an optional gap for alignment purposes and another AVTrackReferences is applicaple */
} AVTrackReferences;

+/**
+ * Describes the speaker position of a single audio channel of a single track
+ *
+ * The name is chosen in a slightly obscure manner as to allow a more
+ * natural name to take its place when the system supports FFmpeg's
+ * native channel positions.
+ */
+typedef struct AVAudioTrackChannelPositionISOIEC23001_8 {
+ AVSpeakerPositionISOIEC23001_8 speaker_position; /** an OutputChannelPosition from ISO/IEC 23001-8 */
+
+ /** The following are used if speaker_position == AV_SPEAKER_POSITION_ISOIEC23001_8_EXPL */
+ int16_t azimuth; /** Degrees -180..180. Values increment counterclockwise from above. */
+ int8_t elevation; /** Degrees -90..90. >0 is above horizon. */
+} AVAudioTrackChannelPositionISOIEC23001_8;
+
+/**
+ * Describes the channel layout (ie. speaker position) of a single audio track
+ *
+ * The name is chosen in a slightly obscure manner as to allow a more
+ * natural name to take its place when the system supports FFmpeg's
+ * native channel positions.
+ */
+typedef struct AVAudioTrackChannelCompleteLayoutISOIEC23001_8 {
+ int nb_positions;
+ AVAudioTrackChannelPositionISOIEC23001_8 positions[64];
+} AVAudioTrackChannelCompleteLayoutISOIEC23001_8;
+
+/**
+ * Describes the channel layout based on predefined layout of a single
+ * track by providing the layout and the list of channels are are
+ * omitted. For example, you may choose a layout that has 6.1 channels
+ * and then choose to omit the LFE channel from your channels.
+ *
+ * The name is chosen in a slightly obscure manner as to allow a more natural
+ * name to take its place when the system supports FFmpeg's native layouts.
+ */
+typedef struct AVAudioTrackChannelPredefinedLayoutISOIEC23001_8 {
+ AVChannelLayoutISOIEC23001_8 layout; /** ChannelConfiguration from ISO/IEC 23001-8 */
+ uint64_t omitted_channels; /** lsb 1 means the first channel is omitted and so on */
+} AVAudioTrackChannelPredefinedLayoutISOIEC23001_8;
+
+typedef enum AVComplexAudioTrackChannelLayoutType {
+ AV_COMPLEX_CHANNEL_LAYOUT_OBJECTS_ONLY,
+ AV_COMPLEX_CHANNEL_LAYOUT_PREDEFINED_ISOIEC23001_8,
+ AV_COMPLEX_CHANNEL_LAYOUT_COMPLETE_ISOIEC23001_8,
+} AVComplexAudioTrackChannelLayoutType;
+
+typedef struct AVAudioTrackChannelLayout {
+ AVComplexAudioTrackChannelLayoutType type;
+ union {
+ AVAudioTrackChannelPredefinedLayoutISOIEC23001_8 predefined;
+ AVAudioTrackChannelCompleteLayoutISOIEC23001_8 complete;
+ };
+
+ /**
+ * Describes the channel layout to be object-structured with given
+ * number of objects. Object-structured audio is means to describe
+ * an audio scene without a fixed channel layout that can be mixed
+ * to varying channel configurations.
+ */
+ int nb_audio_objects; /** Number of audio objects */
+} AVAudioTrackChannelLayout;
+
enum AVPacketSideDataType {
AV_PKT_DATA_PALETTE,

@@ -1559,6 +1623,13 @@ enum AVPacketSideDataType {
* meta data configuration. The value is of type AVTimedMetadataInfo.
*/
AV_PKT_DATA_TIMED_METADATA_INFO,
+
+ /**
+ * Channel layout, describing the position of spakers for the
+ * channels of a track, following the structure
+ * AVAudioTrackChannelLayout.
+ */
+ AV_PKT_DATA_AUDIO_TRACK_CHANNEL_LAYOUT,
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 883fa57..6e179ef 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -557,6 +557,75 @@ static unsigned compute_avg_bitrate(MOVTrack *track)
return size * 8 * track->timescale / track->track_duration;
}

+static int mov_write_chnl_tag(AVIOContext *pb, MOVTrack *track)
+{
+ AVAudioTrackChannelLayout *side_data =
+ (void*) av_stream_get_side_data(track->st, AV_PKT_DATA_AUDIO_TRACK_CHANNEL_LAYOUT,
+ NULL);
+
+ AVAudioTrackChannelPredefinedLayoutISOIEC23001_8 *predefined =
+ side_data->type == AV_COMPLEX_CHANNEL_LAYOUT_PREDEFINED_ISOIEC23001_8
+ ? &side_data->predefined
+ : NULL;
+
+ AVAudioTrackChannelCompleteLayoutISOIEC23001_8 *complete =
+ side_data->type == AV_COMPLEX_CHANNEL_LAYOUT_COMPLETE_ISOIEC23001_8
+ ? &side_data->complete
+ : NULL;
+
+ int object_count = side_data->nb_audio_objects;
+
+ if (!predefined && !complete && !object_count) {
+ return 0;
+ } else {
+ int64_t pos = avio_tell(pb);
+
+ int channel_structured = predefined || complete;
+ int object_structured = !!object_count;
+
+ // ChannelConfiguration from ISO/IEC 23001-8
+ int defined_layout = predefined ? predefined->layout : 0;
+ int channel_count = track->par->channels;
+
+ int stream_structure = (channel_structured << 0) | (object_structured << 1);
+
+ avio_wb32(pb, 0); // size
+ ffio_wfourcc(pb, "chnl");
+ avio_wb32(pb, 0); // Version
+
+ avio_w8(pb, stream_structure);
+
+ if (channel_structured) {
+ avio_w8(pb, defined_layout);
+ if (defined_layout == 0) {
+ AVAudioTrackChannelPositionISOIEC23001_8* positions;
+ int i;
+ av_assert0(complete);
+ av_assert0(complete->nb_positions >= channel_count);
+
+ positions = complete->positions;
+
+ for (i = 0; i < channel_count; ++i) {
+ AVAudioTrackChannelPositionISOIEC23001_8 *pos = &positions[i];
+ avio_w8(pb, pos->speaker_position);
+ if (pos->speaker_position == 126) {
+ avio_wb16(pb, pos->azimuth);
+ avio_w8(pb, pos->elevation);
+ }
+ }
+ } else {
+ av_assert0(predefined);
+
+ avio_wb64(pb, predefined->omitted_channels);
+ }
+ }
+ if (object_structured)
+ avio_w8(pb, object_count);
+
+ return update_size(pb, pos);
+ }
+}
+
static int mov_write_esds_tag(AVIOContext *pb, MOVTrack *track) // Basic
{
AVCPBProperties *props;
@@ -996,8 +1065,10 @@ static int mov_write_audio_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex
(mov_pcm_le_gt16(track->par->codec_id) && version==1) ||
(mov_pcm_be_gt16(track->par->codec_id) && version==1)))
mov_write_wave_tag(s, pb, track);
- else if (track->tag == MKTAG('m','p','4','a'))
+ else if (track->tag == MKTAG('m','p','4','a')) {
+ mov_write_chnl_tag(pb, track);
mov_write_esds_tag(pb, track);
+ }
else if (track->par->codec_id == AV_CODEC_ID_AMR_NB)
mov_write_amr_tag(pb, track);
else if (track->par->codec_id == AV_CODEC_ID_AC3)
diff --git a/libavutil/channel_layout_isoiec23001_8.h b/libavutil/channel_layout_isoiec23001_8.h
new file mode 100644
index 0000000..72f39c1
--- /dev/null
+++ b/libavutil/channel_layout_isoiec23001_8.h
@@ -0,0 +1,97 @@
+/*
+ * copyright (c) 2016 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_CHANNEL_LAYOUT_ISOIEC23001_8_H
+#define AVUTIL_CHANNEL_LAYOUT_ISOIEC23001_8_H
+
+/** Speaker positions according to ISO/IEC 23001-8 */
+typedef enum AVSpeakerPositionISOIEC23001_8 {
+ AV_SPEAKER_POSITION_ISOIEC23001_8_L = 0, /// Left front
+ AV_SPEAKER_POSITION_ISOIEC23001_8_R = 1, /// Right front
+ AV_SPEAKER_POSITION_ISOIEC23001_8_C = 2, /// Centre front
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LFE = 3, /// Low frequency enhancement
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LS = 4, /// Left surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RS = 5, /// Right surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LC = 6, /// Left front centre
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RC = 7, /// Right front centre
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LSR = 8, /// Rear surround left
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RSR = 9, /// Rear surround right
+ AV_SPEAKER_POSITION_ISOIEC23001_8_CS = 10, /// Rear centre
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LSD = 11, /// Left surround direct
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RSD = 12, /// Right surround direct
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LSS = 13, /// Left side surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RSS = 14, /// Right side surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LW = 15, /// Left wide front
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RW = 16, /// Right wide front
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LV = 17, /// Left front vertical height
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RV = 18, /// Right front vertical height
+ AV_SPEAKER_POSITION_ISOIEC23001_8_CV = 19, /// Centre front vertical height
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LVR = 20, /// Left surround vertical height rear
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RVR = 21, /// Right surround vertical height rear
+ AV_SPEAKER_POSITION_ISOIEC23001_8_CVR = 22, /// Centre vertical height rear
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LVSS = 23, /// Left vertical height side surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RVSS = 24, /// Right vertical height side surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_TS = 25, /// Top centre surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LFE2 = 26, /// E2 Low frequency enhancement 2
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LB = 27, /// Left front vertical bottom
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RB = 28, /// Right front vertical bottom
+ AV_SPEAKER_POSITION_ISOIEC23001_8_CB = 29, /// Centre front vertical bottom
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LVS = 30, /// Left vertical height surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RVS = 31, /// Right vertical height surround
+ /// 32-45 Reserved
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LFE3 = 36, /// E3 Low frequency enhancement 3
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LEOS = 37, /// Left edge of screen
+ AV_SPEAKER_POSITION_ISOIEC23001_8_REOS = 38, /// Right edge of screen
+ AV_SPEAKER_POSITION_ISOIEC23001_8_HWBCAL = 39, /// half-way between centre of screen and left edge of screen
+ AV_SPEAKER_POSITION_ISOIEC23001_8_HWBCAR = 40, /// half-way between centre of screen and right edge of screen
+ AV_SPEAKER_POSITION_ISOIEC23001_8_LBS = 41, /// Left back surround
+ AV_SPEAKER_POSITION_ISOIEC23001_8_RBS = 42, /// Right back surround
+ /// 43–125 Reserved
+ AV_SPEAKER_POSITION_ISOIEC23001_8_EXPL = 126, /// Explicit position (see text)
+ /// 127 Unknown / undefined
+} AVSpeakerPositionISOIEC23001_8;
+
+/** Channel layouts according to ISO/IEC 23001-8 */
+typedef enum AVChannelLayoutISOIEC23001_8 {
+ AV_CH_LAYOUT_ISOIEC23001_8_ANY,
+ AV_CH_LAYOUT_ISOIEC23001_8_1_0_0, /// 1 centre front
+ AV_CH_LAYOUT_ISOIEC23001_8_2_0_0, /// 2 left front, right front
+ AV_CH_LAYOUT_ISOIEC23001_8_3_0_0, /// 3 centre front, left front, right front
+ AV_CH_LAYOUT_ISOIEC23001_8_3_1_0, /// 4 centre front, left front, right front, rear centre
+ AV_CH_LAYOUT_ISOIEC23001_8_3_2_0, /// 5 centre front, left front, right front, left surround, right surround
+ AV_CH_LAYOUT_ISOIEC23001_8_3_2_1, /// 6 centre front, left front, right front, left surround, right surround, LFE
+ AV_CH_LAYOUT_ISOIEC23001_8_5_2_1A, /// 7 centre front, left front centre, right front centre, left front, right front, left surround, right surround, LFE
+ AV_CH_LAYOUT_ISOIEC23001_8_1P1, /// 8 channel1 channel2
+ AV_CH_LAYOUT_ISOIEC23001_8_2_1_0, /// 9 left front, right front, rear centre
+ AV_CH_LAYOUT_ISOIEC23001_8_2_2_0, /// 10 left front, right front, left surround, right surround
+ AV_CH_LAYOUT_ISOIEC23001_8_3_3_1, /// 11 centre front, left front, right front, left surround, right surround, rear centre, LFE
+ AV_CH_LAYOUT_ISOIEC23001_8_3_4_1, /// 12 centre front, left front, right front, left surround, right surround, rear surround left, rear surround right, LFE
+ AV_CH_LAYOUT_ISOIEC23001_8_11_11_2, /// 13 centre front, left front centre, right front centre, left front, right front, left side surround, right side surround, rear left surround, rear right surround, rear centre, left front LFE, right front LFE, centre front vertical height, left front vertical height, right front vertical height, left vertical height side surround, r ight vertical height side surround, top centre surround, left surround vertical height rear, r ight surround vertical height rear , centre vertical height rear, centre front vertical bottom, left front vertical bottom, right front vertical bottom
+ AV_CH_LAYOUT_ISOIEC23001_8_5_2_1B, /// 14 centre front, left front, right front, left surround, right surround, LFE, left front vertical height, right front vertical height
+ AV_CH_LAYOUT_ISOIEC23001_8_5_5_2, /// 15 centre front, left front, right front, left side surround, right side surround, left surround, right surround, left front vertical height, right front vertical height, centre vertical height rear, LFE1, LFE2
+ AV_CH_LAYOUT_ISOIEC23001_8_5_4_1, /// 16 centre front, left front, right front, left surround, right surround, LFE, left front vertical height, right front vertical height, left vertical height surround, right vertical height surround
+ AV_CH_LAYOUT_ISOIEC23001_8_6_5_1, /// 17 centre front, left front, right front, left surround, right surround, LFE, left front vertical height, right front vertical height, centre front vertical height, left vertical height surround, right vertical height surround, top centre surround
+ AV_CH_LAYOUT_ISOIEC23001_8_6_7_1, /// 18 centre front, left front, right front, left surround, right surround, left back surround, right back surround LFE, left front vertical height, right front vertical height, centre front vertical height, left vertical height surround, right vertical height surround, top centre surround
+ AV_CH_LAYOUT_ISOIEC23001_8_5_6_1, /// 19 centre front, left front, right front, left side surround, right side surround, rear surround left, rear surround right, LFE, left front vertical height, right front vertical height, left surround vertical height rear, right surround vertical height rear
+ AV_CH_LAYOUT_ISOIEC23001_8_7_6_1, /// 20 centre front, left edge of screen, right edge of screen, left front, right front, left side surround, right side surround, rear surround left, rear surround right, LFE, left front vertical height, right front vertical height, left vertical height surround, right vertical height surround
+ /// 21..63: reserved
+} AVChannelLayoutISOIEC23001_8;
+
+#endif /* AVUTIL_CHANNEL_LAYOUT_ISOIEC23001_8_H */
--
2.7.4
Carl Eugen Hoyos
2016-08-31 14:00:37 UTC
Permalink
Hi!
Post by e***@nokia.com
Added support for passing complex channel layout configuration as side
packet data (AV_PKT_DATA_AUDIO_TRACK_CHANNEL_LAYOUT) to ISO
media files as specified by ISO/IEC 14496-12. AVAudioTrackChannelLayout
has the fields to setting both predefined audio layouts, completely
configuring the azimuth and elevation of each speaker as well as describing
the number of audio objects in the scene.
(I used to work on parts of the FFmpeg channel layout code, I know it is
broken to some degree but I still hope to get it fixed one day...)

Are you using this feature already (internally) or did you just feel like it is
missing? (Sorry, it is not meant offending.) I wonder if all this complexity
really has a real-world use-case...

How is "chnl" related to "chan"? I thought I remember that "chan" already
offers possibilities for custom channel layouts.
Can both be used in the same file?

Thank you, Carl Eugen
Erkki Seppälä
2016-09-01 13:24:25 UTC
Permalink
Hello!
Post by Carl Eugen Hoyos
Are you using this feature already (internally) or did you just feel like it is
missing? (Sorry, it is not meant offending.) I wonder if all this complexity
really has a real-world use-case...
We may or may not be using this functionality internally. Maybe the
signed off-tag gives a hint where :).
Post by Carl Eugen Hoyos
How is "chnl" related to "chan"? I thought I remember that "chan" already
offers possibilities for custom channel layouts.
Can both be used in the same file?
To me it seems these two tags are probably historically connected, chan
being a precursor to chnl, though there seem to be some big differences
in the range of options, if not in practical functionality. For example,
the predefined layouts are completely different - mov_chan.c mentions 85
predefined channel layouts whereas channel_layouts_isoiec23001_8.h has 20.

The conversions in mov_chan.c seem a bit lossy, though, for example:

{ MOV_CH_LAYOUT_MPEG_4_0_A, AV_CH_LAYOUT_4POINT0 }, // L,
R, C, Cs
{ MOV_CH_LAYOUT_MPEG_4_0_B, AV_CH_LAYOUT_4POINT0 }, // C,
L, R, Cs
{ MOV_CH_LAYOUT_AC3_3_1, AV_CH_LAYOUT_4POINT0 }, // L,
C, R, Cs

Is this not losing the channel order? It doesn't seem like it's
compensated anywhere.

I doubt the two tags can be used in the same track in a way that it
makes sense. Perhaps more so in the same file, with alternative audio
tracks where audio tracks originate from different sources?

It seems the mov_chan.c approach would be applicable to these layouts as
well, except the mapping between existing FFmpeg layouts might be
error-prone and one wouldn't want to have duplicate layouts meaning the
same thing. Also mov_chan.c seems to have code for indicating speaker
positions (by their coordinates) in ff_mov_read_chan, but it just throws
them away (it doesn't even try to write anything complicated in
mov_write_chan_tag).

The trickiest parts in my opinion would be:

1) The same channels can be in different order in the same file in
different channel layouts. Doesn't FFmpeg throw away this permutation
information? Does some higher level care about that?

2) There is no API or internals for dealing with speakers at arbitrary
azimuth/elevation.

I'm of course just selling this patch, but it does seem less risky to
have a side-data-based implementation in first ;-). (Perhaps even by
adding a suffix _EXPERIMENTAL to the side data, but it seems FFmpeg
hasn't done that so far.)
Erkki Seppälä
2016-08-31 11:35:52 UTC
Permalink
Signed-off-by: Erkki Seppälä <***@vincit.fi>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/utils.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index 8b55464..f0e22b9 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -2880,6 +2880,8 @@ static int do_encode(AVCodecContext *avctx, const AVFrame *frame, int *got_packe
} else if (avctx->codec_type == AVMEDIA_TYPE_AUDIO) {
ret = avcodec_encode_audio2(avctx, avctx->internal->buffer_pkt,
frame, got_packet);
+ } else if (avctx->codec_type == AVMEDIA_TYPE_DATA) {
+ ret = avctx->codec->encode2(avctx, avctx->internal->buffer_pkt, frame, got_packet);
} else {
ret = AVERROR(EINVAL);
}
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:55 UTC
Permalink
Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/mov.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/libavformat/mov.c b/libavformat/mov.c
index 1012b3c..0544d13 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -2257,6 +2257,7 @@ int ff_mov_read_stsd_entries(MOVContext *c, AVIOContext *pb, int entries)
(format >> 0) & 0xff, (format >> 8) & 0xff, (format >> 16) & 0xff,
(format >> 24) & 0xff, format, st->codecpar->codec_type);

+ ret = 0;
if (st->codecpar->codec_type==AVMEDIA_TYPE_VIDEO) {
st->codecpar->codec_id = id;
mov_parse_stsd_video(c, pb, st, sc);
@@ -2267,12 +2268,16 @@ int ff_mov_read_stsd_entries(MOVContext *c, AVIOContext *pb, int entries)
st->codecpar->codec_id = id;
mov_parse_stsd_subtitle(c, pb, st, sc,
size - (avio_tell(pb) - start_pos));
+ } else if (st->codecpar->codec_type==AVMEDIA_TYPE_DATA){
+ st->codecpar->codec_id = id;
+ ret = mov_parse_stsd_data(c, pb, st, sc,
+ size - (avio_tell(pb) - start_pos));
} else {
ret = mov_parse_stsd_data(c, pb, st, sc,
size - (avio_tell(pb) - start_pos));
- if (ret < 0)
- return ret;
}
+ if (ret < 0)
+ return ret;
/* this will read extra atoms at the end (wave, alac, damr, avcC, hvcC, SMI ...) */
a.size = size - (avio_tell(pb) - start_pos);
if (a.size > 8) {
--
2.7.4
OZO Play and Live
2016-08-31 21:44:03 UTC
Permalink
Hi all,

The ***@nokia.com email address was used for sales and marketing queries and is no longer monitored. Should these messages be redirected elsewhere?

Liz

-----Original Message-----
From: Seppala Erkki (EXT-Vincit/Tampere)
Sent: Wednesday, August 31, 2016 2:36 PM
To: ffmpeg-***@ffmpeg.org
Cc: Seppala Erkki (EXT-Vincit/Tampere) <***@nokia.com>; OZO Play and Live <***@nokia.com>
Subject: [PATCH v2 12/18] avformat/mov: ff_mov_read_stsd_entries now deals with AVMEDIA_TYPE_DATA

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/mov.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/libavformat/mov.c b/libavformat/mov.c index 1012b3c..0544d13 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -2257,6 +2257,7 @@ int ff_mov_read_stsd_entries(MOVContext *c, AVIOContext *pb, int entries)
(format >> 0) & 0xff, (format >> 8) & 0xff, (format >> 16) & 0xff,
(format >> 24) & 0xff, format, st->codecpar->codec_type);

+ ret = 0;
if (st->codecpar->codec_type==AVMEDIA_TYPE_VIDEO) {
st->codecpar->codec_id = id;
mov_parse_stsd_video(c, pb, st, sc); @@ -2267,12 +2268,16 @@ int ff_mov_read_stsd_entries(MOVContext *c, AVIOContext *pb, int entries)
st->codecpar->codec_id = id;
mov_parse_stsd_subtitle(c, pb, st, sc,
size - (avio_tell(pb) - start_pos));
+ } else if (st->codecpar->codec_type==AVMEDIA_TYPE_DATA){
+ st->codecpar->codec_id = id;
+ ret = mov_parse_stsd_data(c, pb, st, sc,
+ size - (avio_tell(pb) -
+ start_pos));
} else {
ret = mov_parse_stsd_data(c, pb, st, sc,
size - (avio_tell(pb) - start_pos));
- if (ret < 0)
- return ret;
}
+ if (ret < 0)
+ return ret;
/* this will read extra atoms at the end (wave, alac, damr, avcC, hvcC, SMI ...) */
a.size = size - (avio_tell(pb) - start_pos);
if (a.size > 8) {
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:51 UTC
Permalink
This allows using avcodec_send_packet with data frames (ie. timed
metadata)

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/utils.c | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index 138125a..8b55464 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -2737,6 +2737,10 @@ static int do_decode(AVCodecContext *avctx, AVPacket *pkt)
} else if (avctx->codec_type == AVMEDIA_TYPE_AUDIO) {
ret = avcodec_decode_audio4(avctx, avctx->internal->buffer_frame,
&got_frame, pkt);
+ } else if (avctx->codec_type == AVMEDIA_TYPE_DATA) {
+ ret = avctx->codec->decode(avctx, avctx->internal->buffer_frame, &got_frame, pkt);
+ if (ret == 0 && got_frame)
+ ret = pkt->size;
} else {
ret = AVERROR(EINVAL);
}
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:47 UTC
Permalink
This includes creating an AVCodecTag table ff_codec_metadata_tags as
there are for video, audio and subtitles. The tag table is used for
mov-compatiblity.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.c | 5 +++++
libavformat/isom.h | 1 +
libavformat/movenc.c | 14 ++++++++++++++
3 files changed, 20 insertions(+)

diff --git a/libavformat/isom.c b/libavformat/isom.c
index cb457dd..1a90d00 100644
--- a/libavformat/isom.c
+++ b/libavformat/isom.c
@@ -355,6 +355,11 @@ const AVCodecTag ff_codec_movsubtitle_tags[] = {
{ AV_CODEC_ID_NONE, 0 },
};

+const AVCodecTag ff_codec_metadata_tags[] = {
+ { AV_CODEC_ID_META, MKTAG('m', 'e', 't', 'a') },
+ { AV_CODEC_ID_NONE, 0 },
+};
+
/* map numeric codes from mdhd atom to ISO 639 */
/* cf. QTFileFormat.pdf p253, qtff.pdf p205 */
/* http://developer.apple.com/documentation/mac/Text/Text-368.html */
diff --git a/libavformat/isom.h b/libavformat/isom.h
index df6c15a..49c8996 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -33,6 +33,7 @@ extern const AVCodecTag ff_mp4_obj_type[];
extern const AVCodecTag ff_codec_movvideo_tags[];
extern const AVCodecTag ff_codec_movaudio_tags[];
extern const AVCodecTag ff_codec_movsubtitle_tags[];
+extern const AVCodecTag ff_codec_metadata_tags[];

int ff_mov_iso639_to_lang(const char lang[4], int mp4);
int ff_mov_lang_to_iso639(unsigned code, char to[4]);
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 571c2a7..f02458b 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1470,6 +1470,8 @@ static int mov_get_codec_tag(AVFormatContext *s, MOVTrack *track)
}
} else if (track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)
tag = ff_codec_get_tag(ff_codec_movsubtitle_tags, track->par->codec_id);
+ else if (track->par->codec_type == AVMEDIA_TYPE_DATA)
+ tag = ff_codec_get_tag(ff_codec_metadata_tags, track->par->codec_id);
}

return tag;
@@ -2242,6 +2244,9 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra
} else if (track->par->codec_tag == MKTAG('t','m','c','d')) {
hdlr_type = "tmcd";
descr = "TimeCodeHandler";
+ } else if (track->par->codec_type == AVMEDIA_TYPE_DATA) {
+ hdlr_type = "meta";
+ descr = "DataHandler";
} else {
char tag_buf[32];
av_get_codec_tag_string(tag_buf, sizeof(tag_buf),
@@ -5308,6 +5313,7 @@ static void enable_tracks(AVFormatContext *s)
case AVMEDIA_TYPE_VIDEO:
case AVMEDIA_TYPE_AUDIO:
case AVMEDIA_TYPE_SUBTITLE:
+ case AVMEDIA_TYPE_DATA:
if (enabled[i] > 1)
mov->per_stream_grouping = 1;
if (!enabled[i] && first[i] >= 0)
@@ -6110,6 +6116,7 @@ AVOutputFormat ff_mov_muxer = {
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = CONFIG_LIBX264_ENCODER ?
AV_CODEC_ID_H264 : AV_CODEC_ID_MPEG4,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6129,6 +6136,7 @@ AVOutputFormat ff_tgp_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AMR_NB,
.video_codec = AV_CODEC_ID_H263,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6148,6 +6156,7 @@ AVOutputFormat ff_mp4_muxer = {
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = CONFIG_LIBX264_ENCODER ?
AV_CODEC_ID_H264 : AV_CODEC_ID_MPEG4,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6166,6 +6175,7 @@ AVOutputFormat ff_psp_muxer = {
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = CONFIG_LIBX264_ENCODER ?
AV_CODEC_ID_H264 : AV_CODEC_ID_MPEG4,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6183,6 +6193,7 @@ AVOutputFormat ff_tg2_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AMR_NB,
.video_codec = AV_CODEC_ID_H263,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6201,6 +6212,7 @@ AVOutputFormat ff_ipod_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = AV_CODEC_ID_H264,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6219,6 +6231,7 @@ AVOutputFormat ff_ismv_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = AV_CODEC_ID_H264,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
@@ -6237,6 +6250,7 @@ AVOutputFormat ff_f4v_muxer = {
.priv_data_size = sizeof(MOVMuxContext),
.audio_codec = AV_CODEC_ID_AAC,
.video_codec = AV_CODEC_ID_H264,
+ .data_codec = AV_CODEC_ID_META,
.write_header = mov_write_header,
.write_packet = mov_write_packet,
.write_trailer = mov_write_trailer,
--
2.7.4
Carl Eugen Hoyos
2016-08-31 13:04:28 UTC
Permalink
Hi!

2016-08-31 13:35 GMT+02:00 Erkki Seppälä <***@nokia.com>:

Iirc, this has the (intended or unintended) side-effect of making
mov output of ffmpeg (the app) suddenly defaulting to an
additional track.
Is this correct? Is it intended?

Thank you, Carl Eugen
Erkki Seppälä
2016-08-31 13:28:30 UTC
Permalink
Hello!
Post by Carl Eugen Hoyos
Iirc, this has the (intended or unintended) side-effect of making
mov output of ffmpeg (the app) suddenly defaulting to an
additional track.
Is this correct? Is it intended?
Hmm, it seems mov files still have one track according to these commands:

% ffmpeg -i foo.png foo.mov
% MP4Box -diso foo.mov
% grep ' TrackID=' foo_info.xml
<TrackHeaderBox CreationTime="0" ModificationTime="0" TrackID="1"
Duration="40" Width="2.00" Height="2.00">

Certtainly that kind of variation would not be intended. The fate tests
(lavf-moov) have bitwise comparisons for mov results, but of course they
don't use the defaults.

So I imagine you are referring to the enable_tracks function. The change
is probably not strictly required for this functionality to work in
general and it was added for uniformity in dealing with this the same as
with other track types.

Thanks for input!
Carl Eugen Hoyos
2016-08-31 13:42:46 UTC
Permalink
Hi!
Post by Erkki Seppälä
Post by Carl Eugen Hoyos
Iirc, this has the (intended or unintended) side-effect of making
mov output of ffmpeg (the app) suddenly defaulting to an
additional track.
Is this correct? Is it intended?
% ffmpeg -i foo.png foo.mov
Sorry!

I meant "defaulting to an additional track in the output mov file if the
input file contains such a data track (instead of ignoring this track
of the input file by default)".

Carl Eugen
Erkki Seppälä
2016-08-31 13:46:16 UTC
Permalink
Post by Carl Eugen Hoyos
Sorry!
No problem :).
Post by Carl Eugen Hoyos
I meant "defaulting to an additional track in the output mov file if the
input file contains such a data track (instead of ignoring this track
of the input file by default)".
I'll look into how that part of code works and try it out a bit.

Thanks!
Erkki Seppälä
2016-09-06 11:13:09 UTC
Permalink
Hi,

(Cc'ing as it's been some time.)
Post by Carl Eugen Hoyos
I meant "defaulting to an additional track in the output mov file if the
input file contains such a data track (instead of ignoring this track
of the input file by default)".
So I looked into it and indeed the data tracks end up with the enabled
flag enabled. If this affects anything is a bit unclear to me, perhaps
in some players? It seems to me they would unlikely try to play
something they don't know how to - in a worse case perhaps the can stop
or crash.

It seems also 'tmcd' tracks end up being enabled (but not by this
function). I imagine this is by design.

I can remove the enabled-flag for the next patch set for the
AVMEDIA_TYPE_DATA tracks (this does not include tmcd as it doesn't have
a corresponding stream). Or perhaps the appropriate solution would be to
not write such data tracks into QuickTime video files in the first place
and only support them in .mp4 by reverting the codec support for MOV
with these kind of codecs.

Thanks for input!
Erkki Seppälä
2016-09-06 11:14:45 UTC
Permalink
(Oops, sorry about the subject, our MTA helpfully prepended it.)
Carl Eugen Hoyos
2016-09-06 11:37:59 UTC
Permalink
Hi!
Post by Erkki Seppälä
(Cc'ing as it's been some time.)
Please don't.
Post by Erkki Seppälä
Post by Carl Eugen Hoyos
I meant "defaulting to an additional track in the output mov file if the
input file contains such a data track (instead of ignoring this track
of the input file by default)".
So I looked into it and indeed the data tracks end up with the enabled
flag enabled. If this affects anything is a bit unclear to me, perhaps in
some players? It seems to me they would unlikely try to play something
they don't know how to - in a worse case perhaps the can stop or crash.
Sorry, that is not what I meant (although it may or may not
be an issue).
I meant that for the same ffmpeg command line and the same
input file, I believe that with your patch, the output file (suddenly)
has one track more than before (because the data codec is now
listed in AVOutputFormat, the ffmpeg cli will try to add this track
if the input file provides such a codec).

Maybe this is wrong because the data codec does not produce
a new track if it is passed to the muxer?

Iirc, my question was equivalent to:
Can you just remove the additional ".data_codec=" lines or is
it intended that they are now default codecs?
(Or am I simply wrong and they either do not lead to a new
track and / or ffmpeg simply ignores them because it only fills
video, audio and subtitle tracks?)

Carl Eugen
Erkki Seppälä
2016-09-06 13:07:21 UTC
Permalink
Post by Carl Eugen Hoyos
I meant that for the same ffmpeg command line and the same
input file, I believe that with your patch, the output file (suddenly)
has one track more than before (because the data codec is now
listed in AVOutputFormat, the ffmpeg cli will try to add this track
if the input file provides such a codec).
Ok, so now I finally understood the case and indeed if I have a file
with video+data, converting it to a .mov without any particular mapping
options results in a video+data track, instead of just video as what
would happen if the .data_codecs are not defined. (I used ffmpeg -i
foo.mp4 -codec copy bar.mp4 (or .mov) for testing where foo.mp4 has a
data track.)

A curiosity: if I copy a file with two video tracks and two data tracks
I end up with a file with one video track and two data tracks (unless I
use -map : to copy all).

Would it be even better to not copy the data tracks at all by default,
so not set the .data_codec field for any format?

Or another view is that it would seem useful though that all the data
tracks would be copied (but so does copying all video tracks). Or it
could copy all the tracks that have track references to tracks copied by
default..
Post by Carl Eugen Hoyos
Can you just remove the additional ".data_codec=" lines or is
it intended that they are now default codecs?
(Or am I simply wrong and they either do not lead to a new
track and / or ffmpeg simply ignores them because it only fills
video, audio and subtitle tracks?)
This seems the correct solution. My next patch set will remove the
.data_codec at least from output formats other than the .mp4.

Thanks!
Erkki Seppälä
2016-09-07 07:48:48 UTC
Permalink
Post by Erkki Seppälä
Would it be even better to not copy the data tracks at
all by default, so not set the .data_codec field for any
format?
This was my original question for which I do not know
the answer: If you feel it doesn't hurt not to copy the
track I am all for not changing behaviour.
I will then opt for not changing the behavior. If the behavior is deemed
change-worthy in the future, it can be in its own patch.

Thanks.
Erkki Seppälä
2016-08-31 11:35:48 UTC
Permalink
Instead of one track reference, allow many, and instead of
of track reference type (ie. 'cdsc'), allow many.

In addition this patch allows client to explicitly add track references
with side packet AV_PKT_DATA_TRACK_REFERENCES containing
AVTrackReferences (which of there can many many).

Internally MOVTrack's track references can be manipulated with helper
functions ff_mov_tref* (and is used by a later patch for reading
MOVTRefs).

Multiple track references can be useful in particular with timed meta
data tracks, indicating the track is related to multiple other tracks.

This information ends up in ISO media file box 'tref' as specified by
ISO/IEC 14496-12.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 19 +++++++-
libavformat/Makefile | 4 +-
libavformat/movenc.c | 127 +++++++++++++++++++++++++++++++++++++++++---------
libavformat/movenc.h | 5 +-
libavformat/movtref.c | 115 +++++++++++++++++++++++++++++++++++++++++++++
libavformat/movtref.h | 75 +++++++++++++++++++++++++++++
6 files changed, 318 insertions(+), 27 deletions(-)
create mode 100644 libavformat/movtref.c
create mode 100644 libavformat/movtref.h

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 0d0447e6..f45cdc2 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1348,6 +1348,15 @@ typedef struct AVCPBProperties {
* Types and functions for working with AVPacket.
* @{
*/
+
+typedef struct AVTrackReferences {
+ int next_tref_ofs; /** offset in bytes to the next AVTrackReferences or 0 if this is the last one*/
+ char tag[4]; /** 4cc used for describing this */
+ int nb_tracks; /** number of tracks */
+ int tracks[1]; /** tracks this track refers to (contains nb_tracks entries) */
+ /** followed by an optional gap for alignment purposes and another AVTrackReferences is applicaple */
+} AVTrackReferences;
+
enum AVPacketSideDataType {
AV_PKT_DATA_PALETTE,

@@ -1525,7 +1534,15 @@ enum AVPacketSideDataType {
* should be associated with a video stream and containts data in the form
* of the AVMasteringDisplayMetadata struct.
*/
- AV_PKT_DATA_MASTERING_DISPLAY_METADATA
+ AV_PKT_DATA_MASTERING_DISPLAY_METADATA,
+
+ /**
+ * Define track references (in particular applicaple for ISO MP4
+ * files). The data is a sequence of type AVTrackReferences
+ * (including the track list that follows it), for as long as
+ * indicated by the key's length.
+ */
+ AV_PKT_DATA_TRACK_REFERENCES,
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/Makefile b/libavformat/Makefile
index fda1e17..6a393ed 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -280,10 +280,10 @@ OBJS-$(CONFIG_MLV_DEMUXER) += mlvdec.o riffdec.o
OBJS-$(CONFIG_MM_DEMUXER) += mm.o
OBJS-$(CONFIG_MMF_DEMUXER) += mmf.o
OBJS-$(CONFIG_MMF_MUXER) += mmf.o rawenc.o
-OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o replaygain.o
+OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o replaygain.o movtref.o
OBJS-$(CONFIG_MOV_MUXER) += movenc.o avc.o hevc.o vpcc.o \
movenchint.o mov_chan.o rtp.o \
- movenccenc.o rawutils.o
+ movenccenc.o rawutils.o movtref.o
OBJS-$(CONFIG_MP2_MUXER) += mp3enc.o rawenc.o id3v2enc.o
OBJS-$(CONFIG_MP3_DEMUXER) += mp3dec.o replaygain.o
OBJS-$(CONFIG_MP3_MUXER) += mp3enc.o rawenc.o id3v2enc.o
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index f02458b..6cc315a 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2631,14 +2631,22 @@ static int mov_write_edts_tag(AVIOContext *pb, MOVMuxContext *mov,
return size;
}

-static int mov_write_tref_tag(AVIOContext *pb, MOVTrack *track)
+static int mov_write_tref_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track)
{
- avio_wb32(pb, 20); // size
+ int64_t pos = avio_tell(pb);
+ int i, j;
+ avio_wb32(pb, 0); // size
ffio_wfourcc(pb, "tref");
- avio_wb32(pb, 12); // size (subatom)
- avio_wl32(pb, track->tref_tag);
- avio_wb32(pb, track->tref_id);
- return 20;
+ for (j = 0; j < track->trefs.nb_trefs; ++j) {
+ int64_t pos_sub = avio_tell(pb);
+ MOVTRef* tref = &track->trefs.trefs[j];
+ avio_wb32(pb, 0); // size (subatom)
+ avio_wl32(pb, tref->tag);
+ for (i = 0; i < tref->nb_track_ids; i++)
+ avio_wb32(pb, tref->track_ids[i]);
+ update_size(pb, pos_sub);
+ }
+ return update_size(pb, pos);
}

// goes at the end of each track! ... Critical for PSP playback ("Incompatible data" without it)
@@ -2749,8 +2757,8 @@ static int mov_write_trak_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
"Not writing any edit list even though one would have been required\n");
}

- if (track->tref_tag)
- mov_write_tref_tag(pb, track);
+ if (track->trefs.nb_trefs)
+ mov_write_tref_tag(pb, mov, track);

if ((ret = mov_write_mdia_tag(s, pb, mov, track)) < 0)
return ret;
@@ -3487,16 +3495,82 @@ static int mov_setup_track_ids(MOVMuxContext *mov, AVFormatContext *s)
return 0;
}

+static int mov_tref_copy_from_side_data(MOVMuxContext *mov, MOVTrack *track, AVFormatContext *s)
+{
+ int size;
+ int ret;
+ int i;
+ MOVTRef *tref;
+ int *ref_tracks = NULL;
+
+ char *ptr = (void*) av_stream_get_side_data(track->st,
+ AV_PKT_DATA_TRACK_REFERENCES,
+ &size);
+
+ if (!ptr)
+ return 0;
+
+ ret = 0;
+ while (ret == 0 && size >= sizeof(AVTrackReferences)) {
+ AVTrackReferences refs;
+ int *track_ids;
+
+ // avoid any potential alignment issues by copying the struct before accessing it
+ // to a well-aligned address
+ memcpy(&refs, ptr, sizeof(refs));
+ // as well as the ids
+ ref_tracks = av_malloc_array(refs.nb_tracks, sizeof(refs.tracks[0]));
+ memcpy(ref_tracks, &((AVTrackReferences*) ptr)->tracks, sizeof(refs.tracks[0]) * refs.nb_tracks);
+
+ ret = ff_mov_tref_find_or_add(&track->trefs,
+ MKTAG(refs.tag[0], refs.tag[1], refs.tag[2], refs.tag[3]),
+ &tref);
+ if (ret < 0)
+ goto error;
+
+ ret = ff_mov_tref_alloc(tref, refs.nb_tracks, &track_ids);
+ if (ret < 0)
+ goto error;
+
+ for (i = 0; i < refs.nb_tracks; i++) {
+ int tref_stream_id = ref_tracks[i];
+ int stream_idx;
+ for (stream_idx = 0; stream_idx < mov->nb_streams; ++stream_idx)
+ if (mov->tracks[stream_idx].st &&
+ mov->tracks[stream_idx].st->id == tref_stream_id) {
+ track_ids[i] = mov->tracks[stream_idx].track_id;
+ break;
+ }
+ }
+
+ size -= refs.next_tref_ofs;
+ ptr += refs.next_tref_ofs;
+
+ av_free(ref_tracks);
+ if (refs.next_tref_ofs == 0)
+ break;
+ }
+ return 0;
+error:
+ av_free(ref_tracks);
+ return ret;
+}
+
static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
AVFormatContext *s)
{
int i;
int64_t pos = avio_tell(pb);
+ int ret;
avio_wb32(pb, 0); /* size placeholder*/
ffio_wfourcc(pb, "moov");

mov_setup_track_ids(mov, s);

+ for (i = 0; i < mov->nb_streams; i++)
+ if (mov->tracks[i].st)
+ mov_tref_copy_from_side_data(mov, &mov->tracks[i], s);
+
for (i = 0; i < mov->nb_streams; i++) {
if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
continue;
@@ -3508,34 +3582,42 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
}

if (mov->chapter_track)
- for (i = 0; i < s->nb_streams; i++) {
- mov->tracks[i].tref_tag = MKTAG('c','h','a','p');
- mov->tracks[i].tref_id = mov->tracks[mov->chapter_track].track_id;
- }
+ for (i = 0; i < s->nb_streams; i++)
+ if (!ff_codec_get_id(ff_codec_metadata_tags, mov->tracks[i].tag)) {
+ ret = ff_mov_tref_add_one_track(&mov->tracks[i].trefs, MKTAG('c','h','a','p'), mov->chapter_track);
+ if (ret < 0)
+ return ret;
+ }
for (i = 0; i < mov->nb_streams; i++) {
MOVTrack *track = &mov->tracks[i];
if (track->tag == MKTAG('r','t','p',' ')) {
- track->tref_tag = MKTAG('h','i','n','t');
- track->tref_id = mov->tracks[track->src_track].track_id;
+ ret = ff_mov_tref_add_one_track(&mov->tracks[i].trefs, MKTAG('h','i','n','t'), mov->tracks[track->src_track].track_id);
+ if (ret < 0)
+ return ret;
} else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) {
int * fallback, size;
fallback = (int*)av_stream_get_side_data(track->st,
AV_PKT_DATA_FALLBACK_TRACK,
&size);
- if (fallback != NULL && size == sizeof(int)) {
- if (*fallback >= 0 && *fallback < mov->nb_streams) {
- track->tref_tag = MKTAG('f','a','l','l');
- track->tref_id = mov->tracks[*fallback].track_id;
- }
+ if (fallback != NULL && size == sizeof(int) &&
+ *fallback >= 0 && *fallback < mov->nb_streams) {
+ ret = ff_mov_tref_add_one_track(&track->trefs, MKTAG('f','a','l','l'), *fallback);
+ if (ret < 0)
+ return ret;
}
}
}
for (i = 0; i < mov->nb_streams; i++) {
if (mov->tracks[i].tag == MKTAG('t','m','c','d')) {
int src_trk = mov->tracks[i].src_track;
- mov->tracks[src_trk].tref_tag = mov->tracks[i].tag;
- mov->tracks[src_trk].tref_id = mov->tracks[i].track_id;
- //src_trk may have a different timescale than the tmcd track
+
+ ret = ff_mov_tref_add_one_track(&mov->tracks[src_trk].trefs,
+ mov->tracks[i].tag,
+ mov->tracks[i].track_id);
+ if (ret < 0)
+ return ret;
+
+ //src_trk may have a different timescale than the tmcd track
mov->tracks[i].track_duration = av_rescale(mov->tracks[src_trk].track_duration,
mov->tracks[i].timescale,
mov->tracks[src_trk].timescale);
@@ -5346,6 +5428,7 @@ static void mov_free(AVFormatContext *s)
av_freep(&mov->tracks[i].vos_data);

ff_mov_cenc_free(&mov->tracks[i].cenc);
+ ff_mov_tref_free(&mov->tracks[i].trefs);
}

av_freep(&mov->tracks);
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index ea76e39..700d8d5 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -26,6 +26,7 @@

#include "avformat.h"
#include "movenccenc.h"
+#include "movtref.h"

#define MOV_FRAG_INFO_ALLOC_INCREMENT 64
#define MOV_INDEX_CLUSTER_SIZE 1024
@@ -110,8 +111,8 @@ typedef struct MOVTrack {
unsigned cluster_capacity;
int audio_vbr;
int height; ///< active picture (w/o VBI) height for D-10/IMX
- uint32_t tref_tag;
- int tref_id; ///< trackID of the referenced track
+
+ MOVTRefs trefs;
int64_t start_dts;
int64_t start_cts;
int64_t end_pts;
diff --git a/libavformat/movtref.c b/libavformat/movtref.c
new file mode 100644
index 0000000..dd6059c
--- /dev/null
+++ b/libavformat/movtref.c
@@ -0,0 +1,115 @@
+/*
+ * ISO base media file format track references
+ * Copyright (c) 2016 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <stdint.h>
+#include <inttypes.h>
+#include <stdlib.h>
+
+#include "movtref.h"
+#include "libavutil/mem.h"
+
+MOVTRef *ff_mov_tref_find(MOVTRefs *trefs, uint32_t tag)
+{
+ int i;
+ MOVTRef *tref = NULL;
+
+ for (i = 0; i < trefs->nb_trefs && !tref; ++i) {
+ if (trefs->trefs[i].tag == tag) {
+ tref = trefs->trefs + i;
+ }
+ }
+
+ return tref;
+}
+
+int ff_mov_tref_alloc(MOVTRef *tref, int n, int **track_ids_ret)
+{
+ int ret = av_reallocp_array(&tref->track_ids, tref->nb_track_ids + n, sizeof(tref->track_ids));
+ *track_ids_ret = NULL;
+ if (ret >= 0) {
+ *track_ids_ret = tref->track_ids + tref->nb_track_ids;
+ tref->nb_track_ids += n;
+ }
+ return ret;
+}
+
+int ff_mov_tref_find_or_add(MOVTRefs *trefs, uint32_t tag, MOVTRef **tref_ret)
+{
+ int ret;
+ int i;
+ MOVTRef *tref = ff_mov_tref_find(trefs, tag);
+ *tref_ret = NULL;
+
+ for (i = 0; i < trefs->nb_trefs && !tref; ++i) {
+ if (trefs->trefs[i].tag == tag) {
+ tref = trefs->trefs + i;
+ }
+ }
+
+ if (!tref) {
+ ret = av_reallocp_array(&trefs->trefs, trefs->nb_trefs + 1, sizeof(*trefs->trefs));
+ if (ret < 0)
+ return ret;
+ tref = trefs->trefs + trefs->nb_trefs;
+ trefs->nb_trefs++;
+ tref->tag = tag;
+ tref->track_ids = NULL;
+ tref->nb_track_ids = 0;
+ }
+
+ *tref_ret = tref;
+ return 0;
+}
+
+int ff_mov_tref_add_one_track(MOVTRefs *trefs, unsigned tag, int track_id)
+{
+ int ret;
+ MOVTRef *tref;
+ int *ids;
+ int track_id_exists = 0;
+ int i;
+
+ ret = ff_mov_tref_find_or_add(trefs, tag, &tref);
+ if (ret < 0)
+ return ret;
+
+ for (i = 0; i < tref->nb_track_ids && !track_id_exists; i++) {
+ if (tref->track_ids[i] == track_id) {
+ track_id_exists = 1;
+ }
+ }
+
+ if (!track_id_exists) {
+ ret = ff_mov_tref_alloc(tref, 1, &ids);
+ if (ret >= 0) {
+ *ids = track_id;
+ }
+ }
+ return ret;
+}
+
+void ff_mov_tref_free(MOVTRefs *trefs)
+{
+ int i;
+ for (i = 0; i < trefs->nb_trefs; ++i)
+ av_freep(&trefs->trefs[i].track_ids);
+ av_freep(&trefs->trefs);
+}
diff --git a/libavformat/movtref.h b/libavformat/movtref.h
new file mode 100644
index 0000000..091f467
--- /dev/null
+++ b/libavformat/movtref.h
@@ -0,0 +1,75 @@
+/*
+ * ISO base media file format track references
+ * Copyright (c) 2016 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef AVFORMAT_MOVTREF_H
+#define AVFORMAT_MOVTREF_H
+
+typedef struct MOVTRef {
+ uint32_t tag;
+ int nb_track_ids;
+ int* track_ids;
+} MOVTRef;
+
+typedef struct MOVTRefs {
+ int nb_trefs;
+ MOVTRef *trefs;
+} MOVTRefs;
+
+/**
+ * @brief Finds a track reference of certain tag from a track; if not
+ * found, return NULL
+ *
+ * @param track The track to search from
+ * @param tag The tag (4cc) to search for
+ * @return a MOVTRef describing the track reference or NULL if not found.
+ */
+MOVTRef *ff_mov_tref_find(MOVTRefs *trefs, uint32_t tag);
+
+/**
+ * @brief Allocate space for n more tracks (and increase nb_ids accordingly) returning
+ * the pointer in *track_ids
+ *
+ * @param tref The track reference to allocate tracks to
+ * @param n Number of tracks to add
+ * @param track_ids_ret A non-null pointer where the pointer to the first free track
+ * id is returned to
+ * @return Returns a value <0 on error, otherwise 0.
+ */
+int ff_mov_tref_alloc(MOVTRef *tref, int n, int **track_ids_ret);
+
+/**
+ * @brief Finds an existing MOVTRef for a track for the given tag, or
+ * creates a new one if it is missing. Returns the tref in tref_ret.
+ */
+
+int ff_mov_tref_find_or_add(MOVTRefs *trefs, uint32_t tag, MOVTRef **tref_ret);
+
+/**
+ * Adds one tref track reference with given track id if it doesn't
+ * already exist
+ */
+int ff_mov_tref_add_one_track(MOVTRefs *trefs, unsigned tag, int track_id);
+
+/**
+ * Release MOVTRefs
+ */
+void ff_mov_tref_free(MOVTRefs *trefs);
+
+#endif /* AVFORMAT_MOVTREF_H */
--
2.7.4
Michael Niedermayer
2016-09-01 17:03:09 UTC
Permalink
Post by Erkki Seppälä
Instead of one track reference, allow many, and instead of
of track reference type (ie. 'cdsc'), allow many.
In addition this patch allows client to explicitly add track references
with side packet AV_PKT_DATA_TRACK_REFERENCES containing
AVTrackReferences (which of there can many many).
Internally MOVTrack's track references can be manipulated with helper
functions ff_mov_tref* (and is used by a later patch for reading
MOVTRefs).
Multiple track references can be useful in particular with timed meta
data tracks, indicating the track is related to multiple other tracks.
This information ends up in ISO media file box 'tref' as specified by
ISO/IEC 14496-12.
---
libavcodec/avcodec.h | 19 +++++++-
libavformat/Makefile | 4 +-
libavformat/movenc.c | 127 +++++++++++++++++++++++++++++++++++++++++---------
libavformat/movenc.h | 5 +-
libavformat/movtref.c | 115 +++++++++++++++++++++++++++++++++++++++++++++
libavformat/movtref.h | 75 +++++++++++++++++++++++++++++
6 files changed, 318 insertions(+), 27 deletions(-)
create mode 100644 libavformat/movtref.c
create mode 100644 libavformat/movtref.h
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 0d0447e6..f45cdc2 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1348,6 +1348,15 @@ typedef struct AVCPBProperties {
* Types and functions for working with AVPacket.
*/
+
+typedef struct AVTrackReferences {
+ int next_tref_ofs; /** offset in bytes to the next AVTrackReferences or 0 if this is the last one*/
+ char tag[4]; /** 4cc used for describing this */
+ int nb_tracks; /** number of tracks */
+ int tracks[1]; /** tracks this track refers to (contains nb_tracks entries) */
This syntax looks wrong

to document the previous field /**< would be used

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Avoid a single point of failure, be that a person or equipment.
Michael Niedermayer
2016-09-01 20:30:40 UTC
Permalink
Post by Erkki Seppälä
Instead of one track reference, allow many, and instead of
of track reference type (ie. 'cdsc'), allow many.
In addition this patch allows client to explicitly add track references
with side packet AV_PKT_DATA_TRACK_REFERENCES containing
AVTrackReferences (which of there can many many).
Internally MOVTrack's track references can be manipulated with helper
functions ff_mov_tref* (and is used by a later patch for reading
MOVTRefs).
Multiple track references can be useful in particular with timed meta
data tracks, indicating the track is related to multiple other tracks.
This information ends up in ISO media file box 'tref' as specified by
ISO/IEC 14496-12.
---
libavcodec/avcodec.h | 19 +++++++-
libavformat/Makefile | 4 +-
libavformat/movenc.c | 127 +++++++++++++++++++++++++++++++++++++++++---------
libavformat/movenc.h | 5 +-
libavformat/movtref.c | 115 +++++++++++++++++++++++++++++++++++++++++++++
libavformat/movtref.h | 75 +++++++++++++++++++++++++++++
[...]
Post by Erkki Seppälä
@@ -2749,8 +2757,8 @@ static int mov_write_trak_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
"Not writing any edit list even though one would have been required\n");
}
- if (track->tref_tag)
- mov_write_tref_tag(pb, track);
+ if (track->trefs.nb_trefs)
+ mov_write_tref_tag(pb, mov, track);
if ((ret = mov_write_mdia_tag(s, pb, mov, track)) < 0)
return ret;
@@ -3487,16 +3495,82 @@ static int mov_setup_track_ids(MOVMuxContext *mov, AVFormatContext *s)
return 0;
}
+static int mov_tref_copy_from_side_data(MOVMuxContext *mov, MOVTrack *track, AVFormatContext *s)
+{
+ int size;
+ int ret;
+ int i;
+ MOVTRef *tref;
+ int *ref_tracks = NULL;
+
+ char *ptr = (void*) av_stream_get_side_data(track->st,
+ AV_PKT_DATA_TRACK_REFERENCES,
+ &size);
+
+ if (!ptr)
+ return 0;
+
+ ret = 0;
+ while (ret == 0 && size >= sizeof(AVTrackReferences)) {
+ AVTrackReferences refs;
+ int *track_ids;
+
+ // avoid any potential alignment issues by copying the struct before accessing it
+ // to a well-aligned address
+ memcpy(&refs, ptr, sizeof(refs));
+ // as well as the ids
+ ref_tracks = av_malloc_array(refs.nb_tracks, sizeof(refs.tracks[0]));
+ memcpy(ref_tracks, &((AVTrackReferences*) ptr)->tracks, sizeof(refs.tracks[0]) * refs.nb_tracks);
missing alloc failure check
Post by Erkki Seppälä
+
+ ret = ff_mov_tref_find_or_add(&track->trefs,
+ MKTAG(refs.tag[0], refs.tag[1], refs.tag[2], refs.tag[3]),
+ &tref);
+ if (ret < 0)
+ goto error;
+
+ ret = ff_mov_tref_alloc(tref, refs.nb_tracks, &track_ids);
+ if (ret < 0)
+ goto error;
+
+ for (i = 0; i < refs.nb_tracks; i++) {
+ int tref_stream_id = ref_tracks[i];
+ int stream_idx;
+ for (stream_idx = 0; stream_idx < mov->nb_streams; ++stream_idx)
+ if (mov->tracks[stream_idx].st &&
+ mov->tracks[stream_idx].st->id == tref_stream_id) {
+ track_ids[i] = mov->tracks[stream_idx].track_id;
+ break;
+ }
+ }
+
+ size -= refs.next_tref_ofs;
+ ptr += refs.next_tref_ofs;
missing checks
anything comming out of av_stream_get_side_data() could originate from
an attacker and have manually choosen evil values.

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When you are offended at any man's fault, turn to yourself and study your
own failings. Then you will forget your anger. -- Epictetus
Erkki Seppälä
2016-09-02 09:27:01 UTC
Permalink
Post by Michael Niedermayer
missing alloc failure check
Fixed.
Post by Michael Niedermayer
missing checks
anything comming out of av_stream_get_side_data() could originate from
an attacker and have manually choosen evil values.
Oh, well it seems I have trusted it quite a bit more. I will harden the
code that uses the function.

Thanks for review!
Erkki Seppälä
2016-08-31 11:35:53 UTC
Permalink
This can be useful in particular with timed meta data tracks related
to multiple tracks.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.h | 3 ++
libavformat/mov.c | 119 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 121 insertions(+), 1 deletion(-)

diff --git a/libavformat/isom.h b/libavformat/isom.h
index 49c8996..4f4ab0e 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -27,6 +27,8 @@
#include "avio.h"
#include "internal.h"
#include "dv.h"
+#include "movmeta.h"
+#include "movtref.h"

/* isom.c */
extern const AVCodecTag ff_mp4_obj_type[];
@@ -168,6 +170,7 @@ typedef struct MOVStreamContext {
int start_pad; ///< amount of samples to skip due to enc-dec delay
unsigned int rap_group_count;
MOVSbgp *rap_group;
+ MOVTRefs trefs;

int nb_frames_for_fps;
int64_t duration_for_fps;
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 1bc3800..ff4c91c 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -3157,6 +3157,121 @@ static void fix_timescale(MOVContext *c, MOVStreamContext *sc)
}
}

+static int mov_read_tref_subatom(MOVContext *c, AVIOContext *pb, MOVTRefs *trefs)
+{
+ uint32_t tref_tag;
+ MOVTRef *tref;
+ int tref_ids_size;
+ int *tref_ids;
+ int ret;
+
+ tref_ids_size = avio_rb32(pb);
+ if (tref_ids_size % 4 != 0)
+ return AVERROR_INVALIDDATA;
+ tref_ids_size -= 4;
+
+ tref_tag = avio_rl32(pb);
+ tref_ids_size -= 4;
+ if (tref_ids_size < 4)
+ return AVERROR_INVALIDDATA;
+
+ ret = ff_mov_tref_find_or_add(trefs, tref_tag, &tref);
+ if (ret != 0)
+ return ret;
+
+ ff_mov_tref_alloc(tref, tref_ids_size / 4, &tref_ids);
+ if (ret != 0)
+ return ret;
+
+ while (tref_ids_size) {
+ *tref_ids = avio_rb32(pb);
+ ++tref_ids;
+ tref_ids_size -= 4;
+ }
+
+ return ret;
+}
+
+static int mov_tref_copy_to_side_data(AVStream *st, MOVTRefs *trefs)
+{
+ if (trefs->nb_trefs) {
+ int i;
+ char *trefs_side_ptr;
+ int end_offset = 0;
+ int offset = 0;
+ AVTrackReferences *trefs_side_prev = NULL;
+
+ for (i = 0; i < trefs->nb_trefs; ++i) {
+ MOVTRef *tref = &trefs->trefs[i];
+ if (tref->nb_track_ids > 0)
+ // Ensure the returned data is easy to access without
+ // worrying about alignment, even if it wastes some memory
+ end_offset = FFALIGN(end_offset + sizeof(AVTrackReferences) +
+ sizeof(int) * (tref->nb_track_ids - 1),
+ sizeof(AVTrackReferences));
+ }
+
+ trefs_side_ptr = (void*) av_stream_new_side_data(st,
+ AV_PKT_DATA_TRACK_REFERENCES,
+ end_offset);
+ if (!trefs_side_ptr)
+ return AVERROR(ENOMEM);
+
+ for (i = 0; i < trefs->nb_trefs; ++i) {
+ MOVTRef *tref = &trefs->trefs[i];
+ if (tref->nb_track_ids > 0) {
+ AVTrackReferences *trefs_side = (AVTrackReferences*) (trefs_side_ptr + offset);
+ int *trefs_tracks;
+
+ trefs_side->nb_tracks = tref->nb_track_ids;
+ trefs_side->tag[0] = (tref->tag >> 0) & 0xff;
+ trefs_side->tag[1] = (tref->tag >> 8) & 0xff;
+ trefs_side->tag[2] = (tref->tag >> 16) & 0xff;
+ trefs_side->tag[3] = (tref->tag >> 24) & 0xff;
+ trefs_tracks = trefs_side->tracks;
+ for (i = 0; i < tref->nb_track_ids; ++i) {
+ trefs_tracks[i] = tref->track_ids[i];
+ }
+
+ if (trefs_side_prev)
+ trefs_side_prev->next_tref_ofs = (char*) trefs_side - (char*) trefs_side_prev;
+ trefs_side_prev = trefs_side;
+ offset = FFALIGN(end_offset + sizeof(AVTrackReferences) +
+ sizeof(int) * (tref->nb_track_ids - 1),
+ sizeof(AVTrackReferences));
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int mov_read_tref(MOVContext *c, AVIOContext *pb, MOVAtom atom)
+{
+ AVStream *st;
+ MOVStreamContext *sc;
+ int64_t end = avio_tell(pb) + atom.size;
+
+ if (c->fc->nb_streams < 1)
+ return 0;
+
+ st = c->fc->streams[c->fc->nb_streams-1];
+ sc = st->priv_data;
+
+ if (atom.size < 12) {
+ return mov_read_default(c, pb, atom);
+ } else {
+ int ret = 0;
+ while (end - avio_tell(pb) > 0 && ret == 0) {
+ ret = mov_read_tref_subatom(c, pb, &sc->trefs);
+ }
+ mov_tref_copy_to_side_data(st, &sc->trefs);
+ avio_seek(pb, end, SEEK_SET);
+ }
+
+ return 0;
+}
+
static int mov_read_trak(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
AVStream *st;
@@ -4414,7 +4529,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = {
{ MKTAG('t','f','h','d'), mov_read_tfhd }, /* track fragment header */
{ MKTAG('t','r','a','k'), mov_read_trak },
{ MKTAG('t','r','a','f'), mov_read_default },
-{ MKTAG('t','r','e','f'), mov_read_default },
+{ MKTAG('t','r','e','f'), mov_read_tref },
{ MKTAG('t','m','c','d'), mov_read_tmcd },
{ MKTAG('c','h','a','p'), mov_read_chap },
{ MKTAG('t','r','e','x'), mov_read_trex },
@@ -4832,6 +4947,8 @@ static int mov_read_close(AVFormatContext *s)
av_freep(&sc->cenc.auxiliary_info);
av_freep(&sc->cenc.auxiliary_info_sizes);
av_aes_ctr_free(sc->cenc.aes_ctr);
+
+ ff_mov_tref_free(&sc->trefs);
}

if (mov->dv_demux) {
--
2.7.4
Erkki Seppälä
2016-08-31 11:36:01 UTC
Permalink
Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
doc/examples/Makefile | 2 +
doc/examples/muxing_with_metadata.c | 885 ++++++++++++++++++++++++++++++++++++
2 files changed, 887 insertions(+)
create mode 100644 doc/examples/muxing_with_metadata.c

diff --git a/doc/examples/Makefile b/doc/examples/Makefile
index c10033e..d1f4e1f 100644
--- a/doc/examples/Makefile
+++ b/doc/examples/Makefile
@@ -22,6 +22,7 @@ EXAMPLES= avio_dir_cmd \
http_multiclient \
metadata \
muxing \
+ muxing_with_metadata \
remuxing \
resampling_audio \
scaling_video \
@@ -35,6 +36,7 @@ avcodec: LDLIBS += -lm
decoding_encoding: LDLIBS += -lm
muxing: LDLIBS += -lm
resampling_audio: LDLIBS += -lm
+muxing_with_metadata: LDLIBS += -lm

.phony: all clean-test clean

diff --git a/doc/examples/muxing_with_metadata.c b/doc/examples/muxing_with_metadata.c
new file mode 100644
index 0000000..3119e55
--- /dev/null
+++ b/doc/examples/muxing_with_metadata.c
@@ -0,0 +1,885 @@
+/*
+ * Copyright (c) 2003 Fabrice Bellard
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+/**
+ * @file
+ * libavformat API example.
+ *
+ * Output a media file in any supported libavformat format. The default
+ * codecs are used.
+ * @example muxing.c
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <math.h>
+
+#include <libavutil/avassert.h>
+#include <libavutil/channel_layout.h>
+#include <libavutil/opt.h>
+#include <libavutil/mathematics.h>
+#include <libavutil/timestamp.h>
+#include <libavformat/avformat.h>
+#include <libswscale/swscale.h>
+#include <libswresample/swresample.h>
+
+#define STREAM_DURATION 10.0
+#define STREAM_FRAME_RATE 25 /* 25 images/s */
+#define STREAM_PIX_FMT AV_PIX_FMT_YUV420P /* default pix_fmt */
+
+#define SCALE_FLAGS SWS_BICUBIC
+
+// a wrapper around a single output AVStream
+typedef struct OutputStream {
+ AVStream *st;
+ AVCodecContext *enc;
+
+ /* pts of the next frame that will be generated */
+ int64_t next_pts;
+ int samples_count;
+
+ AVFrame *frame;
+ AVFrame *tmp_frame;
+
+ float t, tincr, tincr2;
+
+ struct SwsContext *sws_ctx;
+ struct SwrContext *swr_ctx;
+} OutputStream;
+
+typedef struct StreamState {
+ int (*writer)(AVFormatContext *oc, OutputStream *ost);
+ int *flag;
+ OutputStream *stream;
+} StreamState;
+
+static void log_packet(const AVFormatContext *fmt_ctx, const AVPacket *pkt)
+{
+ AVRational *time_base = &fmt_ctx->streams[pkt->stream_index]->time_base;
+
+ printf("pts:%s pts_time:%s dts:%s dts_time:%s duration:%s duration_time:%s stream_index:%d\n",
+ av_ts2str(pkt->pts), av_ts2timestr(pkt->pts, time_base),
+ av_ts2str(pkt->dts), av_ts2timestr(pkt->dts, time_base),
+ av_ts2str(pkt->duration), av_ts2timestr(pkt->duration, time_base),
+ pkt->stream_index);
+}
+
+static int write_frame(AVFormatContext *fmt_ctx, const AVRational *time_base, AVStream *st, AVPacket *pkt)
+{
+ /* rescale output packet timestamp values from codec to stream timebase */
+ av_packet_rescale_ts(pkt, *time_base, st->time_base);
+ pkt->stream_index = st->index;
+
+ /* Write the compressed frame to the media file. */
+ log_packet(fmt_ctx, pkt);
+ return av_interleaved_write_frame(fmt_ctx, pkt);
+}
+
+/* Add an output stream. */
+static void add_stream(OutputStream *ost, AVFormatContext *oc,
+ AVCodec **codec,
+ enum AVCodecID codec_id)
+{
+ AVCodecContext *c;
+ int i;
+
+ /* find the encoder */
+ *codec = avcodec_find_encoder(codec_id);
+ if (!(*codec)) {
+ fprintf(stderr, "Could not find encoder for '%s'\n",
+ avcodec_get_name(codec_id));
+ exit(1);
+ }
+
+ ost->st = avformat_new_stream(oc, NULL);
+ if (!ost->st) {
+ fprintf(stderr, "Could not allocate stream\n");
+ exit(1);
+ }
+ ost->st->id = oc->nb_streams-1;
+ c = avcodec_alloc_context3(*codec);
+ if (!c) {
+ fprintf(stderr, "Could not alloc an encoding context\n");
+ exit(1);
+ }
+ ost->enc = c;
+
+ switch ((*codec)->type) {
+ case AVMEDIA_TYPE_AUDIO:
+ c->sample_fmt = (*codec)->sample_fmts ?
+ (*codec)->sample_fmts[0] : AV_SAMPLE_FMT_FLTP;
+ c->bit_rate = 64000;
+ c->sample_rate = 44100;
+ if ((*codec)->supported_samplerates) {
+ c->sample_rate = (*codec)->supported_samplerates[0];
+ for (i = 0; (*codec)->supported_samplerates[i]; i++) {
+ if ((*codec)->supported_samplerates[i] == 44100)
+ c->sample_rate = 44100;
+ }
+ }
+ c->channels = av_get_channel_layout_nb_channels(c->channel_layout);
+ c->channel_layout = AV_CH_LAYOUT_STEREO;
+ if ((*codec)->channel_layouts) {
+ c->channel_layout = (*codec)->channel_layouts[0];
+ for (i = 0; (*codec)->channel_layouts[i]; i++) {
+ if ((*codec)->channel_layouts[i] == AV_CH_LAYOUT_STEREO)
+ c->channel_layout = AV_CH_LAYOUT_STEREO;
+ }
+ }
+ c->channels = av_get_channel_layout_nb_channels(c->channel_layout);
+ ost->st->time_base = (AVRational){ 1, c->sample_rate };
+ break;
+
+ case AVMEDIA_TYPE_VIDEO:
+ c->codec_id = codec_id;
+
+ c->bit_rate = 400000;
+ /* Resolution must be a multiple of two. */
+ c->width = 352;
+ c->height = 288;
+ /* timebase: This is the fundamental unit of time (in seconds) in terms
+ * of which frame timestamps are represented. For fixed-fps content,
+ * timebase should be 1/framerate and timestamp increments should be
+ * identical to 1. */
+ ost->st->time_base = (AVRational){ 1, STREAM_FRAME_RATE };
+ c->time_base = ost->st->time_base;
+
+ c->gop_size = 12; /* emit one intra frame every twelve frames at most */
+ c->pix_fmt = STREAM_PIX_FMT;
+ if (c->codec_id == AV_CODEC_ID_MPEG2VIDEO) {
+ /* just for testing, we also add B frames */
+ c->max_b_frames = 2;
+ }
+ if (c->codec_id == AV_CODEC_ID_MPEG1VIDEO) {
+ /* Needed to avoid using macroblocks in which some coeffs overflow.
+ * This does not happen with normal video, it just happens here as
+ * the motion of the chroma plane does not match the luma plane. */
+ c->mb_decision = 2;
+ }
+ break;
+
+ case AVMEDIA_TYPE_DATA:
+ c->codec_id = codec_id;
+
+ c->bit_rate = 10;
+ /* timebase: This is the fundamental unit of time (in seconds) in terms
+ * of which frame timestamps are represented. For fixed-fps content,
+ * timebase should be 1/framerate and timestamp increments should be
+ * identical to 1. */
+ ost->st->time_base = (AVRational){ 1, STREAM_FRAME_RATE };
+ c->time_base = ost->st->time_base;
+ break;
+
+ default:
+ break;
+ }
+
+ /* Some formats want stream headers to be separate. */
+ if (oc->oformat->flags & AVFMT_GLOBALHEADER)
+ c->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
+}
+
+/**************************************************************/
+/* audio output */
+
+static AVFrame *alloc_audio_frame(enum AVSampleFormat sample_fmt,
+ uint64_t channel_layout,
+ int sample_rate, int nb_samples)
+{
+ AVFrame *frame = av_frame_alloc();
+ int ret;
+
+ if (!frame) {
+ fprintf(stderr, "Error allocating an audio frame\n");
+ exit(1);
+ }
+
+ frame->format = sample_fmt;
+ frame->channel_layout = channel_layout;
+ frame->sample_rate = sample_rate;
+ frame->nb_samples = nb_samples;
+
+ if (nb_samples) {
+ ret = av_frame_get_buffer(frame, 0);
+ if (ret < 0) {
+ fprintf(stderr, "Error allocating an audio buffer\n");
+ exit(1);
+ }
+ }
+
+ return frame;
+}
+
+static void open_audio(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
+{
+ AVCodecContext *c;
+ int nb_samples;
+ int ret;
+ AVDictionary *opt = NULL;
+
+ c = ost->enc;
+
+ /* open it */
+ av_dict_copy(&opt, opt_arg, 0);
+ ret = avcodec_open2(c, codec, &opt);
+ av_dict_free(&opt);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open audio codec: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ /* init signal generator */
+ ost->t = 0;
+ ost->tincr = 2 * M_PI * 110.0 / c->sample_rate;
+ /* increment frequency by 110 Hz per second */
+ ost->tincr2 = 2 * M_PI * 110.0 / c->sample_rate / c->sample_rate;
+
+ if (c->codec->capabilities & AV_CODEC_CAP_VARIABLE_FRAME_SIZE)
+ nb_samples = 10000;
+ else
+ nb_samples = c->frame_size;
+
+ ost->frame = alloc_audio_frame(c->sample_fmt, c->channel_layout,
+ c->sample_rate, nb_samples);
+ ost->tmp_frame = alloc_audio_frame(AV_SAMPLE_FMT_S16, c->channel_layout,
+ c->sample_rate, nb_samples);
+
+ /* copy the stream parameters to the muxer */
+ ret = avcodec_parameters_from_context(ost->st->codecpar, c);
+ if (ret < 0) {
+ fprintf(stderr, "Could not copy the stream parameters\n");
+ exit(1);
+ }
+
+ /* create resampler context */
+ ost->swr_ctx = swr_alloc();
+ if (!ost->swr_ctx) {
+ fprintf(stderr, "Could not allocate resampler context\n");
+ exit(1);
+ }
+
+ /* set options */
+ av_opt_set_int (ost->swr_ctx, "in_channel_count", c->channels, 0);
+ av_opt_set_int (ost->swr_ctx, "in_sample_rate", c->sample_rate, 0);
+ av_opt_set_sample_fmt(ost->swr_ctx, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
+ av_opt_set_int (ost->swr_ctx, "out_channel_count", c->channels, 0);
+ av_opt_set_int (ost->swr_ctx, "out_sample_rate", c->sample_rate, 0);
+ av_opt_set_sample_fmt(ost->swr_ctx, "out_sample_fmt", c->sample_fmt, 0);
+
+ /* initialize the resampling context */
+ if ((ret = swr_init(ost->swr_ctx)) < 0) {
+ fprintf(stderr, "Failed to initialize the resampling context\n");
+ exit(1);
+ }
+}
+
+static AVFrame *alloc_meta_frame(int nb_samples)
+{
+ AVFrame *frame = av_frame_alloc();
+
+ if (!frame) {
+ fprintf(stderr, "Error allocating a meta frame\n");
+ exit(1);
+ }
+
+ frame->nb_samples = nb_samples;
+
+ if (nb_samples) {
+ frame->buf[0] = av_buffer_alloc(nb_samples);
+
+ av_assert0(frame->buf[0]);
+ av_assert0(frame->buf[0]->data);
+
+ frame->data[0] = frame->buf[0]->data;
+ }
+
+ return frame;
+}
+
+
+static void open_data(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
+{
+ AVCodecContext *c;
+ int nb_samples = 100;
+ int ret;
+ AVDictionary *opt = NULL;
+
+ c = ost->enc;
+
+ /* open it */
+ av_dict_copy(&opt, opt_arg, 0);
+ ret = avcodec_open2(c, codec, &opt);
+ av_dict_free(&opt);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open meta codec: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ ost->frame = alloc_meta_frame(nb_samples);
+ ost->tmp_frame = alloc_meta_frame(nb_samples);
+
+ /* copy the stream parameters to the muxer */
+ ret = avcodec_parameters_from_context(ost->st->codecpar, c);
+ if (ret < 0) {
+ fprintf(stderr, "Could not copy the stream parameters\n");
+ exit(1);
+ }
+}
+
+
+/* Prepare a 16 bit dummy audio frame of 'frame_size' samples and
+ * 'nb_channels' channels. */
+static AVFrame *get_audio_frame(OutputStream *ost)
+{
+ AVFrame *frame = ost->tmp_frame;
+ int j, i, v;
+ int16_t *q = (int16_t*)frame->data[0];
+
+ /* check if we want to generate more frames */
+ if (av_compare_ts(ost->next_pts, ost->enc->time_base,
+ STREAM_DURATION, (AVRational){ 1, 1 }) >= 0)
+ return NULL;
+
+ for (j = 0; j <frame->nb_samples; j++) {
+ v = (int)(sin(ost->t) * 10000);
+ for (i = 0; i < ost->enc->channels; i++)
+ *q++ = v;
+ ost->t += ost->tincr;
+ ost->tincr += ost->tincr2;
+ }
+
+ frame->pts = ost->next_pts;
+ ost->next_pts += frame->nb_samples;
+
+ return frame;
+}
+
+static AVFrame *get_meta_frame(OutputStream *ost)
+{
+ AVFrame *frame = ost->tmp_frame;
+ unsigned char *buffer = (unsigned char*) frame->data[0];
+
+ static int n = 0;
+ ++n;
+
+ frame->pts = ost->next_pts;
+ ost->next_pts += 1;
+
+ snprintf((char*) buffer, 42, "urn:example.com:%d", n);
+ frame->data[0] = buffer;
+ frame->nb_samples = strlen((char*) buffer);
+ frame->pts = ost->next_pts++;
+
+ return frame;
+}
+
+static int write_timed_meta_frame(AVFormatContext *oc, OutputStream *ost)
+{
+ AVCodecContext *c;
+ AVPacket pkt = { 0 }; // data and size must be 0;
+ AVFrame *frame;
+ int ret;
+
+ av_init_packet(&pkt);
+ c = ost->enc;
+
+ frame = get_meta_frame(ost);
+
+ ost->samples_count += 1;
+
+ if ((ret = avcodec_send_frame(c, frame))) {
+ fprintf(stderr, "Error while encoding meta frame: %s\n",
+ av_err2str(ret));
+ exit(1);
+ }
+
+ while (avcodec_receive_packet(c, &pkt) == 0) {
+ c->frame_number++;
+
+ ret = write_frame(oc, &c->time_base, ost->st, &pkt);
+ if (ret < 0) {
+ fprintf(stderr, "Error while writing meta frame: %s\n",
+ av_err2str(ret));
+ exit(1);
+ }
+ av_packet_unref(&pkt);
+ }
+
+ return 0;
+}
+
+
+/*
+ * encode one audio frame and send it to the muxer
+ * return 1 when encoding is finished, 0 otherwise
+ */
+static int write_audio_frame(AVFormatContext *oc, OutputStream *ost)
+{
+ AVCodecContext *c;
+ AVPacket pkt = { 0 }; // data and size must be 0;
+ AVFrame *frame;
+ int ret;
+ int dst_nb_samples;
+ int got_packet = 0;
+ int eof = 0;
+
+ c = ost->enc;
+
+ frame = get_audio_frame(ost);
+
+ if (frame) {
+ /* convert samples from native format to destination codec format, using the resampler */
+ /* compute destination number of samples */
+ dst_nb_samples = av_rescale_rnd(swr_get_delay(ost->swr_ctx, c->sample_rate) + frame->nb_samples,
+ c->sample_rate, c->sample_rate, AV_ROUND_UP);
+ av_assert0(dst_nb_samples == frame->nb_samples);
+
+ /* when we pass a frame to the encoder, it may keep a reference to it
+ * internally;
+ * make sure we do not overwrite it here
+ */
+ ret = av_frame_make_writable(ost->frame);
+ if (ret < 0)
+ exit(1);
+
+ /* convert to destination format */
+ ret = swr_convert(ost->swr_ctx,
+ ost->frame->data, dst_nb_samples,
+ (const uint8_t **)frame->data, frame->nb_samples);
+ if (ret < 0) {
+ fprintf(stderr, "Error while converting\n");
+ exit(1);
+ }
+ frame = ost->frame;
+
+ frame->pts = av_rescale_q(ost->samples_count, (AVRational){1, c->sample_rate}, c->time_base);
+ ost->samples_count += dst_nb_samples;
+ }
+
+ av_init_packet(&pkt);
+ ret = avcodec_send_frame(c, frame);
+ if (ret == AVERROR_EOF)
+ eof = 1;
+ else
+ if (ret < 0) {
+ fprintf(stderr, "Error encoding audio frame: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ while (avcodec_receive_packet(c, &pkt) == 0) {
+ ret = write_frame(oc, &c->time_base, ost->st, &pkt);
+ if (ret < 0) {
+ fprintf(stderr, "Error while writing audio frame: %s\n",
+ av_err2str(ret));
+ exit(1);
+ } else
+ got_packet = 1;
+ av_packet_unref(&pkt);
+ }
+
+ return ((frame || got_packet) && !eof) ? 0 : 1;
+}
+
+/**************************************************************/
+/* video output */
+
+static AVFrame *alloc_picture(enum AVPixelFormat pix_fmt, int width, int height)
+{
+ AVFrame *picture;
+ int ret;
+
+ picture = av_frame_alloc();
+ if (!picture)
+ return NULL;
+
+ picture->format = pix_fmt;
+ picture->width = width;
+ picture->height = height;
+
+ /* allocate the buffers for the frame data */
+ ret = av_frame_get_buffer(picture, 32);
+ if (ret < 0) {
+ fprintf(stderr, "Could not allocate frame data.\n");
+ exit(1);
+ }
+
+ return picture;
+}
+
+static void open_video(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
+{
+ int ret;
+ AVCodecContext *c = ost->enc;
+ AVDictionary *opt = NULL;
+
+ av_dict_copy(&opt, opt_arg, 0);
+
+ /* open the codec */
+ ret = avcodec_open2(c, codec, &opt);
+ av_dict_free(&opt);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open video codec: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ /* allocate and init a re-usable frame */
+ ost->frame = alloc_picture(c->pix_fmt, c->width, c->height);
+ if (!ost->frame) {
+ fprintf(stderr, "Could not allocate video frame\n");
+ exit(1);
+ }
+
+ /* If the output format is not YUV420P, then a temporary YUV420P
+ * picture is needed too. It is then converted to the required
+ * output format. */
+ ost->tmp_frame = NULL;
+ if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
+ ost->tmp_frame = alloc_picture(AV_PIX_FMT_YUV420P, c->width, c->height);
+ if (!ost->tmp_frame) {
+ fprintf(stderr, "Could not allocate temporary picture\n");
+ exit(1);
+ }
+ }
+
+ /* copy the stream parameters to the muxer */
+ ret = avcodec_parameters_from_context(ost->st->codecpar, c);
+ if (ret < 0) {
+ fprintf(stderr, "Could not copy the stream parameters\n");
+ exit(1);
+ }
+}
+
+/* Prepare a dummy image. */
+static void fill_yuv_image(AVFrame *pict, int frame_index,
+ int width, int height)
+{
+ int x, y, i, ret;
+
+ /* when we pass a frame to the encoder, it may keep a reference to it
+ * internally;
+ * make sure we do not overwrite it here
+ */
+ ret = av_frame_make_writable(pict);
+ if (ret < 0)
+ exit(1);
+
+ i = frame_index;
+
+ /* Y */
+ for (y = 0; y < height; y++)
+ for (x = 0; x < width; x++)
+ pict->data[0][y * pict->linesize[0] + x] = x + y + i * 3;
+
+ /* Cb and Cr */
+ for (y = 0; y < height / 2; y++) {
+ for (x = 0; x < width / 2; x++) {
+ pict->data[1][y * pict->linesize[1] + x] = 128 + y + i * 2;
+ pict->data[2][y * pict->linesize[2] + x] = 64 + x + i * 5;
+ }
+ }
+}
+
+static AVFrame *get_video_frame(OutputStream *ost)
+{
+ AVCodecContext *c = ost->enc;
+
+ /* check if we want to generate more frames */
+ if (av_compare_ts(ost->next_pts, ost->enc->time_base,
+ STREAM_DURATION, (AVRational){ 1, 1 }) >= 0)
+ return NULL;
+
+ if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
+ /* as we only generate a YUV420P picture, we must convert it
+ * to the codec pixel format if needed */
+ if (!ost->sws_ctx) {
+ ost->sws_ctx = sws_getContext(c->width, c->height,
+ AV_PIX_FMT_YUV420P,
+ c->width, c->height,
+ c->pix_fmt,
+ SCALE_FLAGS, NULL, NULL, NULL);
+ if (!ost->sws_ctx) {
+ fprintf(stderr,
+ "Could not initialize the conversion context\n");
+ exit(1);
+ }
+ }
+ fill_yuv_image(ost->tmp_frame, ost->next_pts, c->width, c->height);
+ sws_scale(ost->sws_ctx,
+ (const uint8_t * const *)ost->tmp_frame->data, ost->tmp_frame->linesize,
+ 0, c->height, ost->frame->data, ost->frame->linesize);
+ } else {
+ fill_yuv_image(ost->frame, ost->next_pts, c->width, c->height);
+ }
+
+ ost->frame->pts = ost->next_pts++;
+
+ return ost->frame;
+}
+
+/*
+ * encode one video frame and send it to the muxer
+ * return 1 when encoding is finished, 0 otherwise
+ */
+static int write_video_frame(AVFormatContext *oc, OutputStream *ost)
+{
+ int ret;
+ AVCodecContext *c;
+ AVFrame *frame;
+ int got_packet = 0;
+
+ c = ost->enc;
+
+ frame = get_video_frame(ost);
+
+ if (oc->oformat->flags & AVFMT_RAWPICTURE) {
+ /* a hack to avoid data copy with some raw video muxers */
+ AVPacket pkt;
+ av_init_packet(&pkt);
+
+ if (!frame)
+ return 1;
+
+ pkt.flags |= AV_PKT_FLAG_KEY;
+ pkt.stream_index = ost->st->index;
+ pkt.data = (uint8_t *)frame;
+ pkt.size = sizeof(AVPicture);
+
+ pkt.pts = pkt.dts = frame->pts;
+ av_packet_rescale_ts(&pkt, c->time_base, ost->st->time_base);
+
+ ret = av_interleaved_write_frame(oc, &pkt);
+ } else {
+ AVPacket pkt = { 0 };
+ av_init_packet(&pkt);
+
+ /* encode the image */
+ ret = avcodec_send_frame(c, frame);
+ if (ret < 0) {
+ fprintf(stderr, "Error encoding video frame: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ while (avcodec_receive_packet(c, &pkt) == 0) {
+ ret = write_frame(oc, &c->time_base, ost->st, &pkt);
+ av_packet_unref(&pkt);
+ }
+ }
+
+ if (ret < 0) {
+ fprintf(stderr, "Error while writing video frame: %s\n", av_err2str(ret));
+ exit(1);
+ }
+
+ return (frame || got_packet) ? 0 : 1;
+}
+
+static void close_stream(AVFormatContext *oc, OutputStream *ost)
+{
+ avcodec_close(ost->enc);
+ av_frame_free(&ost->frame);
+ av_frame_free(&ost->tmp_frame);
+ sws_freeContext(ost->sws_ctx);
+ swr_free(&ost->swr_ctx);
+}
+
+/**************************************************************/
+/* media file output */
+
+int main(int argc, char **argv)
+{
+ OutputStream video_st = { 0 }, audio_st = { 0 }, data_st = { 0 };
+ const char *filename;
+ AVOutputFormat *fmt;
+ AVFormatContext *oc;
+ AVCodec *audio_codec, *video_codec, *data_codec;
+ int ret;
+ int have_video = 0, have_audio = 0, have_data = 0;
+ int encode_video = 0, encode_audio = 0, mux_meta = 0;
+ AVDictionary *opt = NULL;
+
+ StreamState streams[3] = {
+ { write_video_frame,
+ &encode_video,
+ &video_st },
+ { write_audio_frame,
+ &encode_audio,
+ &audio_st },
+ { write_timed_meta_frame,
+ &mux_meta,
+ &data_st }
+ };
+
+ /* Initialize libavcodec, and register all codecs and formats. */
+ av_register_all();
+
+ if (argc < 2) {
+ printf("usage: %s output_file\n"
+ "API example program to output a media file with libavformat.\n"
+ "This program generates a synthetic audio and video stream, encodes and\n"
+ "muxes them into a file named output_file.\n"
+ "The output format is automatically guessed according to the file extension.\n"
+ "Raw images can also be output by using '%%d' in the filename.\n"
+ "\n", argv[0]);
+ return 1;
+ }
+
+ filename = argv[1];
+ if (argc > 3 && !strcmp(argv[2], "-flags")) {
+ av_dict_set(&opt, argv[2]+1, argv[3], 0);
+ }
+
+ /* allocate the output media context */
+ avformat_alloc_output_context2(&oc, NULL, NULL, filename);
+ if (!oc) {
+ printf("Could not deduce output format from file extension: using MPEG.\n");
+ avformat_alloc_output_context2(&oc, NULL, "mpeg", filename);
+ }
+ if (!oc)
+ return 1;
+
+ fmt = oc->oformat;
+
+ /* Add the audio and video streams using the default format codecs
+ * and initialize the codecs. */
+ if (fmt->video_codec != AV_CODEC_ID_NONE) {
+ add_stream(&video_st, oc, &video_codec, fmt->video_codec);
+ have_video = 1;
+ encode_video = 1;
+ }
+ if (fmt->audio_codec != AV_CODEC_ID_NONE) {
+ add_stream(&audio_st, oc, &audio_codec, fmt->audio_codec);
+ have_audio = 1;
+ encode_audio = 1;
+ }
+ if (fmt->data_codec != AV_CODEC_ID_NONE) {
+ add_stream(&data_st, oc, &data_codec, fmt->data_codec);
+ have_data = 1;
+ mux_meta = 1;
+ }
+
+ /* Now that all the parameters are set, we can open the audio and
+ * video codecs and allocate the necessary encode buffers. */
+ if (have_video)
+ open_video(oc, video_codec, &video_st, opt);
+
+ if (have_audio)
+ open_audio(oc, audio_codec, &audio_st, opt);
+
+ if (have_data)
+ open_data(oc, data_codec, &data_st, opt);
+
+ av_dump_format(oc, 0, filename, 1);
+
+ /* open the output file, if needed */
+ if (!(fmt->flags & AVFMT_NOFILE)) {
+ ret = avio_open(&oc->pb, filename, AVIO_FLAG_WRITE);
+ if (ret < 0) {
+ fprintf(stderr, "Could not open '%s': %s\n", filename,
+ av_err2str(ret));
+ return 1;
+ }
+ }
+
+ /* Write the stream header, if any. */
+ ret = avformat_write_header(oc, &opt);
+ if (ret < 0) {
+ fprintf(stderr, "Error occurred when opening output file: %s\n",
+ av_err2str(ret));
+ return 1;
+ }
+
+ if (fmt->data_codec != AV_CODEC_ID_NONE) {
+ AVTimedMetadata* metadata;
+ char* uri = "http://example.com/";
+ void* ptr;
+
+ AVTrackReferences* trefs;
+ char* msg = "test config data";
+ int nb_tref_tracks = 2;
+
+ metadata = (AVTimedMetadata*) av_stream_new_side_data(oc->streams[2], AV_PKT_DATA_TIMED_METADATA_INFO,
+ sizeof(AVTimedMetadata) + strlen(uri) + strlen(msg));
+
+ memcpy(metadata->meta_tag, "urim", 4);
+ metadata->meta_length = strlen(uri);
+
+ memset(metadata->conf_tag, 0, sizeof(metadata->conf_tag));
+ memcpy(metadata->conf_tag, "cfg ", 4);
+ metadata->conf_length = strlen(msg);
+
+ ptr = (char*) (metadata + 1);
+ memcpy(ptr, uri, strlen(uri));
+ ptr += strlen(uri);
+ memcpy(ptr, msg, strlen(msg));
+
+ trefs = (AVTrackReferences*) av_stream_new_side_data(oc->streams[2], AV_PKT_DATA_TRACK_REFERENCES,
+ sizeof(AVTrackReferences) + nb_tref_tracks * sizeof(int));
+ trefs->next_tref_ofs = 0;
+ memcpy(trefs->tag, "cdsc", 4);
+ trefs->nb_tracks = nb_tref_tracks;
+ trefs->tracks[0] = 0;
+ trefs->tracks[1] = 1;
+ }
+
+ while (encode_video || encode_audio) {
+ StreamState *stream = NULL;
+ int idx;
+
+ for (idx = 0; idx < 3; ++idx) {
+ if (*streams[idx].flag &&
+ (stream == NULL ||
+ av_compare_ts(streams[idx].stream->next_pts, streams[idx].stream->enc->time_base,
+ stream->stream->next_pts, stream->stream->enc->time_base) <= 0)) {
+ stream = &streams[idx];
+ }
+ }
+
+ if (stream) {
+ *stream->flag = !stream->writer(oc, stream->stream);
+ }
+ }
+
+ /* Write the trailer, if any. The trailer must be written before you
+ * close the CodecContexts open when you wrote the header; otherwise
+ * av_write_trailer() may try to use memory that was freed on
+ * av_codec_close(). */
+ av_write_trailer(oc);
+
+ /* Close each codec. */
+ if (have_video)
+ close_stream(oc, &video_st);
+ if (have_audio)
+ close_stream(oc, &audio_st);
+ if (mux_meta)
+ close_stream(oc, &data_st);
+
+ if (!(fmt->flags & AVFMT_NOFILE))
+ /* Close the output file. */
+ avio_closep(&oc->pb);
+
+ /* free the stream */
+ avformat_free_context(oc);
+
+ return 0;
+}
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:59 UTC
Permalink
Alternate groups previously always generated for ISO media files. With
this addition client code can define track groups arbitrarily.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 11 +++++++++++
libavformat/movenc.c | 9 +++++++++
2 files changed, 20 insertions(+)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 4393ac6..0a5abd8 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1630,6 +1630,17 @@ enum AVPacketSideDataType {
* AVAudioTrackChannelLayout.
*/
AV_PKT_DATA_AUDIO_TRACK_CHANNEL_LAYOUT,
+
+ /**
+ * Assign alternate groups for tracks. An example of alternate
+ * groups would be audio tracks (or video tracks) that are
+ * alternative to each other. Each alternative track shares the
+ * same non-zero alternate group.
+ *
+ * The content is:
+ * uint: The alternate group of this track
+ */
+ AV_PKT_DATA_TRACK_ALTERNATE_GROUP,
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index bbd176f..abe68ce 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2565,6 +2565,8 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov,
int flags = MOV_TKHD_FLAG_IN_MOVIE;
int rotation = 0;
int group = 0;
+ int *alternate_group = NULL;
+ int alternate_group_size;

uint32_t *display_matrix = NULL;
int display_matrix_size, i;
@@ -2581,6 +2583,13 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov,
display_matrix = NULL;
}

+ if (st) {
+ alternate_group = (int*) av_stream_get_side_data(st, AV_PKT_DATA_TRACK_ALTERNATE_GROUP,
+ &alternate_group_size);
+ if (alternate_group && alternate_group_size >= sizeof(int))
+ group = *alternate_group;
+ }
+
if (track->flags & MOV_TRACK_ENABLED)
flags |= MOV_TKHD_FLAG_ENABLED;
--
2.7.4
Erkki Seppälä
2016-08-31 11:35:56 UTC
Permalink
The data is read into side packet AV_PKT_DATA_TIMED_METADATA_INFO of
format AVTimedMetadata.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/mov.c | 141 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)

diff --git a/libavformat/mov.c b/libavformat/mov.c
index 0544d13..a353a35 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -2039,6 +2039,145 @@ static int mov_rewrite_dvd_sub_extradata(AVStream *st)
return 0;
}

+static int mov_parse_urim_uri_data(AVIOContext *pb, AVStream *st, MOVMeta *meta)
+{
+ int64_t size = avio_rb32(pb);
+ uint32_t uri = avio_rb32(pb); /* "uri " */
+ avio_r8(pb); /* version */
+ avio_rb24(pb); /* flags */
+
+ if (size < 12 || size >= 1024)
+ return AVERROR(ENOMEM);
+
+ if (uri == AV_RB32("uri ")) {
+ int remaining = size - 12;
+ int64_t pos = avio_tell(pb);
+ uint32_t string_len = 0;
+
+ while (string_len < remaining && avio_r8(pb) != 0) {
+ ++string_len;
+ }
+ avio_seek(pb, pos, SEEK_SET);
+
+ if (string_len >= remaining)
+ return AVERROR_INVALIDDATA;
+
+ meta->tag = MKTAG('u', 'r', 'i', 'm');
+ meta->data = av_malloc(string_len + 1);
+ if (!meta->data)
+ return AVERROR(ENOMEM);
+
+ avio_read(pb, meta->data, string_len);
+ ((char*) meta->data)[string_len] = 0;
+ meta->length = string_len;
+ remaining -= string_len;
+ avio_r8(pb); /* read the null terminator */
+ remaining--;
+
+ avio_skip(pb, remaining);
+ } else {
+ avio_skip(pb, size - 12);
+ }
+
+ return 0;
+}
+
+static int mov_parse_urim_conf_data(AVIOContext *pb, AVStream *st, int64_t remaining, MOVMeta *meta)
+{
+ int64_t size;
+ uint32_t tag;
+ int64_t data_size;
+
+ if (remaining < 12)
+ return 0;
+
+ size = avio_rb32(pb);
+ tag = avio_rl32(pb);
+ avio_r8(pb); /* version */
+ avio_rb24(pb); /* flags */
+ remaining -= 12;
+ data_size = size - 12;
+
+ if (data_size < 0 || data_size > remaining) {
+ avio_skip(pb, remaining);
+ return 0;
+ }
+
+ meta->conf.data = av_malloc(data_size);
+ if (!meta->conf.data)
+ return AVERROR(ENOMEM);
+
+ meta->conf.tag = tag;
+ meta->conf.length = data_size;
+ avio_read(pb, meta->conf.data, data_size);
+ remaining -= data_size;
+ avio_skip(pb, remaining);
+
+ return 0;
+}
+
+static int mov_parse_urim_data(AVIOContext *pb, AVStream *st, int64_t remaining)
+{
+ int64_t pos = avio_tell(pb);
+ int ret;
+ MOVMeta meta;
+
+ memset(&meta, 0, sizeof(meta));
+
+ ret = mov_parse_urim_uri_data(pb, st, &meta);
+ if (ret)
+ return ret;
+
+ remaining -= avio_tell(pb) - pos;
+ ret = mov_parse_urim_conf_data(pb, st, remaining, &meta);
+
+ // all data has been collected; now build the actual side channel
+ // object from the collected data
+ if (ret == 0) {
+ int tmdLength = sizeof(AVTimedMetadataInfo) + meta.length + meta.conf.length;
+ AVTimedMetadataInfo *tmd = av_malloc(tmdLength);
+ if (!tmd) {
+ ret = -1;
+ } else {
+ AVPacketSideData *sd;
+ ret = av_reallocp_array(&st->side_data,
+ st->nb_side_data + 1, sizeof(*sd));
+ if (ret >= 0) {
+ char* data;
+
+ sd = st->side_data + st->nb_side_data;
+ st->nb_side_data++;
+
+ sd->type = AV_PKT_DATA_TIMED_METADATA_INFO;
+ sd->size = sizeof(*tmd) + meta.length + meta.conf.length;
+ sd->data = (uint8_t*) tmd;
+
+ tmd->meta_tag[0] = (meta.tag >> 0) & 0xff;
+ tmd->meta_tag[1] = (meta.tag >> 8) & 0xff;
+ tmd->meta_tag[2] = (meta.tag >> 16) & 0xff;
+ tmd->meta_tag[3] = (meta.tag >> 24) & 0xff;
+ tmd->meta_length = meta.length;
+ tmd->conf_tag[0] = (meta.conf.tag >> 0) & 0xff;
+ tmd->conf_tag[1] = (meta.conf.tag >> 8) & 0xff;
+ tmd->conf_tag[2] = (meta.conf.tag >> 16) & 0xff;
+ tmd->conf_tag[3] = (meta.conf.tag >> 24) & 0xff;
+ tmd->conf_length = meta.conf.length;
+ data = (char*) (tmd + 1);
+
+ memcpy(data, meta.data, meta.length);
+ data += meta.length;
+ memcpy(data, meta.conf.data, meta.conf.length);
+ } else {
+ av_freep(&tmd);
+ }
+ }
+ }
+ av_freep(&meta.data);
+ av_freep(&meta.conf.data);
+
+ return ret;
+}
+
static int mov_parse_stsd_data(MOVContext *c, AVIOContext *pb,
AVStream *st, MOVStreamContext *sc,
int64_t size)
@@ -2098,6 +2237,8 @@ FF_ENABLE_DEPRECATION_WARNINGS
}
}
}
+ } else if (st->codecpar->codec_tag == MKTAG('u', 'r', 'i', 'm')) {
+ ret = mov_parse_urim_data(pb, st, size);
} else {
/* other codec type, just skip (rtp, mp4s ...) */
avio_skip(pb, size);
--
2.7.4
Erkki Seppälä
2016-08-31 11:59:57 UTC
Permalink
Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
Changelog | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/Changelog b/Changelog
index 71abe8c..bada511 100644
--- a/Changelog
+++ b/Changelog
@@ -17,7 +17,17 @@ version <next>:
- acrusher audio filter
- bitplanenoise video filter
- floating point support in als decoder
-
+- Added support for muxing and demuxing timed metadata tracks
+ (AV_CODEC_ID_META of type AMEDIA_TYPE_DATA). With examples.
+- Added side data AV_PKT_DATA_TRACK_ALTERNATE_GROUP for exlicitly
+ setting the alternate track groups (supported by isom)
+- Added support for multiple track reference types as well as
+ multiple tracks references by such track references (supported
+ by isom for both reading and writing)
+- Added support for setting advanced audio channel track layout
+ with side data AV_PKT_DATA_AUDIO_TRACK_CHANNEL_LAYOUT
+- Added ability to set stream ids directly as track numbers in
+ ISO media files with option "use_stream_ids_as_track_ids"

version 3.1:
- DXVA2-accelerated HEVC Main10 decoding
--
2.7.4
Carl Eugen Hoyos
2016-08-31 12:58:15 UTC
Permalink
[...]

Imo, this should add one line to the Changelog and it should definitely not
remove an empty line.

Carl Eugen
Erkki Seppälä
2016-08-31 13:47:19 UTC
Permalink
---
libavutil/Makefile | 1 +
1 file changed, 1 insertion(+)

diff --git a/libavutil/Makefile b/libavutil/Makefile
index 1e06176..731b852 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -18,6 +18,7 @@ HEADERS = adler32.h \
cast5.h \
camellia.h \
channel_layout.h \
+ channel_layout_isoiec23001_8.h \
common.h \
cpu.h \
crc.h \
--
2.7.4
Erkki Seppälä
2016-09-19 13:25:24 UTC
Permalink
when the option "brand" is used. This allows custom brands to end up in
the compatible brands as well.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/movenc.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 525d103..571c2a7 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -4210,6 +4210,8 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s)

avio_wb32(pb, minor);

+ if (mov->mode == MODE_MP4 && mov->major_brand)
+ ffio_wfourcc(pb, mov->major_brand); /* write major brand as a compatible brand */
if (mov->mode == MODE_MOV)
ffio_wfourcc(pb, "qt ");
else if (mov->mode == MODE_ISM) {
--
2.7.4
Erkki Seppälä
2016-09-19 13:25:26 UTC
Permalink
Instead of one track reference, allow many, and instead of
of track reference type (ie. 'cdsc'), allow many.

In addition this patch allows client to explicitly add track references
with side packet AV_PKT_DATA_TRACK_REFERENCES containing
AVTrackReferences (which of there can many many).

Internally MOVTrack's track references can be manipulated with helper
functions ff_mov_tref* (and is used by a later patch for reading
MOVTRefs).

Multiple track references can be useful in particular with timed meta
data tracks, indicating the track is related to multiple other tracks.

This information ends up in ISO media file box 'tref' as specified by
ISO/IEC 14496-12.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 19 ++++++-
libavformat/Makefile | 4 +-
libavformat/movenc.c | 153 ++++++++++++++++++++++++++++++++++++++++++--------
libavformat/movenc.h | 5 +-
libavformat/movtref.c | 115 +++++++++++++++++++++++++++++++++++++
libavformat/movtref.h | 75 +++++++++++++++++++++++++
6 files changed, 344 insertions(+), 27 deletions(-)
create mode 100644 libavformat/movtref.c
create mode 100644 libavformat/movtref.h

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 6ac6646..56bb9b0 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1348,6 +1348,15 @@ typedef struct AVCPBProperties {
* Types and functions for working with AVPacket.
* @{
*/
+
+typedef struct AVTrackReferences {
+ int next_tref_ofs; /**< offset in bytes to the next AVTrackReferences or 0 if this is the last one*/
+ char tag[4]; /**< 4cc used for describing this */
+ int nb_tracks; /**< number of tracks */
+ int tracks[1]; /**< tracks this track refers to (contains nb_tracks entries) */
+ /** followed by an optional gap for alignment purposes and another AVTrackReferences is applicaple */
+} AVTrackReferences;
+
enum AVPacketSideDataType {
AV_PKT_DATA_PALETTE,

@@ -1525,7 +1534,15 @@ enum AVPacketSideDataType {
* should be associated with a video stream and containts data in the form
* of the AVMasteringDisplayMetadata struct.
*/
- AV_PKT_DATA_MASTERING_DISPLAY_METADATA
+ AV_PKT_DATA_MASTERING_DISPLAY_METADATA,
+
+ /**
+ * Define track references (in particular applicaple for ISO MP4
+ * files). The data is a sequence of type AVTrackReferences
+ * (including the track list that follows it), for as long as
+ * indicated by the key's length.
+ */
+ AV_PKT_DATA_TRACK_REFERENCES,
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/Makefile b/libavformat/Makefile
index fda1e17..6a393ed 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -280,10 +280,10 @@ OBJS-$(CONFIG_MLV_DEMUXER) += mlvdec.o riffdec.o
OBJS-$(CONFIG_MM_DEMUXER) += mm.o
OBJS-$(CONFIG_MMF_DEMUXER) += mmf.o
OBJS-$(CONFIG_MMF_MUXER) += mmf.o rawenc.o
-OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o replaygain.o
+OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o replaygain.o movtref.o
OBJS-$(CONFIG_MOV_MUXER) += movenc.o avc.o hevc.o vpcc.o \
movenchint.o mov_chan.o rtp.o \
- movenccenc.o rawutils.o
+ movenccenc.o rawutils.o movtref.o
OBJS-$(CONFIG_MP2_MUXER) += mp3enc.o rawenc.o id3v2enc.o
OBJS-$(CONFIG_MP3_DEMUXER) += mp3dec.o replaygain.o
OBJS-$(CONFIG_MP3_MUXER) += mp3enc.o rawenc.o id3v2enc.o
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 571c2a7..aed30dc 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2626,14 +2626,22 @@ static int mov_write_edts_tag(AVIOContext *pb, MOVMuxContext *mov,
return size;
}

-static int mov_write_tref_tag(AVIOContext *pb, MOVTrack *track)
+static int mov_write_tref_tag(AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track)
{
- avio_wb32(pb, 20); // size
+ int64_t pos = avio_tell(pb);
+ int i, j;
+ avio_wb32(pb, 0); // size
ffio_wfourcc(pb, "tref");
- avio_wb32(pb, 12); // size (subatom)
- avio_wl32(pb, track->tref_tag);
- avio_wb32(pb, track->tref_id);
- return 20;
+ for (j = 0; j < track->trefs.nb_trefs; ++j) {
+ int64_t pos_sub = avio_tell(pb);
+ MOVTRef* tref = &track->trefs.trefs[j];
+ avio_wb32(pb, 0); // size (subatom)
+ avio_wl32(pb, tref->tag);
+ for (i = 0; i < tref->nb_track_ids; i++)
+ avio_wb32(pb, tref->track_ids[i]);
+ update_size(pb, pos_sub);
+ }
+ return update_size(pb, pos);
}

// goes at the end of each track! ... Critical for PSP playback ("Incompatible data" without it)
@@ -2744,8 +2752,8 @@ static int mov_write_trak_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext
"Not writing any edit list even though one would have been required\n");
}

- if (track->tref_tag)
- mov_write_tref_tag(pb, track);
+ if (track->trefs.nb_trefs)
+ mov_write_tref_tag(pb, mov, track);

if ((ret = mov_write_mdia_tag(s, pb, mov, track)) < 0)
return ret;
@@ -3482,16 +3490,108 @@ static int mov_setup_track_ids(MOVMuxContext *mov, AVFormatContext *s)
return 0;
}

+static int mov_tref_copy_from_side_data(MOVMuxContext *mov, MOVTrack *track, AVFormatContext *s)
+{
+ int size;
+ int ret;
+ int i;
+ MOVTRef *tref;
+ int *ref_tracks = NULL;
+
+ char *ptr = (void*) av_stream_get_side_data(track->st,
+ AV_PKT_DATA_TRACK_REFERENCES,
+ &size);
+ char *end = ptr + size;
+
+ if (!ptr)
+ return 0;
+
+ ret = 0;
+ while (ptr < end) {
+ AVTrackReferences refs;
+ int *track_ids;
+ int cur_size;
+ int next_track_index;
+
+ if (ptr + sizeof(AVTrackReferences) > end) {
+ ret = AVERROR(EINVAL);
+ goto error;
+ }
+
+ // avoid any potential alignment issues by copying the struct before accessing it
+ // to a well-aligned address
+ memcpy(&refs, ptr, sizeof(refs));
+ cur_size = sizeof(AVTrackReferences) + (refs.nb_tracks - 1) * sizeof(refs.tracks[0]);
+
+ if (refs.nb_tracks <= 0 ||
+ ptr + refs.next_tref_ofs > end ||
+ ptr + cur_size > end ||
+ refs.next_tref_ofs < 0 ||
+ (refs.next_tref_ofs > 0 && refs.next_tref_ofs < cur_size)) {
+ ret = AVERROR(EINVAL);
+ goto error;
+ }
+
+ // as well as the ids
+ ref_tracks = av_malloc_array(refs.nb_tracks, sizeof(refs.tracks[0]));
+ if (!ref_tracks)
+ goto error;
+ memcpy(ref_tracks, &((AVTrackReferences*) ptr)->tracks, sizeof(refs.tracks[0]) * refs.nb_tracks);
+
+ ret = ff_mov_tref_find_or_add(&track->trefs,
+ MKTAG(refs.tag[0], refs.tag[1], refs.tag[2], refs.tag[3]),
+ &tref);
+ if (ret < 0)
+ goto error;
+
+ ret = ff_mov_tref_alloc(tref, refs.nb_tracks, &track_ids);
+ if (ret < 0)
+ goto error;
+
+ next_track_index = 0;
+ for (i = 0; i < refs.nb_tracks; i++) {
+ int tref_stream_id = ref_tracks[i];
+ int stream_idx;
+ int found = 0;
+ for (stream_idx = 0; stream_idx < mov->nb_streams && !found; ++stream_idx)
+ if (mov->tracks[stream_idx].st &&
+ mov->tracks[stream_idx].st->id == tref_stream_id) {
+ track_ids[next_track_index] = mov->tracks[stream_idx].track_id;
+ found = 1;
+ }
+ if (found)
+ next_track_index++;
+ else
+ tref->nb_track_ids--;
+ }
+
+ ptr += refs.next_tref_ofs;
+
+ av_free(ref_tracks);
+ if (refs.next_tref_ofs == 0)
+ break;
+ }
+ return 0;
+error:
+ av_free(ref_tracks);
+ return ret;
+}
+
static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
AVFormatContext *s)
{
int i;
int64_t pos = avio_tell(pb);
+ int ret;
avio_wb32(pb, 0); /* size placeholder*/
ffio_wfourcc(pb, "moov");

mov_setup_track_ids(mov, s);

+ for (i = 0; i < mov->nb_streams; i++)
+ if (mov->tracks[i].st)
+ mov_tref_copy_from_side_data(mov, &mov->tracks[i], s);
+
for (i = 0; i < mov->nb_streams; i++) {
if (mov->tracks[i].entry <= 0 && !(mov->flags & FF_MOV_FLAG_FRAGMENT))
continue;
@@ -3503,34 +3603,42 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
}

if (mov->chapter_track)
- for (i = 0; i < s->nb_streams; i++) {
- mov->tracks[i].tref_tag = MKTAG('c','h','a','p');
- mov->tracks[i].tref_id = mov->tracks[mov->chapter_track].track_id;
- }
+ for (i = 0; i < s->nb_streams; i++)
+ if (!ff_codec_get_id(ff_codec_metadata_tags, mov->tracks[i].tag)) {
+ ret = ff_mov_tref_add_one_track(&mov->tracks[i].trefs, MKTAG('c','h','a','p'), mov->chapter_track);
+ if (ret < 0)
+ return ret;
+ }
for (i = 0; i < mov->nb_streams; i++) {
MOVTrack *track = &mov->tracks[i];
if (track->tag == MKTAG('r','t','p',' ')) {
- track->tref_tag = MKTAG('h','i','n','t');
- track->tref_id = mov->tracks[track->src_track].track_id;
+ ret = ff_mov_tref_add_one_track(&mov->tracks[i].trefs, MKTAG('h','i','n','t'), mov->tracks[track->src_track].track_id);
+ if (ret < 0)
+ return ret;
} else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) {
int * fallback, size;
fallback = (int*)av_stream_get_side_data(track->st,
AV_PKT_DATA_FALLBACK_TRACK,
&size);
- if (fallback != NULL && size == sizeof(int)) {
- if (*fallback >= 0 && *fallback < mov->nb_streams) {
- track->tref_tag = MKTAG('f','a','l','l');
- track->tref_id = mov->tracks[*fallback].track_id;
- }
+ if (fallback != NULL && size == sizeof(int) &&
+ *fallback >= 0 && *fallback < mov->nb_streams) {
+ ret = ff_mov_tref_add_one_track(&track->trefs, MKTAG('f','a','l','l'), *fallback);
+ if (ret < 0)
+ return ret;
}
}
}
for (i = 0; i < mov->nb_streams; i++) {
if (mov->tracks[i].tag == MKTAG('t','m','c','d')) {
int src_trk = mov->tracks[i].src_track;
- mov->tracks[src_trk].tref_tag = mov->tracks[i].tag;
- mov->tracks[src_trk].tref_id = mov->tracks[i].track_id;
- //src_trk may have a different timescale than the tmcd track
+
+ ret = ff_mov_tref_add_one_track(&mov->tracks[src_trk].trefs,
+ mov->tracks[i].tag,
+ mov->tracks[i].track_id);
+ if (ret < 0)
+ return ret;
+
+ //src_trk may have a different timescale than the tmcd track
mov->tracks[i].track_duration = av_rescale(mov->tracks[src_trk].track_duration,
mov->tracks[i].timescale,
mov->tracks[src_trk].timescale);
@@ -5340,6 +5448,7 @@ static void mov_free(AVFormatContext *s)
av_freep(&mov->tracks[i].vos_data);

ff_mov_cenc_free(&mov->tracks[i].cenc);
+ ff_mov_tref_free(&mov->tracks[i].trefs);
}

av_freep(&mov->tracks);
diff --git a/libavformat/movenc.h b/libavformat/movenc.h
index ea76e39..700d8d5 100644
--- a/libavformat/movenc.h
+++ b/libavformat/movenc.h
@@ -26,6 +26,7 @@

#include "avformat.h"
#include "movenccenc.h"
+#include "movtref.h"

#define MOV_FRAG_INFO_ALLOC_INCREMENT 64
#define MOV_INDEX_CLUSTER_SIZE 1024
@@ -110,8 +111,8 @@ typedef struct MOVTrack {
unsigned cluster_capacity;
int audio_vbr;
int height; ///< active picture (w/o VBI) height for D-10/IMX
- uint32_t tref_tag;
- int tref_id; ///< trackID of the referenced track
+
+ MOVTRefs trefs;
int64_t start_dts;
int64_t start_cts;
int64_t end_pts;
diff --git a/libavformat/movtref.c b/libavformat/movtref.c
new file mode 100644
index 0000000..dd6059c
--- /dev/null
+++ b/libavformat/movtref.c
@@ -0,0 +1,115 @@
+/*
+ * ISO base media file format track references
+ * Copyright (c) 2016 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <stdint.h>
+#include <inttypes.h>
+#include <stdlib.h>
+
+#include "movtref.h"
+#include "libavutil/mem.h"
+
+MOVTRef *ff_mov_tref_find(MOVTRefs *trefs, uint32_t tag)
+{
+ int i;
+ MOVTRef *tref = NULL;
+
+ for (i = 0; i < trefs->nb_trefs && !tref; ++i) {
+ if (trefs->trefs[i].tag == tag) {
+ tref = trefs->trefs + i;
+ }
+ }
+
+ return tref;
+}
+
+int ff_mov_tref_alloc(MOVTRef *tref, int n, int **track_ids_ret)
+{
+ int ret = av_reallocp_array(&tref->track_ids, tref->nb_track_ids + n, sizeof(tref->track_ids));
+ *track_ids_ret = NULL;
+ if (ret >= 0) {
+ *track_ids_ret = tref->track_ids + tref->nb_track_ids;
+ tref->nb_track_ids += n;
+ }
+ return ret;
+}
+
+int ff_mov_tref_find_or_add(MOVTRefs *trefs, uint32_t tag, MOVTRef **tref_ret)
+{
+ int ret;
+ int i;
+ MOVTRef *tref = ff_mov_tref_find(trefs, tag);
+ *tref_ret = NULL;
+
+ for (i = 0; i < trefs->nb_trefs && !tref; ++i) {
+ if (trefs->trefs[i].tag == tag) {
+ tref = trefs->trefs + i;
+ }
+ }
+
+ if (!tref) {
+ ret = av_reallocp_array(&trefs->trefs, trefs->nb_trefs + 1, sizeof(*trefs->trefs));
+ if (ret < 0)
+ return ret;
+ tref = trefs->trefs + trefs->nb_trefs;
+ trefs->nb_trefs++;
+ tref->tag = tag;
+ tref->track_ids = NULL;
+ tref->nb_track_ids = 0;
+ }
+
+ *tref_ret = tref;
+ return 0;
+}
+
+int ff_mov_tref_add_one_track(MOVTRefs *trefs, unsigned tag, int track_id)
+{
+ int ret;
+ MOVTRef *tref;
+ int *ids;
+ int track_id_exists = 0;
+ int i;
+
+ ret = ff_mov_tref_find_or_add(trefs, tag, &tref);
+ if (ret < 0)
+ return ret;
+
+ for (i = 0; i < tref->nb_track_ids && !track_id_exists; i++) {
+ if (tref->track_ids[i] == track_id) {
+ track_id_exists = 1;
+ }
+ }
+
+ if (!track_id_exists) {
+ ret = ff_mov_tref_alloc(tref, 1, &ids);
+ if (ret >= 0) {
+ *ids = track_id;
+ }
+ }
+ return ret;
+}
+
+void ff_mov_tref_free(MOVTRefs *trefs)
+{
+ int i;
+ for (i = 0; i < trefs->nb_trefs; ++i)
+ av_freep(&trefs->trefs[i].track_ids);
+ av_freep(&trefs->trefs);
+}
diff --git a/libavformat/movtref.h b/libavformat/movtref.h
new file mode 100644
index 0000000..091f467
--- /dev/null
+++ b/libavformat/movtref.h
@@ -0,0 +1,75 @@
+/*
+ * ISO base media file format track references
+ * Copyright (c) 2016 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef AVFORMAT_MOVTREF_H
+#define AVFORMAT_MOVTREF_H
+
+typedef struct MOVTRef {
+ uint32_t tag;
+ int nb_track_ids;
+ int* track_ids;
+} MOVTRef;
+
+typedef struct MOVTRefs {
+ int nb_trefs;
+ MOVTRef *trefs;
+} MOVTRefs;
+
+/**
+ * @brief Finds a track reference of certain tag from a track; if not
+ * found, return NULL
+ *
+ * @param track The track to search from
+ * @param tag The tag (4cc) to search for
+ * @return a MOVTRef describing the track reference or NULL if not found.
+ */
+MOVTRef *ff_mov_tref_find(MOVTRefs *trefs, uint32_t tag);
+
+/**
+ * @brief Allocate space for n more tracks (and increase nb_ids accordingly) returning
+ * the pointer in *track_ids
+ *
+ * @param tref The track reference to allocate tracks to
+ * @param n Number of tracks to add
+ * @param track_ids_ret A non-null pointer where the pointer to the first free track
+ * id is returned to
+ * @return Returns a value <0 on error, otherwise 0.
+ */
+int ff_mov_tref_alloc(MOVTRef *tref, int n, int **track_ids_ret);
+
+/**
+ * @brief Finds an existing MOVTRef for a track for the given tag, or
+ * creates a new one if it is missing. Returns the tref in tref_ret.
+ */
+
+int ff_mov_tref_find_or_add(MOVTRefs *trefs, uint32_t tag, MOVTRef **tref_ret);
+
+/**
+ * Adds one tref track reference with given track id if it doesn't
+ * already exist
+ */
+int ff_mov_tref_add_one_track(MOVTRefs *trefs, unsigned tag, int track_id);
+
+/**
+ * Release MOVTRefs
+ */
+void ff_mov_tref_free(MOVTRefs *trefs);
+
+#endif /* AVFORMAT_MOVTREF_H */
--
2.7.4
Erkki Seppälä
2016-09-19 13:25:27 UTC
Permalink
This can be useful in particular with timed meta data tracks related
to multiple tracks.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.h | 3 ++
libavformat/mov.c | 119 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 121 insertions(+), 1 deletion(-)

diff --git a/libavformat/isom.h b/libavformat/isom.h
index df6c15a..609b7b7 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -27,6 +27,8 @@
#include "avio.h"
#include "internal.h"
#include "dv.h"
+#include "movmeta.h"
+#include "movtref.h"

/* isom.c */
extern const AVCodecTag ff_mp4_obj_type[];
@@ -167,6 +169,7 @@ typedef struct MOVStreamContext {
int start_pad; ///< amount of samples to skip due to enc-dec delay
unsigned int rap_group_count;
MOVSbgp *rap_group;
+ MOVTRefs trefs;

int nb_frames_for_fps;
int64_t duration_for_fps;
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 1bc3800..ff4c91c 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -3157,6 +3157,121 @@ static void fix_timescale(MOVContext *c, MOVStreamContext *sc)
}
}

+static int mov_read_tref_subatom(MOVContext *c, AVIOContext *pb, MOVTRefs *trefs)
+{
+ uint32_t tref_tag;
+ MOVTRef *tref;
+ int tref_ids_size;
+ int *tref_ids;
+ int ret;
+
+ tref_ids_size = avio_rb32(pb);
+ if (tref_ids_size % 4 != 0)
+ return AVERROR_INVALIDDATA;
+ tref_ids_size -= 4;
+
+ tref_tag = avio_rl32(pb);
+ tref_ids_size -= 4;
+ if (tref_ids_size < 4)
+ return AVERROR_INVALIDDATA;
+
+ ret = ff_mov_tref_find_or_add(trefs, tref_tag, &tref);
+ if (ret != 0)
+ return ret;
+
+ ff_mov_tref_alloc(tref, tref_ids_size / 4, &tref_ids);
+ if (ret != 0)
+ return ret;
+
+ while (tref_ids_size) {
+ *tref_ids = avio_rb32(pb);
+ ++tref_ids;
+ tref_ids_size -= 4;
+ }
+
+ return ret;
+}
+
+static int mov_tref_copy_to_side_data(AVStream *st, MOVTRefs *trefs)
+{
+ if (trefs->nb_trefs) {
+ int i;
+ char *trefs_side_ptr;
+ int end_offset = 0;
+ int offset = 0;
+ AVTrackReferences *trefs_side_prev = NULL;
+
+ for (i = 0; i < trefs->nb_trefs; ++i) {
+ MOVTRef *tref = &trefs->trefs[i];
+ if (tref->nb_track_ids > 0)
+ // Ensure the returned data is easy to access without
+ // worrying about alignment, even if it wastes some memory
+ end_offset = FFALIGN(end_offset + sizeof(AVTrackReferences) +
+ sizeof(int) * (tref->nb_track_ids - 1),
+ sizeof(AVTrackReferences));
+ }
+
+ trefs_side_ptr = (void*) av_stream_new_side_data(st,
+ AV_PKT_DATA_TRACK_REFERENCES,
+ end_offset);
+ if (!trefs_side_ptr)
+ return AVERROR(ENOMEM);
+
+ for (i = 0; i < trefs->nb_trefs; ++i) {
+ MOVTRef *tref = &trefs->trefs[i];
+ if (tref->nb_track_ids > 0) {
+ AVTrackReferences *trefs_side = (AVTrackReferences*) (trefs_side_ptr + offset);
+ int *trefs_tracks;
+
+ trefs_side->nb_tracks = tref->nb_track_ids;
+ trefs_side->tag[0] = (tref->tag >> 0) & 0xff;
+ trefs_side->tag[1] = (tref->tag >> 8) & 0xff;
+ trefs_side->tag[2] = (tref->tag >> 16) & 0xff;
+ trefs_side->tag[3] = (tref->tag >> 24) & 0xff;
+ trefs_tracks = trefs_side->tracks;
+ for (i = 0; i < tref->nb_track_ids; ++i) {
+ trefs_tracks[i] = tref->track_ids[i];
+ }
+
+ if (trefs_side_prev)
+ trefs_side_prev->next_tref_ofs = (char*) trefs_side - (char*) trefs_side_prev;
+ trefs_side_prev = trefs_side;
+ offset = FFALIGN(end_offset + sizeof(AVTrackReferences) +
+ sizeof(int) * (tref->nb_track_ids - 1),
+ sizeof(AVTrackReferences));
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int mov_read_tref(MOVContext *c, AVIOContext *pb, MOVAtom atom)
+{
+ AVStream *st;
+ MOVStreamContext *sc;
+ int64_t end = avio_tell(pb) + atom.size;
+
+ if (c->fc->nb_streams < 1)
+ return 0;
+
+ st = c->fc->streams[c->fc->nb_streams-1];
+ sc = st->priv_data;
+
+ if (atom.size < 12) {
+ return mov_read_default(c, pb, atom);
+ } else {
+ int ret = 0;
+ while (end - avio_tell(pb) > 0 && ret == 0) {
+ ret = mov_read_tref_subatom(c, pb, &sc->trefs);
+ }
+ mov_tref_copy_to_side_data(st, &sc->trefs);
+ avio_seek(pb, end, SEEK_SET);
+ }
+
+ return 0;
+}
+
static int mov_read_trak(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
AVStream *st;
@@ -4414,7 +4529,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = {
{ MKTAG('t','f','h','d'), mov_read_tfhd }, /* track fragment header */
{ MKTAG('t','r','a','k'), mov_read_trak },
{ MKTAG('t','r','a','f'), mov_read_default },
-{ MKTAG('t','r','e','f'), mov_read_default },
+{ MKTAG('t','r','e','f'), mov_read_tref },
{ MKTAG('t','m','c','d'), mov_read_tmcd },
{ MKTAG('c','h','a','p'), mov_read_chap },
{ MKTAG('t','r','e','x'), mov_read_trex },
@@ -4832,6 +4947,8 @@ static int mov_read_close(AVFormatContext *s)
av_freep(&sc->cenc.auxiliary_info);
av_freep(&sc->cenc.auxiliary_info_sizes);
av_aes_ctr_free(sc->cenc.aes_ctr);
+
+ ff_mov_tref_free(&sc->trefs);
}

if (mov->dv_demux) {
--
2.7.4
Erkki Seppälä
2016-09-19 13:25:25 UTC
Permalink
---
Changelog | 2 ++
1 file changed, 2 insertions(+)

diff --git a/Changelog b/Changelog
index 71abe8c..e1c8010 100644
--- a/Changelog
+++ b/Changelog
@@ -18,6 +18,8 @@ version <next>:
- bitplanenoise video filter
- floating point support in als decoder

+- Custom major brand (the "brand" option) of MPEG4 is written
+ as a compatible brand as well.

version 3.1:
- DXVA2-accelerated HEVC Main10 decoding
--
2.7.4
Erkki Seppälä
2016-09-19 13:25:32 UTC
Permalink
It has the codec id AV_CODEC_ID_META and type of AVMEDIA_TYPE_DATA.

This codec basically passes the data forward and is used for referring
timed meta data tracks by a codec. It is useful for dealing with the
metadata in a similar way as other kinds of codecs.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
configure | 4 ++--
libavcodec/Makefile | 2 ++
libavcodec/allcodecs.c | 3 +++
libavcodec/avcodec.h | 1 +
libavcodec/codec_desc.c | 8 +++++++
libavcodec/metadec.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++
libavcodec/metaenc.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++
7 files changed, 133 insertions(+), 2 deletions(-)
create mode 100644 libavcodec/metadec.c
create mode 100644 libavcodec/metaenc.c

diff --git a/configure b/configure
index 9b92426..ded5452 100755
--- a/configure
+++ b/configure
@@ -2848,11 +2848,11 @@ matroska_demuxer_select="iso_media riffdec"
matroska_demuxer_suggest="bzlib lzo zlib"
matroska_muxer_select="iso_media riffenc"
mmf_muxer_select="riffenc"
-mov_demuxer_select="iso_media riffdec"
+mov_demuxer_select="iso_media riffdec meta_decoder"
mov_demuxer_suggest="zlib"
mov_muxer_select="iso_media riffenc rtpenc_chain"
mp3_demuxer_select="mpegaudio_parser"
-mp4_muxer_select="mov_muxer"
+mp4_muxer_select="mov_muxer meta_encoder"
mpegts_demuxer_select="iso_media"
mpegts_muxer_select="adts_muxer latm_muxer"
mpegtsraw_demuxer_select="mpegts_demuxer"
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index a6e79ce..534f201 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -757,6 +757,8 @@ OBJS-$(CONFIG_ADPCM_VIMA_DECODER) += vima.o adpcm_data.o
OBJS-$(CONFIG_ADPCM_XA_DECODER) += adpcm.o adpcm_data.o
OBJS-$(CONFIG_ADPCM_YAMAHA_DECODER) += adpcm.o adpcm_data.o
OBJS-$(CONFIG_ADPCM_YAMAHA_ENCODER) += adpcmenc.o adpcm_data.o
+OBJS-$(CONFIG_META_ENCODER) += metaenc.o
+OBJS-$(CONFIG_META_DECODER) += metadec.o

# hardware accelerators
OBJS-$(CONFIG_D3D11VA) += dxva2.o
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 4c6b94e..30d0243 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -687,4 +687,7 @@ void avcodec_register_all(void)
REGISTER_PARSER(VP3, vp3);
REGISTER_PARSER(VP8, vp8);
REGISTER_PARSER(VP9, vp9);
+
+ /* data, meta data */
+ REGISTER_ENCDEC(META, meta);
}
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 3be54d4..4219a9f 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -639,6 +639,7 @@ enum AVCodecID {
AV_CODEC_ID_DVD_NAV,
AV_CODEC_ID_TIMED_ID3,
AV_CODEC_ID_BIN_DATA,
+ AV_CODEC_ID_META,


AV_CODEC_ID_PROBE = 0x19000, ///< codec_id is not known (like AV_CODEC_ID_NONE) but lavf should attempt to identify it
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 24948ca..e85b51d 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -2965,6 +2965,14 @@ static const AVCodecDescriptor codec_descriptors[] = {
.mime_types= MT("application/octet-stream"),
},

+ {
+ .id = AV_CODEC_ID_META,
+ .type = AVMEDIA_TYPE_DATA,
+ .name = "meta_data",
+ .long_name = NULL_IF_CONFIG_SMALL("binary data"),
+ .mime_types= MT("application/octet-stream"),
+ },
+
/* deprecated codec ids */
};

diff --git a/libavcodec/metadec.c b/libavcodec/metadec.c
new file mode 100644
index 0000000..2c1d461
--- /dev/null
+++ b/libavcodec/metadec.c
@@ -0,0 +1,59 @@
+/*
+ * Copyright (c) 2015-2016 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "internal.h"
+#include "libavformat/avio.h"
+
+static int meta_decode(AVCodecContext *avctx, void *data, int *got_packet_ptr, AVPacket *avpkt)
+{
+ AVFrame *metadata = data;
+
+ *got_packet_ptr = 0;
+
+ av_frame_unref(metadata);
+ if (avpkt->buf) {
+ metadata->buf[0] = av_buffer_ref(avpkt->buf);
+ if (!metadata->buf[0])
+ return AVERROR(ENOMEM);
+ metadata->buf[0]->size = avpkt->size;
+ } else {
+ metadata->buf[0] = av_buffer_alloc(avpkt->size);
+ if (!metadata->buf[0])
+ return AVERROR(ENOMEM);
+ metadata->data[0] = metadata->buf[0]->data;
+ memcpy(metadata->data[0], avpkt->data, avpkt->size);
+ }
+
+ metadata->nb_samples = avpkt->size;
+ metadata->pts = avpkt->pts;
+ metadata->pkt_pts = avpkt->pts;
+ metadata->pkt_dts = avpkt->dts;
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+AVCodec ff_meta_decoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Timed Metadata Decoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .decode = meta_decode,
+};
diff --git a/libavcodec/metaenc.c b/libavcodec/metaenc.c
new file mode 100644
index 0000000..58c9c39
--- /dev/null
+++ b/libavcodec/metaenc.c
@@ -0,0 +1,58 @@
+/*
+ * Copyright (c) 2015 Erkki Seppälä <***@nokia.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/intreadwrite.h"
+#include "internal.h"
+#include "libavutil/opt.h"
+#include "libavformat/avio.h"
+
+static int meta_encode(AVCodecContext *avctx, AVPacket *avpkt, const AVFrame *frame,
+ int *got_packet_ptr)
+{
+ *got_packet_ptr = 0;
+
+ av_packet_unref(avpkt);
+ if (frame->buf) {
+ avpkt->buf = av_buffer_ref(frame->buf[0]);
+ if (!avpkt->buf)
+ return AVERROR(ENOMEM);
+ avpkt->data = frame->data[0];
+ } else {
+ avpkt->buf = av_buffer_alloc(frame->nb_samples);
+ if (!avpkt->buf)
+ return AVERROR(ENOMEM);
+ avpkt->data = avpkt->buf->data;
+ memcpy(avpkt->data, frame->data[0], frame->nb_samples);
+ }
+ avpkt->size = frame->nb_samples;
+ avpkt->pts = frame->pts;
+ avpkt->dts = frame->pts;
+ *got_packet_ptr = 1;
+
+ return 0;
+}
+
+AVCodec ff_meta_encoder = {
+ .name = "meta",
+ .long_name = NULL_IF_CONFIG_SMALL("Timed Metadata Encoder"),
+ .type = AVMEDIA_TYPE_DATA,
+ .id = AV_CODEC_ID_META,
+ .encode2 = meta_encode,
+};
--
2.7.4
Erkki Seppälä
2016-09-19 13:25:33 UTC
Permalink
This includes creating an AVCodecTag table ff_codec_metadata_tags as
there are for video, audio and subtitles. The tag table is used for
mov-compatiblity.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavformat/isom.c | 5 +++++
libavformat/isom.h | 1 +
libavformat/movenc.c | 6 ++++++
3 files changed, 12 insertions(+)

diff --git a/libavformat/isom.c b/libavformat/isom.c
index cb457dd..1a90d00 100644
--- a/libavformat/isom.c
+++ b/libavformat/isom.c
@@ -355,6 +355,11 @@ const AVCodecTag ff_codec_movsubtitle_tags[] = {
{ AV_CODEC_ID_NONE, 0 },
};

+const AVCodecTag ff_codec_metadata_tags[] = {
+ { AV_CODEC_ID_META, MKTAG('m', 'e', 't', 'a') },
+ { AV_CODEC_ID_NONE, 0 },
+};
+
/* map numeric codes from mdhd atom to ISO 639 */
/* cf. QTFileFormat.pdf p253, qtff.pdf p205 */
/* http://developer.apple.com/documentation/mac/Text/Text-368.html */
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 7b521d8..89b15ea 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -35,6 +35,7 @@ extern const AVCodecTag ff_mp4_obj_type[];
extern const AVCodecTag ff_codec_movvideo_tags[];
extern const AVCodecTag ff_codec_movaudio_tags[];
extern const AVCodecTag ff_codec_movsubtitle_tags[];
+extern const AVCodecTag ff_codec_metadata_tags[];

int ff_mov_iso639_to_lang(const char lang[4], int mp4);
int ff_mov_lang_to_iso639(unsigned code, char to[4]);
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index b2e87ed..a6f234e 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -1470,6 +1470,8 @@ static int mov_get_codec_tag(AVFormatContext *s, MOVTrack *track)
}
} else if (track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)
tag = ff_codec_get_tag(ff_codec_movsubtitle_tags, track->par->codec_id);
+ else if (track->par->codec_type == AVMEDIA_TYPE_DATA)
+ tag = ff_codec_get_tag(ff_codec_metadata_tags, track->par->codec_id);
}

return tag;
@@ -2242,6 +2244,9 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra
} else if (track->par->codec_tag == MKTAG('t','m','c','d')) {
hdlr_type = "tmcd";
descr = "TimeCodeHandler";
+ } else if (track->par->codec_type == AVMEDIA_TYPE_DATA) {
+ hdlr_type = "meta";
+ descr = "DataHandler";
} else {
char tag_buf[32];
av_get_codec_tag_string(tag_buf, sizeof(tag_buf),
@@ -5425,6 +5430,7 @@ static void enable_tracks(AVFormatContext *s)
case AVMEDIA_TYPE_VIDEO:
case AVMEDIA_TYPE_AUDIO:
case AVMEDIA_TYPE_SUBTITLE:
+ case AVMEDIA_TYPE_DATA:
if (enabled[i] > 1)
mov->per_stream_grouping = 1;
if (!enabled[i] && first[i] >= 0)
--
2.7.4
Erkki Seppälä
2016-09-19 13:25:29 UTC
Permalink
Alternate groups previously always generated for ISO media files. With
this addition client code can define track groups arbitrarily.

Signed-off-by: Erkki Seppälä <***@nokia.com>
Signed-off-by: OZOPlayer <***@nokia.com>
---
libavcodec/avcodec.h | 11 +++++++++++
libavformat/movenc.c | 9 +++++++++
2 files changed, 20 insertions(+)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 56bb9b0..3be54d4 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1543,6 +1543,17 @@ enum AVPacketSideDataType {
* indicated by the key's length.
*/
AV_PKT_DATA_TRACK_REFERENCES,
+
+ /**
+ * Assign alternate groups for tracks. An example of alternate
+ * groups would be audio tracks (or video tracks) that are
+ * alternative to each other. Each alternative track shares the
+ * same non-zero alternate group.
+ *
+ * The content is:
+ * uint: The alternate group of this track
+ */
+ AV_PKT_DATA_TRACK_ALTERNATE_GROUP,
};

#define AV_PKT_DATA_QUALITY_FACTOR AV_PKT_DATA_QUALITY_STATS //DEPRECATED
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index aed30dc..b2e87ed 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2407,6 +2407,8 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov,
int flags = MOV_TKHD_FLAG_IN_MOVIE;
int rotation = 0;
int group = 0;
+ int *alternate_group = NULL;
+ int alternate_group_size;

uint32_t *display_matrix = NULL;
int display_matrix_size, i;
@@ -2423,6 +2425,13 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov,
display_matrix = NULL;
}

+ if (st) {
+ alternate_group = (int*) av_stream_get_side_data(st, AV_PKT_DATA_TRACK_ALTERNATE_GROUP,
+ &alternate_group_size);
+ if (alternate_group && alternate_group_size >= sizeof(int))
+ group = *alternate_group;
+ }
+
if (track->flags & MOV_TRACK_ENABLED)
flags |= MOV_TKHD_FLAG_ENABLED;
--
2.7.4
Loading...