Discussion:
dcp encoding + new pixel format request/sponsorship
(too old to reply)
Michaël Cinquin
2013-01-21 18:15:02 UTC
Permalink
Hi

I just sent my first pull request from my first commits on GitHub : michaelcinquin.com:master.

The pull request is about adding an rgb2xyz av_filter, which enables the processing of colors for digital cinema, from sRGB colorspace to XYZ, based on a 3x3 color matrix. This will hopefully make FFmpeg more useful for DCP encoding.

There is one last show-stopper for making FFmpeg the main tool of opensource DCP encoding.

FFmpeg does not support saving to RGB 3x12 bpp formats : this means we cannot output the 12-bit files necessary for Digital Cinema. Without a new pixel format, one has to save TIFFs out of ffmpeg, then encode them to j2K with another tool, which as far as I know cannot be piped.

Could I get help or sponsor the development of a new pixel format that will enable libopenjpeg to save 12 bips per channel J2K ?

best regards

Michael Cinquin
www.michaelcinquin.com

PS : this is my first contribution to opensource software, and my first post on a developper mailing list. Please don't be too harsh if I'm doing something wrong
Paul B Mahol
2013-01-21 21:12:02 UTC
Permalink
Post by Michaël Cinquin
Hi
michaelcinquin.com:master.
The pull request is about adding an rgb2xyz av_filter, which enables the
processing of colors for digital cinema, from sRGB colorspace to XYZ, based
on a 3x3 color matrix. This will hopefully make FFmpeg more useful for DCP
encoding.
It would be best if this is in libswscale instead ....
You are using tabs which is not allowed, please find some time to read:

http://ffmpeg.org/developer.html
Post by Michaël Cinquin
There is one last show-stopper for making FFmpeg the main tool of opensource DCP encoding.
FFmpeg does not support saving to RGB 3x12 bpp formats : this means we
cannot output the 12-bit files necessary for Digital Cinema. Without a new
pixel format, one has to save TIFFs out of ffmpeg, then encode them to j2K
with another tool, which as far as I know cannot be piped.
Could I get help or sponsor the development of a new pixel format that will
enable libopenjpeg to save 12 bips per channel J2K ?
best regards
Michael Cinquin
www.michaelcinquin.com
PS : this is my first contribution to opensource software, and my first post
on a developper mailing list. Please don't be too harsh if I'm doing
something wrong
_______________________________________________
ffmpeg-devel mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Michaël Cinquin
2013-01-21 22:03:48 UTC
Permalink
It would be best if this is in libswscale instead ….
that's way above my skills, sorry
that was within my skill and has been committed.

Hope I'll get answers about this new pixel format.
Reimar Döffinger
2013-01-22 08:12:15 UTC
Permalink
Post by Michaël Cinquin
Hi
I just sent my first pull request from my first commits on GitHub : michaelcinquin.com:master.
The pull request is about adding an rgb2xyz av_filter, which enables the processing of colors for digital cinema, from sRGB colorspace to XYZ, based on a 3x3 color matrix. This will hopefully make FFmpeg more useful for DCP encoding.
There is one last show-stopper for making FFmpeg the main tool of opensource DCP encoding.
FFmpeg does not support saving to RGB 3x12 bpp formats : this means we cannot output the 12-bit files necessary for Digital Cinema. Without a new pixel format, one has to save TIFFs out of ffmpeg, then encode them to j2K with another tool, which as far as I know cannot be piped.
Could I get help or sponsor the development of a new pixel format that will enable libopenjpeg to save 12 bips per channel J2K ?
Is there any reason to add a 12 bit per component format instead of using the 16 bit one and telling the encoder it should discard 4 bit?
I believe the reason that the 9 and 10 bit formats exist is that they are used for reference frames and using the 16 bit format for the reference frames would slow things down a good bit.
Michaël Cinquin
2013-01-22 08:47:19 UTC
Permalink
Post by Reimar Döffinger
Is there any reason to add a 12 bit per component format instead of using the 16 bit one and telling the encoder it should discard 4 bit?
Indeed, I think it could be fine to ask the encoder to discard 4 bits… as long as the end result image are 12 bpc, and not 16 bpc with information clipped to 12 bit. How could this be done ?
Michael Bradshaw
2013-01-22 14:30:13 UTC
Permalink
On Tue, Jan 22, 2013 at 1:12 AM, Reimar Döffinger
Post by Michaël Cinquin
Post by Michaël Cinquin
Hi
michaelcinquin.com:master.
Post by Michaël Cinquin
The pull request is about adding an rgb2xyz av_filter, which enables the
processing of colors for digital cinema, from sRGB colorspace to XYZ, based
on a 3x3 color matrix. This will hopefully make FFmpeg more useful for DCP
encoding.
Post by Michaël Cinquin
There is one last show-stopper for making FFmpeg the main tool of
opensource DCP encoding.
Post by Michaël Cinquin
FFmpeg does not support saving to RGB 3x12 bpp formats : this means we
cannot output the 12-bit files necessary for Digital Cinema. Without a new
pixel format, one has to save TIFFs out of ffmpeg, then encode them to j2K
with another tool, which as far as I know cannot be piped.
Post by Michaël Cinquin
Could I get help or sponsor the development of a new pixel format that
will enable libopenjpeg to save 12 bips per channel J2K ?
Is there any reason to add a 12 bit per component format instead of using
the 16 bit one and telling the encoder it should discard 4 bit?
Out of curiosity, how would that work? Which 4 bits do you discard?
openjpeg itself can, in theory, save 12-bit RGB, but lavc would provide
frames in 16-bit RGB; simply discarding bits would give improper results. A
proper/scaling 16-bit to 12-bit conversion would need to be done, and it
hardly seems like the encoder is the right place to put that.

We also wouldn't be able to decode it without a 12-bit pixel format.

--Michael
Carl Eugen Hoyos
2013-01-22 14:57:38 UTC
Permalink
Post by Michael Bradshaw
Post by Reimar Döffinger
Is there any reason to add a 12 bit per component format
instead of using the 16 bit one and telling the encoder
it should discard 4 bit?
Out of curiosity, how would that work? Which 4 bits do you discard?
I believe when we "scale" from 16 to 8 bits, we simply discard
the 8 least significant bit, the same can be done for the
16 -> 12 case.
Post by Michael Bradshaw
openjpeg itself can, in theory, save 12-bit RGB, but lavc
would provide frames in 16-bit RGB; simply discarding bits
would give improper results. A proper/scaling 16-bit to
12-bit conversion would need to be done, and it
hardly seems like the encoder is the right place to put that.
It would simplify things to do it like that.
Post by Michael Bradshaw
We also wouldn't be able to decode it without a 12-bit pixel format.
The decoder would simply shift the most significant 4 bits
into the least significant bits and set bits_per_coded_sample
to 12.

Is "Digital Cinema" JPEG 2000?

Carl Eugen
Michael Bradshaw
2013-01-22 15:08:22 UTC
Permalink
Post by Carl Eugen Hoyos
Post by Michael Bradshaw
Post by Reimar Döffinger
Is there any reason to add a 12 bit per component format
instead of using the 16 bit one and telling the encoder
it should discard 4 bit?
Out of curiosity, how would that work? Which 4 bits do you discard?
I believe when we "scale" from 16 to 8 bits, we simply discard
the 8 least significant bit, the same can be done for the
16 -> 12 case.
Post by Michael Bradshaw
openjpeg itself can, in theory, save 12-bit RGB, but lavc
would provide frames in 16-bit RGB; simply discarding bits
would give improper results. A proper/scaling 16-bit to
12-bit conversion would need to be done, and it
hardly seems like the encoder is the right place to put that.
It would simplify things to do it like that.
Post by Michael Bradshaw
We also wouldn't be able to decode it without a 12-bit pixel format.
The decoder would simply shift the most significant 4 bits
into the least significant bits and set bits_per_coded_sample
to 12.
Thanks for the added thoughts. Thinking about it, this would be easy enough
to implement.

I'd be fine with it, I suppose, but I'd prefer if it weren't a permanent
solution
Paul B Mahol
2013-01-22 15:08:35 UTC
Permalink
This post might be inappropriate. Click to display it.
Michaël Cinquin
2013-01-22 15:54:38 UTC
Permalink
Post by Paul B Mahol
FYI, contributor from Libav, with irc nick Buxiness is hacking on j2k
code is also working on XYZ.
I think the XYZ part of Digital Cinema encoding with FFmpeg has a good start-of-a-solution with vf_rgb2xyz ; I hope my commit will eventually make it to the trunk (though I do realize there may be a more elegant / efficient way to do it).
I'll try to get in touch with Buxiness if we don't find a solution here for those 4 extra bits to save 12bpc files.
Post by Paul B Mahol
Is "Digital Cinema" JPEG 2000?
Yes, I'm trying to use FFmpeg as the one-stop solution to opensource DCP mastering for Digital Cinema.
On the input, you put whatever FFmpeg accepts
On the output, you get jpeg-2000 .j2c files DCI-compliant, XYZ encoded, with a 2.6 gamma, 12 bpc, ready to pack into an mxf by asdcp-test.

You can find lots of opensource workflows on the net, all of them require you make a TIFF intermediate, which is in my mind a waste of resources, especially when most tools use libopenjpeg in the end.
Dave Rice
2013-01-23 04:49:13 UTC
Permalink
Post by Michaël Cinquin
Post by Carl Eugen Hoyos
Is "Digital Cinema" JPEG 2000?
Yes, I'm trying to use FFmpeg as the one-stop solution to opensource DCP mastering for Digital Cinema.
On the input, you put whatever FFmpeg accepts
On the output, you get jpeg-2000 .j2c files DCI-compliant, XYZ encoded, with a 2.6 gamma, 12 bpc, ready to pack into an mxf by asdcp-test.
To point out another barrier to FFmpeg as one-stop DCP, FFmpeg does not currently support muxing jpeg2000 into mxf. This would require an implementation of SMPTE 422m. There is a ticket for this request here: https://ffmpeg.org/trac/ffmpeg/ticket/1542.
Dave Rice
Michael Niedermayer
2013-01-22 16:02:18 UTC
Permalink
Post by Carl Eugen Hoyos
Post by Michael Bradshaw
Post by Reimar Döffinger
Is there any reason to add a 12 bit per component format
instead of using the 16 bit one and telling the encoder
it should discard 4 bit?
Out of curiosity, how would that work? Which 4 bits do you discard?
I believe when we "scale" from 16 to 8 bits, we simply discard
the 8 least significant bit, the same can be done for the
16 -> 12 case.
There are 2 different cases here
12bit stored in 16bit and true 16bit
while they are very similar, converting to 12bit can be done in 2
ways,
A shift by 4
B more or less exact rescaling

The difference is quite small but for true 16bit rescaling should
and i hope is used.
for 12bit stored in 16bit both the shift and the rescaling gives the
same result as long as the 12bit are reasonably correctly stored into
16bit, for example (i<<4) + (i>>8) will work

A encoder that wished to support 12bit in 16bit would thus be fine
with just using a >>4 and wouldnt need to care about the rest

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 1
"Used only once" - "Some unspecified defect prevented a second use"
"In good condition" - "Can be repaird by experienced expert"
"As is" - "You wouldnt want it even if you were payed for it, if you knew ..."
Michael Niedermayer
2013-01-22 19:45:57 UTC
Permalink
Post by Michael Niedermayer
Post by Carl Eugen Hoyos
Post by Michael Bradshaw
Post by Reimar Döffinger
Is there any reason to add a 12 bit per component format
instead of using the 16 bit one and telling the encoder
it should discard 4 bit?
Out of curiosity, how would that work? Which 4 bits do you discard?
I believe when we "scale" from 16 to 8 bits, we simply discard
the 8 least significant bit, the same can be done for the
16 -> 12 case.
There are 2 different cases here
12bit stored in 16bit and true 16bit
while they are very similar, converting to 12bit can be done in 2
ways,
A shift by 4
B more or less exact rescaling
The difference is quite small but for true 16bit rescaling should
and i hope is used.
for 12bit stored in 16bit both the shift and the rescaling gives the
same result as long as the 12bit are reasonably correctly stored into
16bit, for example (i<<4) + (i>>8) will work
A encoder that wished to support 12bit in 16bit would thus be fine
with just using a >>4 and wouldnt need to care about the rest
also we do have AV_PIX_FMT_GBRP12BE, AV_PIX_FMT_GBRP12LE and
AV_PIX_FMT_GBRP12

[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Awnsering whenever a program halts or runs forever is
On a turing machine, in general impossible (turings halting problem).
On any real computer, always possible as a real computer has a finite number
of states N, and will either halt in less than N cycles or never halt.
Michaël Cinquin
2013-01-23 04:13:31 UTC
Permalink
Post by Michael Niedermayer
also we do have AV_PIX_FMT_GBRP12BE, AV_PIX_FMT_GBRP12LE and
AV_PIX_FMT_GBRP12
Wouldn't they shift channels around going from rgb to gbr ?

They are not supported by libopenjpeg ; I already tried without success to add them into libopenjpeg.c, apparently it's not that simple
Post by Michael Niedermayer
Incompatible pixel format 'gbrp12le' for codec 'libopenjpeg', auto-selecting format 'rgb48le'
Michael Bradshaw
2013-01-23 04:38:16 UTC
Permalink
Post by Michaël Cinquin
Post by Michael Niedermayer
also we do have AV_PIX_FMT_GBRP12BE, AV_PIX_FMT_GBRP12LE and
AV_PIX_FMT_GBRP12
Wouldn't they shift channels around going from rgb to gbr ?
They are not supported by libopenjpeg ; I already tried without success to
add them into libopenjpeg.c, apparently it's not that simple
Post by Michael Niedermayer
Incompatible pixel format 'gbrp12le' for codec 'libopenjpeg',
auto-selecting format 'rgb48le'
It's a bit more involved than that, particularly because those GBR pix fmts
are planar, whereas all the RGB ones are packed.

I was about to start working on your request to be able to have 12 bit (or
any other >8 and < 16) by using shifts, like Reimar suggested. However, I
just noticed OpenJPEG 2.0 has been released and I spend some time trying to
get the encoder & decoder updated to use OpenJPEG 2.0 (while maintaining
backwards compatibility with older versions).

I'm just about to post a patch for 9-15 bit RGB output for the encoder. I
haven't looked at the decoder yet; it should be fairly simple but not as
dead simple as it was for the encoder.
Michael Bradshaw
2013-01-23 04:51:06 UTC
Permalink
Post by Michael Bradshaw
Post by Michaël Cinquin
Post by Michael Niedermayer
also we do have AV_PIX_FMT_GBRP12BE, AV_PIX_FMT_GBRP12LE and
AV_PIX_FMT_GBRP12
Wouldn't they shift channels around going from rgb to gbr ?
They are not supported by libopenjpeg ; I already tried without success
to add them into libopenjpeg.c, apparently it's not that simple
Post by Michael Niedermayer
Incompatible pixel format 'gbrp12le' for codec 'libopenjpeg',
auto-selecting format 'rgb48le'
It's a bit more involved than that, particularly because those GBR pix
fmts are planar, whereas all the RGB ones are packed.
Sorry, I lied; it's not the planar vs packed, but reordering the GBR to be
RGB so it's what OpenJPEG expects. It would take a little bit of extra code.
Michaël Cinquin
2013-01-23 06:41:36 UTC
Permalink
Post by Dave Rice
Post by Michaël Cinquin
On the output, you get jpeg-2000 .j2c files DCI-compliant, XYZ encoded, with a 2.6 gamma, 12 bpc, ready to pack into an mxf by asdcp-test.
To point out another barrier to FFmpeg as one-stop DCP, FFmpeg does not currently support muxing jpeg2000 into mxf. This would require an implementation of SMPTE 422m. There is a ticket for this request here: https://ffmpeg.org/trac/ffmpeg/ticket/1542.
So much the better if dci mxf packaging comes to FFmpeg !

In the meanwhile skipping a 16-bit TIFF intermediate will bring quite a saving in terms of disk i/o and space
1 hour film 1998x1080 in 16 bpc TIFF +- 1TB
1 hour film 1998x1080 j2c DCI +- 100 GB
1 hour film mxf : packs j2c +- 100 GB

present opensource workflow : 1 TB (tiff) + 100 GB (j2c) + 100 GB (mxf)

new opensource workflow when FFMpeg is ready : 100 GB (tiff) + 100 GB (mxf)
Post by Dave Rice
It's a bit more involved than that, particularly because those GBR pix fmts
are planar, whereas all the RGB ones are packed.
I was about to start working on your request to be able to have 12 bit (or
any other >8 and < 16) by using shifts, like Reimar suggested. However, I
just noticed OpenJPEG 2.0 has been released and I spend some time trying to
get the encoder & decoder updated to use OpenJPEG 2.0 (while maintaining
backwards compatibility with older versions).
I'm just about to post a patch for 9-15 bit RGB output for the encoder. I
haven't looked at the decoder yet; it should be fairly simple but not as
dead simple as it was for the encoder.
Amazing ! Can't wait to test it

Loading...