android - How to initialize MediaFormat to configure a MediaCodec to decode raw AAC data?

Question

I have a strange issue with my StreamPlayer and I need any help I can get.

The main goal I need to achieve is StreamPlayer which is able to play back MPEG-2 Transport Streams with smallest possible latency. For this I am following this approach:

The stream is parsed by a Java based TS Parser. I have implemented a TSExtractor which is similar to the MediaExtractor and which works fine. I can receive all the media samples for a selected track the same way it is possible using the MediaExtractor with

extractor.readSampleData(...);
extractor.advance();

To decode the AAC data I want to create and configure an instance of MediaCodec. Using the MediaExtractor class this is usually done by

MediaFormat mediaFormat = extractor.getTrackFormat(i);
decoder = MediaCodec.createDecoderByType(mediaFormat.getString(MediaFormat.KEY_MIME));
decoder.configure(mediaFormat, null, null, 0);

As I have to initialize the MediaFormat in the TSExtractor.getTrackFormat(int track) method I use

MediaFormat mf = MediaFormat.createAudioFormat ("audio/mp4a-latm", getSampleRate(), getChannelCount());

and because all my AAC samples include an ADTS I do

mediaFormat.setInteger(MediaFormat.KEY_IS_ADTS, 1); 

After reading this post I finally add an ESDS frame using the "csd-0" key

mediaFormat.setByteBuffer("csd-0", ByteBuffer.allocate(2).put(new byte[]{(byte) 0x11, (byte)0x90}));

where the values 0x11 and 0x90 are extracted from the ADTS.

When I now want to decode the AAC samples the decoder posts

AAC decoder returned error 4097, substituting silence

to the Log.

To verify that my TSExtractor extracts the samples correctly I recorded the same stream using VLC remuxing it to an mp4 file without transcoding so the raw stream is unchanged. Now I can initialize the MediaExtractor with the recorded mp4 file and compare the samples created by my TSExtractor and the MediaExtractor. Using trail and error I found a very strange behavior:

When I configure the MediaCodec using the mediaFormat created by the MediaExtractor the MediaCodec decodes the AAC samples returned by my TSExtractor without any problems. Comparing the MediaFormat, which basically wraps a HashMap, created by my TSExtractor and the one created by the MediaExtractor gives this differences:

Created by MediaExtractor:

mediaFormat: {max-input-size=1212, durationUs=77428875, is-adts=1, channel-count=2, mime=audio/mp4a-latm, csd-0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], sample-rate=48000}

Created by TSExtractor:

mediaFormat: {is-adts=1, channel-count=2, mime=audio/mp4a-latm, csd-0=java.nio.ByteArrayBuffer[position=2,limit=2,capacity=2], sample-rate=48000}

Even when I adopt the MediaFormat created by the TSExtractor to be similar to the one created by the MediaExtractor the decoder gives the same error using the self created and decodes without any problems using the other one.

Any help would be really helpful.

Answer 1

I really don't know why, but it turns out that initializing the "csd-0" ByteBuffer this way

mediaFormat.setByteBuffer("csd-0", ByteBuffer.allocate(2).put(new byte[]{(byte) 0x11, (byte)0x90}));

doesn't work, but initializing it this way

byte[] bytes = new byte[]{(byte) 0x11, (byte)0x90};
ByteBuffer bb = ByteBuffer.wrap(bytes);
mediaFormat.setByteBuffer("csd-0", bb);

does.

BTW, comparing these two byteBuffers using

bb1.equals(bb2);

returns true.

Very strange!