android - Inconsistent video quality when encoding camera preview frames with MediaCodec on Xperia Z1

Question

I am encoding NV21 frames coming from camera preview. For some reason schema which is working fine on other devices works incorrectly on Sony Xperia Z1 with Android 4.3. It sends back encoded frames with incorrect (low) quality.

MediaCodec's format is COLOR_FormatYUV420SemiPlanar which is NV12 (I convert NV21 to NV12 by swapping U and V components). Output buffers sent back to me by MediaCodec are with very low size which don't correspond to resolution (1280x720) and bitrate (1000000) I am using. First few frames are coming with a good quality, but then it's dropping significantly:

int encoderStatus = mMediaCodec.dequeueOutputBuffer(mBufferInfo, TIMEOUT_USEC);
// a few encoderStatus checks skipped
ByteBuffer encodedData = outputBuffers[encoderStatus];
Log.i(Constants.TAG, "Buffer size " + mBufferInfo.size);

Which gives me the following log on Xperia Z1:

Buffer size 26
Buffer size 52172
Buffer size 23650
Buffer size 14394
Buffer size 3591
Buffer size 1849
Buffer size 3908

...

Buffer size 1043
Buffer size 248
Buffer size 836
Buffer size 518
Buffer size 1112

Example log from Sony Xperia ZR on which this works properly:

Buffer size 21
Buffer size 51048
Buffer size 21063
Buffer size 24228
Buffer size 28040
...
Buffer size 44959
Buffer size 44972
Buffer size 44957
Buffer size 45004
Buffer size 44999
Buffer size 44957

Any advice would be appreciated.

Answer 1

Are you sure you pass timestamps in the right unit (microseconds) - and that it has set a sensible framerate?

• Some encoders might ignore the timestamps and only set a fixed bitrate budget per frame, based on the framerate.

• Others might try to calculate how many bits they're allowed to use per frame based on the timestamp.

If the timestamps are given e.g. in milliseconds instead, this could lead to the encoder reducing the size of encoded frames down towards zero.