android - how to build BufferReceived() to capture voice using RecognizerIntent?

Question

i am working on an android application using RecognizerIntent.ACTION_RECOGNIZE_SPEECH,,, my problem is that i don't know how to create the buffer which will capture the voice that the user inputs. i read alot on stack overflow, but i just don't understand how i will include the buffer and the recognition service call back into my code. AND HOW WILL I DO PLAY BACK FOR THE CONTENTS WHICH WERE SAVED INTO THE BUFFER.

this is my code:

       public class Voice extends Activity implements OnClickListener {
   byte[] sig = new byte[500000] ;
   int sigPos = 0 ;
       ListView lv;
   static final int check =0;
   protected static final String TAG = null;

@Override
protected void onCreate(Bundle savedInstanceState) {



    // TODO Auto-generated method stub
    super.onCreate(savedInstanceState);


    setContentView(R.layout.voice);

    Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
            RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
    intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,
            "com.domain.app");

    SpeechRecognizer recognizer = SpeechRecognizer
            .createSpeechRecognizer(this.getApplicationContext());

    RecognitionListener listener = new RecognitionListener() {

        @Override
        public void onResults(Bundle results) {
            ArrayList<String> voiceResults = results
                    .getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
            if (voiceResults == null) {
                Log.e(TAG, "No voice results");
            } else {
                Log.d(TAG, "Printing matches: ");
                for (String match : voiceResults) {
                    Log.d(TAG, match);
                }
            }
        }

        @Override
        public void onReadyForSpeech(Bundle params) {
            Log.d(TAG, "Ready for speech");
        }

        @Override
        public void onError(int error) {
            Log.d(TAG,
                    "Error listening for speech: " + error);
        }

        @Override
        public void onBeginningOfSpeech() {
            Log.d(TAG, "Speech starting");
        }

        @Override
        public void onBufferReceived(byte[] buffer) {
            // TODO Auto-generated method stub
            TextView display=(TextView)findViewById (R.id.Text1);
                    display.setText("True");


              System.arraycopy(buffer, 0, sig, sigPos, buffer.length) ;
              sigPos += buffer.length ;

        }

        @Override
        public void onEndOfSpeech() {
            // TODO Auto-generated method stub

        }

        @Override
        public void onEvent(int eventType, Bundle params) {
            // TODO Auto-generated method stub

        }

        @Override
        public void onPartialResults(Bundle partialResults) {
            // TODO Auto-generated method stub

        }

        @Override
        public void onRmsChanged(float rmsdB) {
            // TODO Auto-generated method stub

        }
    };
    recognizer.setRecognitionListener(listener);
    recognizer.startListening(intent);




    startActivityForResult(intent,check);

}

@Override
public void onClick(View arg0) {
    // TODO Auto-generated method stub

}



}

Answer 1

The Android speech recognition API (as of API level 17) does not offer a reliable way to capture audio.

You can use the "buffer received" callback but note that

RecognitionListener says about onBufferReceived:

More sound has been received. The purpose of this function is to allow giving feedback to the user regarding the captured audio. There is no guarantee that this method will be called.

buffer: a buffer containing a sequence of big-endian 16-bit integers representing a single channel audio stream. The sample rate is implementation dependent.

and RecognitionService.Callback says about bufferReceived:

The service should call this method when sound has been received. The purpose of this function is to allow giving feedback to the user regarding the captured audio.

buffer: a buffer containing a sequence of big-endian 16-bit integers representing a single channel audio stream. The sample rate is implementation dependent.

So this callback is for feedback regarding the captured audio and not necessarily the captured audio itself, i.e. maybe a reduced version of it for visualization purposes. Also, "there is no guarantee that this method will be called", i.e. Google Voice Search might provide it in v1 but then decide to remove it in v2.

Note also that this method can be called multiple times during recognition. It is not documented however if the buffer represents the complete recorded audio or only the snippet since the last call. (I'd assume the latter, but you need to test it with your speech recognizer.)

So, in your implementation you should copy the buffer into a global variable to be saved e.g. into a wav-file once the recognition has finished.