WebRTC: how to use audio process module?

WebRTC
WebRTC

My current job responsiblity is researching on WebRTC, and the first task is wrapping a class from WebRTC to process audio frames to implement functions of audio AEC, AGC, NS, High pass filter etc.

Information list below is from WebRTC.org, you can also view it by visiting http://www.webrtc.org, or it’s code.

The Audio Processing Module (APM) provides a collection of voice processing components designed for real-time communications software.

APM operates on two audio streams on a frame-by-frame basis. Frames of the primary stream, on which all processing is applied, are passed to |ProcessStream()|. Frames of the reverse direction stream, which are used for analysis by some components, are passed to |AnalyzeReverseStream()|. On the client-side, this will typically be the near-end (capture) and far-end (render) streams, respectively. APM should be placed in the signal chain as close to the audio hardware abstraction layer (HAL) as possible.

On the server-side, the reverse stream will normally not be used, with processing occurring on each incoming stream.

Component interfaces follow a similar pattern and are accessed through corresponding getters in APM. All components are disabled at create-time, with default settings that are recommended for most situations. New settings can be applied without enabling a component. Enabling a component triggers memory allocation and initialization to allow it to start processing the streams.

Thread safety is provided with the following assumptions to reduce locking overhead:
1. The stream getters and setters are called from the same thread as ProcessStream(). More precisely, stream functions are never called concurrently with ProcessStream().
2. Parameter getters are never called concurrently with the corresponding setter.

APM accepts only 16-bit linear PCM audio data in frames of 10 ms. Multiple channels should be interleaved.

Usage example, omitting error checking:

AudioProcessing* apm = AudioProcessing::Create(0);
apm->set_sample_rate_hz(32000);
Super-wideband processing.
// Mono capture and stereo render.
apm->set_num_channels(1, 1);
apm->set_num_reverse_channels(2);
apm->high_pass_filter()->Enable(true);
apm->echo_cancellation()->enable_drift_compensation(false);
apm->echo_cancellation()->Enable(true);
apm->noise_reduction()->set_level(kHighSuppression);
apm->noise_reduction()->Enable(true);
apm->gain_control()->set_analog_level_limits(0, 255);
apm->gain_control()->set_mode(kAdaptiveAnalog);
apm->gain_control()->Enable(true);
apm->voice_detection()->Enable(true);
// Start a voice call...
// ... Render frame arrives bound for the audio HAL ...
apm->AnalyzeReverseStream(render_frame);
// ... Capture frame arrives from the audio HAL ...
// Call required set_stream_ functions.
apm->set_stream_delay_ms(delay_ms);
apm->gain_control()->set_stream_analog_level(analog_level);
apm->ProcessStream(capture_frame);
// Call required stream_ functions.
analog_level = apm->gain_control()->stream_analog_level();
has_voice = apm->stream_has_voice();
// Repeate render and capture processing for the duration of the call...
// Start a new call...
apm->Initialize();
// Close the application...
AudioProcessing::Destroy(apm);
apm = NULL;

 

Leave a Reply to hudson Cancel reply

Your email address will not be published. Required fields are marked *

13 thoughts on “WebRTC: how to use audio process module?”