Audio processing is without doubt one of the most vital software domains of digital sign processing (DSP) and machine studying. Modeling the acoustic setting is a necessary step within the growth of digital audio processing programs, together with speech recognition, voice enhancement, and acoustic echo cancellation.
The acoustic setting is full of background noise that may have a number of sources. For instance, if you sit in a espresso store, stroll the road, or drive a automobile, you hear sounds which are thought-about interference or background noise. Such interference doesn’t essentially comply with the identical statistical mannequin, so a mix of fashions is beneficial for modeling.
These statistical fashions present a spread of acoustic environments in numerous classes, akin to quiet auditoriums (class 1), or barely noisy rooms with closed home windows (class 2), and third possibility (class 2) that Home windows opens It is usually helpful to categorise it into 3). In each circumstances, the extent of background noise will be modeled utilizing a mix of noise sources, every occurring at completely different acoustic ranges with completely different chance.
One other software of such fashions will be designed to design DSP and machine studying options to unravel particular acoustic issues in actual audio programs akin to interference cancellation, echo cancellation, speech recognition, voice extension, and so on. It’s in simulation of acoustic noise in an setting that may be very comparable. .
A easy statistical mannequin that may be helpful in such a state of affairs is Gaussian Blended Mannequin (GMM) Every of the completely different noise sources is assumed to comply with a particular Gaussian distribution with a particular variance. As additionally proven, all distributions will be assumed to have a mean of zero whereas being correct sufficient for this software. article.
Every GMM distribution has a chance of contributing to background noise. For instance, there might be constant background noise that happens more often than not, however different sources might be intermittent, akin to noise coming from the window. All this needs to be thought-about within the statistical mannequin.
An instance of simulated GMM knowledge over time (normalized to sampling time) is proven within the diagram beneath. This diagram accommodates two sources of Gaussian noise. On this instance, a low variance sign happens extra ceaselessly with a 90% probability, leading to intermittent spikes of generated knowledge representing a sign with the next variance.
In different eventualities, relying on the appliance, it may be the other the place a excessive dispersion noise sign happens extra ceaselessly (as proven within the instance later on this article). The Python code used to generate and analyze GMM knowledge may also be displayed later on this article.
Turning to a extra formal modeling language, the background noise indicators collected (for instance, utilizing high-quality microphones) are independently and identically distributed (IID) random variables based on GMM, as proven beneath. . Assume that it’s modeled as a realization of
Subsequently, the modeling downside is summarized in estimating mannequin parameters (IE, P1, σ²1, and σ²2) utilizing noticed knowledge (IID). This text makes use of a Second Technique (MOM) estimator for such functions.
To simplify issues even additional, we will assume that the noise variance (σ²1 and σ²2) is thought and solely the blended parameters (P1) are estimated. A number of parameters (IE, P1, σ²1, and σ²2) will be estimated utilizing the MOM estimator, as proven in Chapter 9 of the ebook.Statistical sign processing: Estimation principle”, Stephen Kay. Nonetheless, on this instance, we assume that solely P1 is unknown and is estimated.
Since each Gaussians in GMM are averaged zero, we begin with the second second and attempt to get the unknown parameter P1 as a perform of the second second as follows:
Observe that one other easy option to get the second of a random variable (for instance, beneath the second second) is to make use of the second technology perform (MGF). Listed here are some glorious textbooks on chance principle that cowl such subjects and extra:Introducing the chance of information science”, Stanley H. Chan.
Earlier than we go any additional, we wish to quantify this estimator by way of the elemental traits of the estimator, akin to bias, variance, and consistency. This validates this numerically within the Python instance.
Beginning with the estimator bias, we will present that the above estimator of P1 is in reality unbiased as follows:
The variance of the estimator can then be derived as follows:
It is usually clear from the above evaluation that the estimator is constant as a result of it’s not honest, and that growing pattern measurement (n) reduces the variance. We additionally use the above equation for the variance of the P1 estimator in Python numerical examples (see extra on this article) when evaluating the speculation with precise numerical outcomes.
So let’s introduce you to Python code and do one thing enjoyable!
First, we generate knowledge that follows GMM with customary deviations with zero imply and customary deviations equal to 2 and 10, respectively, as proven within the code beneath. On this instance, the blended parameter p1 = 0.2, and the pattern measurement of the information equals 1000.
# Import the Python libraries that we'll want on this GMM instance
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
# GMM knowledge technology
mu = 0 # each gaussians in GMM are zero imply
sigma_1 = 2 # std dev of the primary gaussian
sigma_2 = 10 # std dev of the second gaussian
norm_params = np.array([[mu, sigma_1],
[mu, sigma_2]])
sample_size = 1000
p1 = 0.2 # chance that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the part
GMM_idx = np.random.selection(len(mixing_prob), measurement=sample_size, exchange=True,
p=mixing_prob)
# GMM_data is the GMM pattern knowledge
GMM_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx),
dtype=np.float64)Subsequent, plot the histogram of the generated knowledge and the chance density perform as proven beneath. This diagram reveals each contributions of Gaussian density throughout GMM, with every density being magnified by the corresponding elements.
Under is the Python code used to generate the above diagram:
x1 = np.linspace(GMM_data.min(), GMM_data.max(), sample_size)
y1 = np.zeros_like(x1)
# GMM chance distribution
for (l, s), w in zip(norm_params, mixing_prob):
y1 += stats.norm.pdf(x1, loc=l, scale=s) * w
# Plot the GMM chance distribution versus the information histogram
fig1, ax = plt.subplots()
ax.hist(GMM_data, bins=50, density=True, label="GMM knowledge histogram",
shade = GRAY9)
ax.plot(x1, p1*stats.norm(loc=mu, scale=sigma_1).pdf(x1),
label="p1 × first PDF",shade = GREEN1,linewidth=3.0)
ax.plot(x1, (1-p1)*stats.norm(loc=mu, scale=sigma_2).pdf(x1),
label="(1-p1) × second PDF",shade = ORANGE1,linewidth=3.0)
ax.plot(x1, y1, label="GMM distribution (PDF)",shade = BLUE2,linewidth=3.0)
ax.set_title("Information histogram vs. true distribution", fontsize=14, loc="left")
ax.set_xlabel('Information worth')
ax.set_ylabel('Chance')
ax.legend()
ax.grid()We then calculate the estimates of the beforehand derived blended parameter P1 utilizing MOM.
Under is the Python code used to calculate the above equation utilizing the GMM pattern knowledge:
# Estimate the blending parameter p1 from the pattern knowledge utilizing MoM estimator
p1_hat = (sum(pow(x,2) for x in GMM_data) / len(GMM_data) - pow(sigma_2,2))
/(pow(sigma_1,2) - pow(sigma_2,2))Use this estimator to correctly consider it Monte Carlo Simulation by producing a number of realizations of GMM knowledge and estimating P1 for every realization, as proven within the Python code beneath.
# Monte Carlo simulation of the MoM estimator
num_monte_carlo_iterations = 500
p1_est = np.zeros((num_monte_carlo_iterations,1))
sample_size = 1000
p1 = 0.2 # chance that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the part
GMM_idx = np.random.selection(len(mixing_prob), measurement=sample_size, exchange=True,
p=mixing_prob)
for iteration in vary(num_monte_carlo_iterations):
sample_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx))
p1_est[iteration] = (sum(pow(x,2) for x in sample_data)/len(sample_data)
- pow(sigma_2,2))/(pow(sigma_1,2) - pow(sigma_2,2))Subsequent, we test the bias and variance of the estimator and examine it with the theoretical outcomes derived earlier as proven beneath.
p1_est_mean = np.imply(p1_est)
p1_est_var = np.sum((p1_est-p1_est_mean)**2)/num_monte_carlo_iterations
p1_theoritical_var_num = 3*p1*pow(sigma_1,4) + 3*(1-p1)*pow(sigma_2,4)
- pow(p1*pow(sigma_1,2) + (1-p1)*pow(sigma_2,2),2)
p1_theoritical_var_den = sample_size*pow(sigma_1**2-sigma_2**2,2)
p1_theoritical_var = p1_theoritical_var_num/p1_theoritical_var_den
print('Pattern variance of MoM estimator of p1 = %.6f' % p1_est_var)
print('Theoretical variance of MoM estimator of p1 = %.6f' % p1_theoritical_var)
print('Imply of MoM estimator of p1 = %.6f' % p1_est_mean)
# Under are the outcomes of the above code
Pattern variance of MoM estimator of p1 = 0.001876
Theoretical variance of MoM estimator of p1 = 0.001897
Imply of MoM estimator of p1 = 0.205141From the outcomes above, we will observe that the typical P1 estimate is the same as 0.2051. That is very near the true parameter P1 = 0.2. This common will get even nearer to the true parameters as pattern measurement will increase. Subsequently, the estimator is numerically proven It is honest As confirmed by earlier theoretical outcomes.
Moreover, the pattern variance of the P1 estimator (0.001876) is roughly the identical as the attractive theoretical variance (0.001897).
It’s all the time a cheerful second when principle coincides with observe!
All pictures on this article are from the writer, except in any other case said.
