> For the complete documentation index, see [llms.txt](https://jgoodman8.gitbook.io/iron-data-science-notebook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://jgoodman8.gitbook.io/iron-data-science-notebook/ml-datascience/machine-learning-algorithms/unsupervised-learning/clustering/gaussian-mixture-model.md).

# Gaussian Mixture Model

{% hint style="info" %}
*Sources:*

* [*Gaussian Mixture Models Explained (Towards Data Science)*](https://towardsdatascience.com/gaussian-mixture-models-explained-6986aaf5a95)
* [*Gaussian Mixture Models Algorithm Explained (Cory Maklin)*](https://towardsdatascience.com/gaussian-mixture-models-d13a5e915c8e)
  {% endhint %}

## What is a Gaussian Mixture?

It is **function composed by several Gaussians**. The **number of Gaussians is equal to the number of clusters** (*k*). Each distribution is parametrized by:

* The mean $$\mu$$ which defines its centre.
* The covariance Σ which defines its width.
* The mixing probability $$\pi$$.

The Gaussian density function is given by:

![](https://569842953-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LmjpNbCRLUyGiAxD8kn%2F-LrFieSkMBwxTqjNibEt%2F-LrGIbQoLN9dhzGo5lG8%2Fimage.png?alt=media\&token=305d5bf0-b989-40ea-a457-5a8441c190ab)

where:

* The sum of every mixing probability must be equal one: $$\sum\_{i=1}^K \pi\_i = 1$$
* $$D$$is the number of dimensions or features of each instance.
* Each data point $$X$$represents a data point (a $$1 \times D$$ vector)
* The mean $$\mu$$ is a $$1 \times D$$ vector.
* And the covariance Σ is a $$D \times D$$ matrix.

## How do we fit the algorithm?

By applying the *Expectation-Maximization algorithm* widely used for optimization problems where the objective function is that complex.

![](https://569842953-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LmjpNbCRLUyGiAxD8kn%2F-LrGKU6UBoFU-5e9U2Lm%2F-LrGUAVWXYklpzEyKpEJ%2F1*I0WTzTOyyDVwfPyMSZPzWQ.gif?alt=media\&token=9a60c025-d983-4d27-aa0c-16e5591892fd)

## Differences regarding k-Means

* It **accounts for covariance**, which determines the shape of the distribution  This means that meanwhile the **k-means model is that it places a circle** (or a hyper-sphere) at the center of each cluster, **a GMM model can handle different shapes**.

![](https://569842953-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LmjpNbCRLUyGiAxD8kn%2F-LrGKU6UBoFU-5e9U2Lm%2F-LrGSHIGTKfFCo0lbLS4%2Fimage.png?alt=media\&token=300975b6-1999-4fea-901d-6353b7fc4195)

* **k-Means performs a hard classification**, but a **GMM model carries out a soft one** by returning the probability that each data point belongs to a certain cluster.