> For the complete documentation index, see [llms.txt](https://jgoodman8.gitbook.io/iron-data-science-notebook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://jgoodman8.gitbook.io/iron-data-science-notebook/ml-datascience/machine-learning-algorithms/supervised-learning/classification-algorithms/naive-bayes.md).

# Naive Bayes

{% hint style="info" %}
*Sources:*

* [*Naive Bayes (Scikit-Learn)*](https://scikit-learn.org/stable/modules/naive_bayes.html)
  {% endhint %}

## Overview

It **assumes conditional independence** between every pair of features given the value of the class. This is why this method is called "naive".

The Bayes theorem states:

$$
P(y|x\_1, ..., x\_n) = \frac{P(y)P(x\_1,...,x\_n|y)}{P(x\_1,...,x\_n)}
$$

When assuming conditional independence:

$$
P(y|x\_1, ..., x\_n) = \frac{P(y)P(x\_1,...,x\_n|y)}{P(x\_1,...,x\_n)}
$$

So, we can use the following classification rule:

$$
\hat{y} = \arg\max\_y P(y) \prod\_{i=1}^{n} P(x\_i \mid y)
$$

### Advantages

* NB classifiers **have worked quite well in many real-world situations**, famously document classification, sentiment analysis and spam filtering. 
* They **require a small amount of training data** to estimate the necessary parameters.
* They **can be extremely fast** compared to more sophisticated methods.

### Disadvantages

* Although NB is considered as a decent classifier, **it is known to be a bad estimator**. So their output probabilities are not to be taken too seriously.

## Types of NB classifiers

* [**Multinomial Naive Bayes**](https://towardsdatascience.com/multinomial-naive-bayes-classifier-for-text-analysis-python-8dd6825ece67)
* [**Bernoulli Naive Bayes**](https://stats.stackexchange.com/questions/246101/when-to-use-bernoulli-naive-bayes)
* [**Gaussian Naive Bayes**](https://medium.com/@LSchultebraucks/gaussian-naive-bayes-19156306079b)