The iron ML notebook
  • The iron data science notebook
  • ML & Data Science
    • Frequent Questions
      • Discriminative vs Generative models
      • Supervised vs Unsupervised learning
      • Batch vs Online Learning
      • Instance-based vs Model-based Learning
      • Bias-Variance Tradeoff
      • Probability vs Likelihood
      • Covariance vs Correlation Matrix
      • Precision vs Recall
      • How does a ROC curve work?
      • Ridge vs Lasso
      • Anomaly detection methods
      • How to deal with imbalanced datasets?
      • What is "Statistically Significant"?
      • Recommendation systems methods
    • Statistics
      • The basics
      • Distributions
      • Sampling
      • IQR
      • Z-score
      • F-statistic
      • Outliers
      • The bayesian basis
      • Statistic vs Parameter
      • Markov Monte Carlo Chain
    • ML Techniques
      • Pre-process
        • PCA
      • Loss functions
      • Regularization
      • Optimization
      • Metrics
        • Distance measures
      • Activation Functions
      • Selection functions
      • Feature Normalization
      • Cross-validation
      • Hyperparameter tuning
      • Ensemble methods
      • Hard negative mining
      • ML Serving
        • Quantization
        • Kernel Auto-Tuning
        • NVIDIA TensorRT vs ONNX Runtime
    • Machine Learning Algorithms
      • Supervised Learning
        • Support Vector Machines
        • Adaptative boosting
        • Gradient boosting
        • Regression algorithms
          • Linear Regression
          • Lasso regression
          • Multi Layer Perceptron
        • Classification algorithms
          • Perceptron
          • Logistic Regression
          • Multilayer Perceptron
          • kNN
          • Naive Bayes
          • Decision Trees
          • Random Forest
          • Gradient Boosted Trees
      • Unsupervised learning
        • Clustering
          • Clustering metrics
          • kMeans
          • Gaussian Mixture Model
          • Hierarchical clustering
          • DBSCAN
      • Cameras
        • Intrinsic and extrinsic parameters
    • Computer Vision
      • Object Detection
        • Two-Stage detectors
          • Traditional Detection Models
          • R-CNN
          • Fast R-CNN
          • Faster R-CNN
        • One-Stage detectors
          • YOLO
          • YOLO v2
          • YOLO v3
          • YOLOX
        • Techniques
          • NMS
          • ROI Pooling
        • Metrics
          • Objectness Score
          • Coco Metrics
          • IoU
      • MOT
        • SORT
        • Deep SORT
  • Related Topics
    • Intro
    • Python
      • Global Interpreter Lock (GIL)
      • Mutability
      • AsyncIO
    • SQL
    • Combinatorics
    • Data Engineering Questions
    • Distributed computation
      • About threads & processes
      • REST vs gRPC
  • Algorithms & data structures
    • Array
      • Online Stock Span
      • Two Sum
      • Best time to by and sell stock
      • Rank word combination
      • Largest subarray with zero sum
    • Binary
      • Sum of Two Integers
    • Tree
      • Maximum Depth of Binary Tree
      • Same Tree
      • Invert/Flip Binary Tree
      • Binary Tree Paths
      • Binary Tree Maximum Path Sum
    • Matrix
      • Set Matrix Zeroes
    • Linked List
      • Reverse Linked List
      • Detect Cycle
      • Merge Two Sorted Lists
      • Merge k Sorted Lists
    • String
      • Longest Substring Without Repeating Characters
      • Longest Repeating Character Replacement
      • Minimum Window Substring
    • Interval
    • Graph
    • Heap
    • Dynamic Programming
      • Fibonacci
      • Grid Traveler
      • Can Sum
      • How Sum
      • Best Sum
      • Can Construct
      • Count Construct
      • All Construct
      • Climbing Stairs
Powered by GitBook
On this page
  • Overview
  • Advantages
  • Disadvantages
  • Types of NB classifiers

Was this helpful?

  1. ML & Data Science
  2. Machine Learning Algorithms
  3. Supervised Learning
  4. Classification algorithms

Naive Bayes

PreviouskNNNextDecision Trees

Last updated 5 years ago

Was this helpful?

Sources:

Overview

It assumes conditional independence between every pair of features given the value of the class. This is why this method is called "naive".

The Bayes theorem states:

P(y∣x1,...,xn)=P(y)P(x1,...,xn∣y)P(x1,...,xn)P(y|x_1, ..., x_n) = \frac{P(y)P(x_1,...,x_n|y)}{P(x_1,...,x_n)}P(y∣x1​,...,xn​)=P(x1​,...,xn​)P(y)P(x1​,...,xn​∣y)​

When assuming conditional independence:

P(y∣x1,...,xn)=P(y)P(x1,...,xn∣y)P(x1,...,xn)P(y|x_1, ..., x_n) = \frac{P(y)P(x_1,...,x_n|y)}{P(x_1,...,x_n)}P(y∣x1​,...,xn​)=P(x1​,...,xn​)P(y)P(x1​,...,xn​∣y)​

So, we can use the following classification rule:

y^=arg⁡max⁡yP(y)∏i=1nP(xi∣y)\hat{y} = \arg\max_y P(y) \prod_{i=1}^{n} P(x_i \mid y)y^​=argymax​P(y)i=1∏n​P(xi​∣y)

Advantages

  • NB classifiers have worked quite well in many real-world situations, famously document classification, sentiment analysis and spam filtering.

  • They require a small amount of training data to estimate the necessary parameters.

  • They can be extremely fast compared to more sophisticated methods.

Disadvantages

  • Although NB is considered as a decent classifier, it is known to be a bad estimator. So their output probabilities are not to be taken too seriously.

Types of NB classifiers

Multinomial Naive Bayes
Bernoulli Naive Bayes
Gaussian Naive Bayes
Naive Bayes (Scikit-Learn)