Logistic Regression

Binary logistic regression

• Converges to minimum loss but will just draw linearly boundaries on data

• Can only handle 2 classes

Main characteristics

• Binary Classification Model

• Probabilistic output (0. - 1.)

• Cannot use non-linear decision boundaries

• Uses Sigmoid or Logistic function as the activation function

• Threshold function used to pick a discrete binary output

Relevant properties:

• Assumes:

  • linearity between features and log-odds

  • independent observations

  • minimal multicollinearity

• Supports Lasso (L1) and Ridge (L2) regularization to prevent overfitting.

• Weight coefficients are interpretable

Multiclass Logistic Regression

• Can handle multiple classes

• But it just can use linear boundaries to separate data

Also known as Softmax Regression Model or Multinomial Logistic Regression:

• Softmax activation, instead of Sigmoid or Logistic function

• One-hot Encoded labels, instead of 0/1 labels

• Multiclass Cross Entropy, instead of Binary cross-entropy as the loss function