Data Prediction (Modeling)/Underfitting (bias problem)

• Underfitting

  • 高偏差
  • 低變異
    • 低變異的model在不同的訓練集合上性能更加穩定
    • 無論輸入什麼data，總是預測一個相同的label

from : 偏差和變異之權衡 (Bias-Variance Tradeoff) 2012| 逍遙文工作室

• find the model complexity that gives the smallest test error

from: Statistics - Bias-variance trade-off (between overfitting and underfitting) [Gerardnico]

• bias-variance tradeoff - Bias and variance contributing to total error

• training and testing error curves as a function of training set size

  • will potentially inform us about whether the model has a bias or variance problem and give clues about what to do about it.

  • If the model has a bias problem (underfitting)

    

    

    

* then both the testing and training error curves will plateau quickly and remain high. 
* This implies that getting more data will not help! We can improve model performance by reducing regularization and/or by using an algorithm capable of learning more complex hypothesis functions. 

from: Overfitting, bias-variance and learning curves - rmartinshort

Reference:

• Overfitting, bias-variance and learning curves - rmartinshort