🖍️
gitbook_docs
  • Introduction
  • Machine Learning
    • Recommended Courses
      • For Undergrad Research
      • Math for Machine Learning
    • ML Notes
      • Covariance Correlation
      • Feature Selection
      • Linear Regression
      • Entropy, Cross-Entropy, KL Divergence
      • Bayesian Classifier
        • Terminology Review
        • Bayesian Classifier for Normally Distributed classes
      • Linear Discriminant Analysis
      • Logistic Regression
        • Logistic Regression Math
      • Logistic Regression-MaximumLikelihood
      • SVM
        • SVM concept
        • SVM math
      • Cross Validation
      • Parameter, Density Estimation
        • MAP, MLE
        • Gaussian Mixture Model
      • E-M
      • Density Estimation(non-parametric)
      • Unsupervised Learning
      • Clustering
      • kNN
      • WaveletTransform
      • Decision Tree
    • Probability and Statistics for Machine Learning
      • Introduction
      • Basics of Data Analysis
      • Probability for Discrete Random Variable
      • Poisson Distribution
      • Chi-Square Distribution
      • P-value and Statistical Hypothesis
      • Power and Sample Size
      • Hypothesis Test Old
      • Hypothesis Test
      • Multi Armed Bandit
      • Bayesian Inference
      • Bayesian Updating with Continuous Priors
      • Discrete Distribution
      • Comparison of Bayesian and frequentist inference
      • Confidence Intervals for Normal Data
      • Frequenist Methods
      • Null Hypothesis Significance Testing
      • Confidence Intervals: Three Views
      • Confidence Intervals for the Mean of Non-normal Data
      • Probabilistic Prediction
  • Industrial AI
    • PHM Dataset
    • BearingFault_Journal
      • Support Vector Machine based
      • Autoregressive(AR) model based
      • Envelope Extraction based
      • Wavelet Decomposition based
      • Prediction of RUL with Deep Convolution Nueral Network
      • Prediction of RUL with Information Entropy
      • Feature Model and Feature Selection
    • TempCore Journal
      • Machine learning of mechanical properties of steels
      • Online prediction of mechanical properties of hot rolled steel plate using machine learning
      • Prediction and Analysis of Tensile Properties of Austenitic Stainless Steel Using Artificial Neural
      • Tempcore, new process for the production of high quality reinforcing
      • TEMPCORE, the most convenient process to produce low cost high strength rebars from 8 to 75 mm
      • Experimental investigation and simulation of structure and tensile properties of Tempcore treated re
    • Notes
  • LiDAR
    • Processing of Point Cloud
    • Intro. 3D Object Detection
    • PointNet
    • PointNet++
    • Frustrum-PointNet
    • VoxelNet
    • Point RCNN
    • PointPillars
    • LaserNet
  • Simulator
    • Simulator List
    • CARLA
    • Airsim
      • Setup
      • Tutorial
        • T#1
        • T#2
        • T#3: Opencv CPP
        • T#4: Opencv Py
        • Untitled
        • T#5: End2End Driving
  • Resources
    • Useful Resources
    • Github
    • Jekyll
  • Reinforcement Learning
    • RL Overview
      • RL Bootcamp
      • MIT Deep RL
    • Textbook
    • Basics
    • Continuous Space RL
  • Unsupervised Learning
    • Introduction
  • Unclassified
    • Ethics
    • Conference Guideline
  • FPGA
    • Untitled
  • Numerical Method
    • NM API reference
Powered by GitBook
On this page

Was this helpful?

  1. Machine Learning
  2. Probability and Statistics for Machine Learning

Null Hypothesis Significance Testing

PreviousFrequenist MethodsNextConfidence Intervals: Three Views

Last updated 3 years ago

Was this helpful?

Review: setting up and running a significance test

Notes.

  1. significance level을 사용하지 않고 rejection 영역을 사용해도 됨.

  2. null hypothesis는 'cautious hypothesis'가 될 수도 있음. significance level이 낮을수록 가설을 reject하기 위해 더 많은 evidence가 필요하기 때문.

  3. key point of confusion:

    significance level = 0.05의 의미가 test에서 실수가 일어날 확률이 5%라는 의미 X

    H0이 참일 때 H0을 reject할 확률이라는 의미 O

    !! power of the test = HA가 참일 때 H0를 reject할 확률

    -> HA가 참일 때 H0을 reject하지 못할 확률 = 1 - power

Errors

T-test

  • 많은 significance test는 데이터가 정규 분포에서 추출되었다고 가정하기 때문에 정규성 가정이 합리적인지 확인해야 함

  • z-test와 마찬가지로 one sample과 two sample t-test는 정규성을 가정함

Z-test

Review

Example)

데이터가

data∼N(μ,42)data \sim N(\mu ,4^2 )data∼N(μ,42)

를 따를 때

으로 가정함. 우리가 수집한 데이터가 1, 2, 3, 6, -1이라고 가정한다면 significance level이 0.05일때 H0를 reject해야 하는가?

Answer)

p<0.05이기 때문에 H0를 reject해야한다

The Student t distribution

  • 위키피디아 참고

  • normal distribution처럼 대칭적이고 종 모양임

  • 파라미터 df(degrees of freedom)에 따라 모양이 변함(df가 작을수록 꼬리가 두껍고 df가 클수록 ND에 가까워짐)

  • t-distribution의 모양을 볼 수 있는 곳:http://mathlets.org/mathlets/t-distribution/

One sample t-test

  • z-test는 분산을 안다고 가정하지만 실제로는 알 수 없는 경우가 많기 때문에 데이터에서 추정해야 함

    • 이런 경우에 one sample t-test를 사용함

example)

이전 예제에서 variance를 모른다고 가정함. 우리가 수집한 데이터는 1, 2, 3, 6, -1이고

이라고 가정함. significance level이 0.05일 때 H0을 reject해야 하는가?

Two-sample t-test with equal variance

  • Data: Normal distribution을 따르는 두 가지 데이터셋

    mean과 variance 둘 다 모르는 값이지만 같다고 추정

  • Null Hypothesis:

    μ1=μ2\mu _1 = \mu _2μ1​=μ2​
  • Test statistic:

  • Null distribution:

    f(t∣H0) is the pdf of T∼t(n+m−2).f\left( {t|H_0 } \right)\,is\,the\,pdf\,of\,T\sim t\left( {n + m - 2} \right).f(t∣H0​)isthepdfofT∼t(n+m−2).

Note

  1. pooled variance를 다음과 같이 정의하는 저자도 있음

    이 수업에선 그것을 estimated variance of x-y라고 부름

  2. 그룹간의 분산이 다른 two-sample test도 있음. 조금 더 복잡함

Example

산부인과에 온 여성 1408명의 임신 기간

  • 검진 목적:

    775 observation with x‾M=39.08 and sM2=7.77775\,observation\,with\,\overline x _M = 39.08\,and\,s_M ^2 = 7.77775observationwithxM​=39.08andsM2​=7.77
  • 응급:

    633 observation with x‾E=39.6 and sE2=4.95633\,observation\,with\,\overline x _E = 39.6\,and\,s_E ^2 = 4.95633observationwithxE​=39.6andsE2​=4.95

두 그룹의 평균이 다른지 two-sample t-test를 통해 조사하라

t, z test 등등 테이블로 정리(언제 어떤 거 사용해야하는지)

one sample vs two