> For the complete documentation index, see [llms.txt](https://ykkim.gitbook.io/wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ykkim.gitbook.io/wiki/machine-learning/lecture-notes/fourieranalysis-wavelettransform.md). # Fourier and WaveletTransform ## Review of Fourier Analysis #### Visualizing Fourier Transform [But what is the Fourier Transform? A visual introduction. - YouTube](https://youtu.be/spUNpyF58BY) ### #### Fourier Analysis Lectures Lecture series by Steve Brunton [Fourier Analysis - YouTube](https://www.youtube.com/playlist?list=PLMrJAkhIeNNT_Xh3Oy0Y4LTj0Oxo8GqsC) ## ### Background **Inner Product in Hilbert Space** ![](/files/K5Xgbe2VvC9VMcZlS2xZ) #### Side Note **Conjugate:** If a complex number is z=a+bi, where a is the **real part** and b is the **imaginary part**,\ then its **conjugate** is written as $$ \bar{z}=a−ib, \~\~∣z∣^2=z\bar{z}=a^2+b^2 $$ **Normalizing Inner Product** $$ \/L=\\*dx $$ **Orthogonal basis functions** ![](/files/xj9bWxn9B8yfRkTPKSeX) ### Fourier Series ![](/files/inijUh1aMMsdqEGlaCva) ### **From Fourier Series to Wavelet Functions** ![](/files/TeFnsjWrFsBIgia6Qjpa) **Fourier Transformation** ![](/files/6vc0gmcmChHqROVR0ba6) *** ## Wavelet Decomposition ### Wavelet Introduction #### Basics of Wavelet Watch this video for understanding Wavelet transform concept [Wavelets and Multiresolution Analysis - YouTube](https://youtu.be/y7KLbd7n75g) #### Mathematics of Wavelet Transform for Beginners [The Wavelet Transform for Beginners - YouTube](https://youtu.be/kuuUaqAjeoA) > You need to Review Fourier Transform and Short-Time Fourier Transform. > > Click here for Review Fourier Transform **Limitation of STFT** ![](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2FJ7kUJ3ydAFzSG4a12Aqo%2Fimage.png?alt=media\&token=f411de9d-cad5-421f-b479-0a587834dd60) * Window Function is Finite, so Frequency Resolution decreases. * Fixed length of the window: time and frequency resolution are fixed for entire signals ![](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2FxnVif0dCjcLVZCS8cy10%2Fimage.png?alt=media\&token=85ca30a3-79b9-4f3a-8f93-05d50f5183d6) * Narrow Window: Good Time Resolution, Poor Frequency Resolution * Wide Window: Poor Time Resolution, Good Frequency Resolution **Wavelet Transform** Can analyze different Frequency at different Resolution * High frequency signals: apply narrow window * Low frequency signals: apply wide window ![](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2FIf4VhB1PtbU1qxlmOGre%2Fimage.png?alt=media\&token=f454c24e-27d2-491c-82ee-01da6e0296a2) ![](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2FQnMnx9UbOKJWjb5tZ38W%2Fimage.png?alt=media\&token=fed433e3-a594-4e7f-b2c3-91f62d5a5e41) Wavelet acts as a scalable and movable window function * Scaling s * Translation tau ![](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2FP4dAnjiNdI4q1Ce61czV%2Fimage.png?alt=media\&token=9555eb7c-383f-45f8-9343-3be45ea840c7) Different types of wavelets * Haar, Daubiche, Gaussian, Inverse Mexican Hat etc **Discrete Wavelet Transform** Wavelet transform calculates wavelet coefficients at every possible scale. * This gives a large data * So, use finite numbers of s and \tau * Choose s and \tau based on powers of two for efficient calculation ### Multilevel Decomposition ![](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAwtzMy_pbrChIExFtN%2Fuploads%2Ft8xlKep32HBuNKhLaHRk%2Fimage.png?alt=media\&token=439a838e-aa1e-407e-9c9b-d73c83739f93) #### 1D Wavelet Decomposition * **Algorithms** ![](https://kr.mathworks.com/help/wavelet/ref/ref_0226y.png) ![](https://kr.mathworks.com/help/wavelet/ref/ref_02114y.png) **MATLAB example** `wavedec()` * \*\*\[c,l] = wavedec(x,n,wname) \*\*returns the wavelet decomposition of the 1-D signal x at level n using the wavelet wname. * [MATLAB documentation](https://kr.mathworks.com/help/wavelet/ref/wavedec.html) **Dowload Example Code** [Download MATLAB example code](https://github.com/ykkimhgu/gitbook_docs/blob/master/machine-learning/tutorials/wavedec1d_example.mlx) \[c,l] = wavedec(x,n,wname) returns the wavelet decomposition of the 1-D signal x at level n using the wavelet wname. The output decomposition structure consists of the wavelet decomposition vector c and the bookkeeping vector l, which contains the number of coefficients by level. ![](https://kr.mathworks.com/help/wavelet/ref/ref_0233x.gif) ``` load sumsin plot(sumsin) title('Signal') ``` Perform a 3-level wavelet decomposition of the signal using the order 2 Daubechies wavelet. Extract the coarse scale approximation coefficients and the detail coefficients from the decomposition. * A = appcoef(C,L,wname) returns the approximation coefficients at the coarsest scale using the wavelet decomposition structure \[C,L] of a 1-D signal * D = detcoef(C,L,N) extracts the detail coefficients at the level or levels specified by N. ``` [c,l] = wavedec(sumsin,3,'db2'); approx = appcoef(c,l,'db2'); [cd1,cd2,cd3] = detcoef(c,l,[1 2 3]); Plot the coefficients. figure subplot(4,1,1) plot(approx) title('Approximation Coefficients') subplot(4,1,2) plot(cd3) title('Level 3 Detail Coefficients') subplot(4,1,3) plot(cd2) title('Level 2 Detail Coefficients') subplot(4,1,4) plot(cd1) title('Level 1 Detail Coefficients') ``` ![](/files/IqfkrWyAEFL6x7xQLEEl) ![](/files/eMtigBoCCZ11SqBE0HS9) #### 2D Wavelet Decomposition ![../\_images/plot\_mallat\_2d.png](https://pywavelets.readthedocs.io/en/latest/_images/plot_mallat_2d.png) It can be seen that many of the coefficients are near zero (gray). This ability of the wavelet transform to sparsely represent natural images is a key property that makes it desirable in applications such as image compression and restoration. * For subsequent levels of decomposition, only the approximation coefficients (the **lowpass subband**) are further decomposed. * The top row indicates the coefficient names as used by [`wavedec2()`](https://pywavelets.readthedocs.io/en/latest/ref/2d-dwt-and-idwt.html#pywt.wavedec2) after each level of decomposition. The bottom row shows wavelet coefficients for the camerman image (with each subband independently normalized for easier visualization). ### Application 1. [Wavelet Decomposition for Bearing fault classification](https://app.gitbook.com/s/-MAwtzMy_pbrChIExFtN/c/xHqhvVAiy3lgRCmi71jD/digital-twin/notes-on-data-processing/wavelet-decomposition) 2. Image Compression using Wavelet #### Exercise 1. [Detecting Discontinuities and Breakdown Points](https://kr.mathworks.com/help/wavelet/ug/detecting-discontinuities-and-breakdown-points.html) *** ## ## Analytical Signal and Hilbert Transform ### Analytical Signal and Hilbert Transformation [https://www.gaussianwaves.com/2017/04/analytic-signal-hilbert-transform-and-fft/](/wiki/machine-learning/lecture-notes/fourieranalysis-wavelettransform.md) ### Analytical Signal #### Continuous Analytical Signal The analytic signal is complex-valued but its spectrum will be one-sided (only positive frequencies) that preserved the spectral content of the original real-valued signal. ![](https://www.gaussianwaves.com/gaussianwaves/wp-content/uploads/2017/04/analytic_signal_spectrum_2.png) *Figure 1: (a) Spectrum of continuous signal x(t) and (b) spectrum of analytic signal z(t)* How to get one-sided Magnitude spectrum? * Frequency domain approach * The one-sided spectrum of $z(t)$ is formed from the two-sided spectrum of the real-valued signal $x(t)$ by applying $$ Z(f) = \begin{cases} ;X(0) & for ; f=0 \ 2X(f) & for ; f>0 \ ;;;0 & for ; f<0 \end{cases} $$ * Time domain approach * Using Hilbert transform approach $$ z(t)=zr(t)+jzi(t)=x(t)+jHTx(t) $$ One of the important property of an analytic signal is that its real and imaginary components are orthogonal $$ \displaystyle{\int\_{-\infty}^{\infty} z\_i(t) z\_r(t) = 0} $$ ### Discrete Analytical Signal Discrete-Time Fourier Transform (DTFT). $$ X(f) = \displaystyle{ T \sum\_{n=0}^{N-1} x\[n] e^{-j 2 \pi f n T} } $$ ![](https://www.gaussianwaves.com/gaussianwaves/wp-content/uploads/2017/04/discrete_time_analytic_signal_PNG.png) The analytic signal is complex valued $$ z\[n] = z\_r\[n] + j z\_i\[n] $$ and should satisfy the following two ***desired properties*** * The real part of the analytic signal should be same as the original real-valued signal. $$ z\_r\[n] = x\[n] $$ * The real and imaginary part of the analytic signal should satisfy the following property of orthogonality $$ \displaystyle{ \sum\_{n=0}^{N-1} z\_r\[n] z\_i\[n] = 0 } $$ Given a record of samples x\[n] of even length N, the procedure to construct the analytic signal z\[n] is as follows. This method satisfies both the desired properties listed above. * Compute the N-point DTFT of x\[n] using FFT * N-point periodic one-sided analytic signal is computed by the following transform Compute the N-point DTFT of x\[n] using FFT ● N-point periodic one-sided analytic signal is computed by the following transform equation for constructing analytic signal in frequency domain ![](https://www.gaussianwaves.com/gaussianwaves/wp-content/uploads/2019/06/analytic-signal.png) * Finally, the analytic signal (z\[n]) is obtained by taking the inverse DTFT of $$ z\[n] = \displaystyle{ \frac{1}{NT} \sum\_{m=0}^{N-1} z\[m] ; exp \left( j 2 \pi mn/N\right)} $$ ```matlab Function z = analytic_signal(x) %x is a real-valued record of length N, where N is even %returns the analytic signal z[n]x = x(:); %serialize N = length(x); X = fft(x,N); z = ifft([X(1); 2*X(2:N/2); X(N/2+1); zeros(N/2-1,1)],N); ``` ## *** ## Envelope (Instantaneous Amplitude) Extraction using Hilbert Transformation The amplitude modulated signal is $$ x(t) = m(t) cos\left(\omega\_c t\right) $$ * Information bearing signal m(t). Also called ***instantaneous amplitude***. * Carrier signal cos(wc\*t) Similarly, phase or frequency modulations: $$ x(t) = a cos \left(\phi(t) \right) $$ * The concept of instantaneous phase or instantaneous frequency is required for describing the modulated signal. ***General form*** $$ x(t) = a(t) cos \left\[ \phi (t) \right] $$ #### Problem statement Given the modulated signal, extract the instantaneous amplitude (envelope), instantaneous phase and the instantaneous frequency. * The modulated signal is a real-valued signal. * But, amplitude/phase and frequency can be easily computed if the signal is expressed in complex form. How can we convert a real signal to the complex plane without altering the required properties ? * [Apply Hilbert transform and form the analytic signal](https://www.gaussianwaves.com/2017/04/analytic-signal-hilbert-transform-and-fft/) on the complex plane $$ z(t) = z\_r(t) + j z\_i(t) = x(t) + j HT{x(t)} $$ The instantaneous amplitude (***envelope extraction***) is computed in the complex plane as $$ a(t) = |z(t)| = \sqrt{z\_r^2(t) + z\_i^2(t)} $$ \[ ![](https://www.gaussianwaves.com/gaussianwaves/wp-content/uploads/2017/04/extract_envelope_TFS-analytic-signal-hilbert-transform.png) Figure 2: Amplitude modulation using a chirp signal and extraction of envelope and TFS\* *** #### MATLAB Tutorial on Envelope Extraction [MATLAB example code](https://github.com/ykkimhgu/gitbook_docs/blob/master/machine-learning/tutorials/EnvelopExtraction_example.mlx)