EE 6010
Polytechnic University of Puerto Rico
Department of
Electrical Engineering
Master in
Electrical Engineering
|
| |||||||||||||||||
Course Title : Mathematical Methods for Signal Processing
Course Code : EE 6010
Credits : Three (3) credits
Duration : One Academic Quarter
Schedule : Forty-five credit hours per course
Prerequisites : Undergraduate Calculus and Undergraduate Differential Equations
This course provides part of the extensive mathematical background needed for contemporary signal processing, practice and research. It emphasizes several linear algebra topics. Some of the topics covered are: Vector Spaces and Linear Algebra including Linear Operators, Inverse Matrices, Matrix Factorizations, Eigenvalues and Eigenvectors, Singular Value Decomposition, Some Special Matrices and their Applications, and Kronecker Products. The connection of these topics with signal processing is emphasized.
Topics such as Digital Signal Processing and Digital Communications require the students to have adequate background in Vector Spaces and Linear Algebra. The material selected for this class will provide a foundation for future courses in Communication Theory, Signal processing and Controls, as well as solid grounding for students wishing to pursue Doctoral studies.
To provide the mathematical background needed for practice and research in contemporary signal processing.
Mathematical Methods and Algorithms for Signal Processing (2000)
By T. K. Moon and W. C. Stirling
Prentice Hall, Upper Saddle River, NJ
The final course grade will be determined, unless otherwise is accorded with the instructor, by the following scale
100-90 A
89-90 B
79-70 C
69-60 D
59-0 F
All students are required to build a portfolio with the notes, homework exercises, exams, personal comments and research projects, evaluations etc. Up to 25% of the final grade can be from the portfolio. The other 75% will be from two partial exams and a final test, which will be given in the classroom.
April 2002, prepared by Pedro Torres, Ph.D. Candidate, P.E.
May 2002, revised by Marvi Teixeira, Ph.D., P.E.
February 2003, adapted by Jorge Sarmiento, D.A., M.A., B.S., B.E.E., A.A.S.
September 2003, revised and updated to include an online component, by Jorge Sarmiento, D.A., M.A., B.S., B.E.E., A.A.S.
February 2005, revised and updated to be used with the Blackboard platform, by Jorge Sarmiento, D.A., M.A., B.S., B.E.E., A.A.S.
November 2007: Revision of the 10 Lectures and the corresponding exercises. The course is now presented as a set of 16 Lessons with therie corresponding exercises. All material was updated to be used with the Blackboard platform,Jorge Sarmiento, D.A., M.A., B.S., B.E.E., A.A.S.
Communication Systems Engineering, (2002)
By J. G. Proakis and M. Salehi
2nd Ed., Prentice Hall
ISBN: 0-13-061793-8
Probability, Random Variables and Stochastic Processes. (2001)
By A. Papoulis
4th Ed., Prentice Hall
Wavelets and Filter Banks. (1996)
By G. Strang and T. Nguyen
1st Ed., Wellesly-Cambridge Press
Circulant Matrices. (1994)
By P.J. Davis
1st Ed., Chelsea Pub.
Ten Lectures on Wavelets. (1992)
By Ingrid Daubechies
Rutgers University and AT&T Laboratories
SIAM, Philadelphia, PA`
Advanced Linear Algebra. (1992)
By S. Roman
1st Ed., Springer Verlag
ISBN: 0-387-97837-2
Algorithms for Discrete Fourier Transforms and Convolution. (1989)
By R. Tolimieri, M. An and C. Lu.
1st Ed., Springer Verlag
Linear Algebra and its Applications. (1976)
By G. Strang
Academic Press
Linear Operator Theory. (1971)
By A.W. Naylor and G.R. Sell
1st Ed., Holt Rinehart Winston
ISBN: 0-03-079390-4
Lecture 1: (Moon Chap.1,2). Introduction: Signal Processing. Models for Linear Systems. Difference Equations. Gaussian Random Variables. Signal Spaces. Metric Spaces. Some Topological Terms and Definitions. Cauchy Sequences. Complete Metric Spaces. Vector Spaces. Linear Independence. Bases and Dimension. MATLAB session.
Lecture 2: (Moon Chap. 2). Norms. Normed Linear Spaces and Properties. Inner Product Spaces. Induced Norm. The Cauchy-Schwarz Inequality. Direction Vector and Orthogonality. Orthonormal Vectors and The Kronecker Delta Function. Legendre Polynomials. Hermitian and Positive Definite Matrices. Weighted Inner Product. Chebyshev Polynomials. Expectation as an Inner Product.
Lecture 3: (Moon Chap. 2). Banach and Hilbert Spaces. Orthogonal Spaces. Linear Transfromations. Range and Kernel. Inner and Direct Sum Spaces. Isomorphisms. Projections and Orthogonal Projections. Projection Matrices. The Projection Theorem. Orthogonalization of vectors: The Gram-Schmidt Process (QR factorization). Eigenvalues, Eigenvectors, and Eigenspaces. MATLAB session.
Lecture 4: (Moon Chap. 3). Representation and Approximation in Vector Spaces. Approximation in Hilbert Spaces. Matrix Representation of LS (Least-Square) Problems. Weighted Least-Squares. Statistical Properties of The LS estimate. Minimum Error in Vector-Space Approximations. Applications of The Orthogonality Theorem: Approximation by Continuous Polynomials. Approximation by Discrete Polynomials. Linear and Quadratic Regression. Linear Regression using Weighted LS.
Lecture 5: (Moon Chap. 3). Least-Squares Filtering. Minimum Mean-Square estimation. Minimum Mean-Square Error (MMSE) Filtering. Signal Transformation and Generalized Fourier Series. Bessel?s Inequality and Parseval?s Equality. Sets of Complete Orthogonal Functions: Trigonometric, Polynomial (Legendre and Chebyshev), Sinc, Wavelets (Haar basis).
Extension: A Dual Approximation Problem. The Dual Approximation Theorem. Minimum-Norm Solution of Undetermined Equations.
Lecture 6: (Moon Chap. 4). Linear Operators and Matrix Inverses. The Riesz Representation Theorem. Operator Norms. Adjoints. A Dual Optimization Problem. Geometry of Linear Equations. Fundamental Subspaces. Rank (Full and Deficient) of a matrix. Matrix Inversion Lemma. Fredholm Alternative Theorem. Matrix Conditioning Number.
Lecture 7: (Moon Chap. 5, 6). Matrix Factorizations: LU, Cholesky, and QR ? The Gram-Schmidt Algorithm, The Housholder Transformation, and Givens Rotations. Eigenvalues and Eigenvectors. The Characteristic Equation. Diagonalization of a matrix. Applications. The Jordan form.
Lecture 8: (Moon Chap. 6). Quadratic forms and their classification. Maximum principle. Rayleigh Quotient. Applications of Eigendecomposition Methods. Low Rank Approximation. Principal Components Method. Eigenfilters. Signal Subspace Techniques. Generalized Eigenvalues.
Lecture 9: (Moon Chap. 6, 7). Characteristic and Minimal Polynomials. The Eigenvalues of a Symmetric Matrix. Triagonalization. The Singular Value Decomposition (SVD). Pseudoinverses and the SVD. Total Least-Square Problem. Partial Total Least-Squares (PTLS).
Lecture 10: (Moon Chap. 8, 9). Some Special Matrices and their Applications: Hankel, Permutation, Toeplitz, Persymmetric, Vandermond, Circulant, and Upper and Lower Triangular. Matrix properties preserved and not preserved under multiplication. Kronecker Products (Direct or Tensor) and The Vec Operator. The Kronecker Sum. Applications of the Kronecker Product. Fast Fourier Transform and Hadamard Matrices.
Lecture 11: Summary and Review ? Signal Spaces: Metric Spaces, Vector Spaces, Normed Vector Spaces, Inner Product Spaces, Hilbert and Banach Spaces. The Projection Theorem and The Orthogonalization Process. Representation and Approximation in Vector Spaces. Error Minimization and LS Problems. Linera Regression. Filtering. Linera Operators and Matrices. Matrix Factorizations: LU, QR, Cholesky, SVD. Eigenvalues and Eigenvectors. Applications of Eigendecomposition Methods. Special matrices and their Applications: Permutation, Toep;itz, Vandermond, and Circulant matrices. Kronecker Product and The Vec Operator.
Test 1 (Online). Lectures 1,2,3.
Midterm Test. Lectures 1-6.
Final Exam.Comprehensive: Lectures 1-10.
|