This programming intensive course introduces the student to the algorithms and
data structures used in modern pattern recognition systems with an emphasis on
those that can learn and improve their performance as they are used.
After a short review of some necessary mathematical concepts (Probability,
Stochastic Processes and Vector Spaces), the student is introduced to the
problem of representing real-world problems to a system. Students will,
in teams, design, build, test and report on major subsystems of a pattern
recognition system. Selected real world applications are used to show
examples of some valid representations (e.g., Speech, Handwriting, EKG) to
provide insight and experience in the application of recognition systems.
Several important recognition engines are then studied and analyzed for their
effectiveness as the basis for recognition, synthesis, and learning
systems. The use of additional knowledge bases dealing with the problem
environment is then introduced to increase system performance.
|
Instructor |
Jeffrey N. Denenberg |
Office location |
|
|
E-mail - Fairfield |
Zoom Office Hours |
I'll be in my "office" whenever I am online, but just contact me and we can schedule a meeting, |
|
|
E-Mail - Personal |
Google Voice |
203-513-9427 |
|
|
Quickest Contact |
Sending a private Quip chat message with a "@jdenenberg" is the quickest way to reach me as it alerts on my phone I will normally respond within a few hours, but not after 10 pm or before 9 am. |
|
|
ABET CS Student Outcomes (1 - 5), and Bloom Cognitive Level in parenthesis
|
No. |
Course Goals |
Student Learning Outcomes |
|
1 |
Study the fundamental concepts and approaches to building pattern Recognition systems. |
Students will be able to find solutions to complex
problems that are poorly defined and under specified. |
|
2 |
Students will be able to apply several generic and
application specific data reduction techniques on real world data. |
|
|
3 |
Study several classical and modern methods to classify input data as belonging to one of a set of predefined states and understand their relative strengths and weaknesses. |
Students will be able use one or more of the studied
recognition engines to classify data. |
|
4 |
Work in small collaborative teams developing software in their preferred language to implement a number of data reduction algorithms that work on real world data. |
Students will develop teaming skills that will allow them
to work on large projects in the future. |
|
5 |
Students will demonstrate the use and application of their
programming skills on advanced algorithms for analyzing "Big Data". |
· “Pattern Classification”, Richard O. Duda, Peter E. Hart, and David G. Stork, Wiley-Interscience; Ed 2 (November 9, 2000) , ISBN 978-0471056690. - This downloadable PDF of the classic text on this subject although it is dated, it is a great reference on many topics. The appendices include all the required mathematical backgrounds for studying Pattern Recognition.
· “Pattern Recognition and Machine Learning“, Christopher M. Bishop, Springer, ISBN 0-387-31073-8, (2006). This downloadable PDF is a later classic textbook on Pattern Recognition. I have the solutions to many of the end of chapter problems so I should be able help you if you try to do some of them.
· “Mathematics for Machine Learning“, Marc Peter Deisenroth, Cambridge University Press, ISBN 110845514X, 2020 - Available in Paperback, Hard Cover, Kindle and free PDF versions. It covers the math background for most ML methods and requires very little previous math background to read.
· “Pattern Recognition”, Sergios Theodoridis and Konstantinos Koutroumbas, Academic Press, Ed. 4, 2009, ISBN 978-1597492720. - A low cost text covering most of the topics in our course.
· https://www.coursera.org/learn/machine-learning/
· http://en.wikipedia.org/wiki/Machine_learning,www.sci.utah.edu/~gerig/CS6640-F2010/prob-tut.pdf
· http://en.wikipedia.org/wiki/Variable-order_Markov_model
· https://spectrum.ieee.org/consumer-electronics/audiovideo/deep-learning-reinvents-the-hearing-aid
· https://nptel.ac.in/courses/117108048/: Pattern Recognition - Prof. P. S. Sastry, Indian Institute of Science, Bangalore
· https://www.youtube.com/playlist?list=PL17567A1A3F5DB5E4: Probability Primer - MathematicalMonk on youtube
· https://www.youtube.com/playlist?list=PL17567A1A3F5DB5E4: Markov Models - MathematicalMonk on youtube
· https://www.engineer4free.com/linear-algebra.html: Linear-algebra: Engineer4Free.com
· Each student collaborative team will develop and test their code on a sequence of mini-projects to demonstrate their acquired knowledge, skills and understanding.Through what performance tasks will students demonstrate the desired knowledge, skills and understanding? (e.g., portfolios, written or visual texts, discussions, projects, etc.)
· Each student will submit a “Reflection” on what was learned in the performance of each week’s project.
· Team’s will produce a presentation describing selected projects and their results.
·
A Rubric will be filled out for each team
project by each team member as well as the course leader
The other students will be asked to comment on each team’s approach to the
problem.
· A separate Rubric will be filled out for each “Reflection” by the course leader.
|
Item |
Number |
Weighting |
Total |
|
Mini Projects |
4 |
10% |
40% |
|
Reflections on Mini Projects |
4 |
5% |
20% |
|
Recognition Project |
1 |
20% |
20% |
|
Reflection on Recognition Project |
1 |
5% |
5% |
|
Class Participation |
1 |
15% |
15% |
|
|
|
Total |
100% |
· What knowledge and skills will students need to achieve these outcomes?
o
They, as Computer
Science or Data Sciences students should have the required programming
skills.
(This course is programming language
agnostic: Python, Matlab, Java, C/C++ ... )
o They will have to acquire some Probability Theory and Linear systems knowledge from supplied resources to complete some of the assigned projects.
· What activities will equip students with needed knowledge and skills?
o Their focused research to solve some of the problems that they encounter in completing the assigned projects.
o Writing a reflection on their team’s activities and giving others feedback on their reflections
· What materials and tools will best support students to acquire the knowledge, skills and outcomes?
o Their chosen Programming Integrated Development Environment (IDE)
o Their ability to do effective searches for additional resources on the Internet.
Instructional Activities (Preliminary Breakdown)
· Prior to class start 5/18/2020
o Install Quip App(s) and familiarize yourself with Quip
o Introduce yourself (record short video and insert it into the Intro document)
· Week 1
o Read and comment on course introduction, Quip, Course Procedures.
o exchange personal contact information (emails, cellphone).
o Establish small teams for projects.
o Read and discuss “A Structured Approach to Pattern Recognition”.
·
Weeks 2-3 - Time domain segmentation
Finding “critical points” and segment real data into smooth “strokes” (choose
ECG or Handwriting data)
o Read and comment on the suggested first mini-project topics as well as the supporting reference materials
o Establish which first project topic assignment each group accepts.
o Produce working code (include many comments in the source file to make it obvious what each portion of the code is supposed to do).
o Upload the source code and example results of running the code to your team folder in a sub folder for this project.
o Write a reflection on your team’s activity on the first mini-project.
o Read and comment (suggestions for improvement, not critiques) on the other teams results.
o Fill out the rubric sheet to evaluate your team’s performance on the first project.
· Weeks 4-5 - Algorithms to reduce input space complexity
o Complexity reduction of critical time segments
§ Vector Quantization,
§ Low order Polynomial fit,
§ Spline Graphics
o
An alternative approach: Wavelet Compression of
data sequence
(Two dimensional analysis of the input data: Time and
Frequency)
· Week 6 - Speech Data reduction (“cepstrum”, LPC, Wavelet Transform)
o Definitions:
§ A cepstrum (/ˈkɛpstrʌm, ˈsɛp-, -strəm/; plural cepstra) is the result of taking the inverse Fourier transform (IFT) of the logarithm of the estimated signal spectrum in each time frame window.
§ Linear Predictive Coding (LPC) - is a method for signal source modelling in speech signal processing. It is often used by linguists as a formant extraction tool.
§ Wavelet Transform - is similar to the Fourier transform (or much more to the windowed Fourier transform) with a completely different merit function. The main difference is this: Fourier transform decomposes the signal into sines and cosines, i.e. the functions localized in Fourier space; in contrary the wavelet transform uses functions that are localized in both the real (time) and Fourier space.
· Week 7 - k-means clustering - aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. ... Applying the 1-nearest neighbor classifier to the cluster centers obtained by k-means classifies new data into the existing clusters.
· Weeks :8-11 - Implement one of 4 “Recognition Engines”; Template Matching (AKA Dynamic Programming), Hidden Markov, Nth-Order Markov, Neural Networks
· Weeks 12-13 - Use your recognition engine to classify the data you generated in the 2nd portion of this course. Use the results to improve your implementation and report on performance.
Types of Tech Tools Needed
· By Instructor
o Quip Apps - Already installed on my PC and Android phone
o Zoom - already in use
o MS Office - Already in use
o A compatible Internet Browser (I have Chrome and Opera, both seem to work well)
· By Students
o They will need to install and get familiar with Quip before or in the first few days of the course
o They will need to have one or more working IDEs for developing their project software (Matlab, Python, Java or C/C++ as the course is programming language agnostic).
o A working Internet Browser (Firefox, Chrome, Safari, ...) (I have found that Opera also works)
o The ability to create/edit/render several file types (PDF, MPEG4, DOCX)
As indicated by Fairfield University’s policy, any student missing classes or
other assigned work due to religious holiday observances shall have the
opportunity to make up that work during the term. It is the student’s
responsibility to notify the instructor in writing at least two weeks prior to
any observed holiday.
It is expected that all work done for this class will be in strict compliance
with the principles of academic honesty and integrity, as outlined in the
Student Catalog. Cheating, plagiarism, copying, or dishonesty of any kind is
not acceptable and will result in a zero for the exam or assignment and
possible a failing grade for the class, and/or expulsion from the program.
Fairfield University is committed to achieving equal educational opportunities
and providing students who have documented disabilities equal access to all
University programs, services and activities. For this
course to be accessible to all students, different accommodations or
adjustments may need to be implemented. If you require accommodations for this
course because of a disability, please contact the Office of Accessibility
(OOA) as early as possible this semester as they are the designated department
on campus responsible for approving and coordinating reasonable accommodations
for students with disabilities. The Office of Accessibility will help you
understand your rights and responsibilities under the American with
Disabilities Act and Section 504 of the Rehabilitation Act and will provide you
further assistance with requesting and arranging accommodations for your
courses. Once you receive your accommodations letter, please make an
appointment to meet with me as soon as possible so that we can review your
accommodations together and discuss how best to help you achieve equal access
in this course this semester.
Office of Accessibility, 203-254-4081, ooa@fairfield.edu