|
|
| BILL7253 | @Algorithms and Applications in Multimodal Signal Processing | 3+0+0 | ECTS:7.5 | | Year / Semester | Fall Semester | | Level of Course | Third Cycle | | Status | Elective | | Department | DEPARTMENT of COMPUTER ENGINEERING | | Prerequisites and co-requisites | None | | Mode of Delivery | | | Contact Hours | 14 weeks - 3 hours of lectures per week | | Lecturer | Dr. Öğr. Üyesi Bahar HATİPOĞLU YILMAZ | | Co-Lecturer | - | | Language of instruction | Turkish | | Professional practise ( internship ) | None | | | | The aim of the course: | | The aim of this course is to ensure that the student becomes well-acquainted with the current literature on the subject, gains an in-depth understanding of multimodal signal processing strategies, and applies these strategies both theoretically and practically with various recognition algorithms. |
| Programme Outcomes | CTPO | TOA | | Upon successful completion of the course, the students will be able to : | | | | PO - 1 : | Proficiency in multimodal signal processing and machine learning methodology. | 1,2 | 1,3,6, | | PO - 2 : | Gaining the ability to integrate modalities in multimodal systems and develop a comprehensive understanding for this data. | 1,2 | 1,3,6, | | PO - 3 : | They will possess the ability to utilize, adapt, and if necessary update machine learning algorithms in multimodal signal processing systems, thereby transforming theoretical knowledge into practical application. | 1,2 | 1,3,6, | | PO - 4 : | They will have the ability to solve complex recognition and classification problems encountered in real-world applications, as well as the skills to cope with and address challenges in multimodal signal processing and recognition. | 1,2 | 1,3,6, | | CTPO : Contribution to programme outcomes, TOA :Type of assessment (1: written exam, 2: Oral exam, 3: Homework assignment, 4: Laboratory exercise/exam, 5: Seminar / presentation, 6: Term paper), PO : Learning Outcome | | |
| The course involves essential stages such as the holistic analysis, processing, and fusion of multimodal signals obtained through different sensors (such as facial images, audio, and biomedical signals). Additionally, the course content will commence with fundamental signal processing processes and continue with the application of machine learning methods used for integrating different modalities. As a result, the course content encompasses the processes of analyzing-synthesizing multimodal datasets and applying the problem to real-world scenarios. |
| |
| Course Syllabus | | Week | Subject | Related Notes / Files | | Week 1 | Introduction to machine learning algorithms for biomedical signal processing applications. | | | Week 2 | Introduction to signal processing, modeling, and related mathematical tools (introduction to statistical learning). | | | Week 3 | Applications for signal processing and modeling. | | | Week 4 | Image and video processing analysis for brain-computer interfaces. | | | Week 5 | Analysis of handwriting and other modalities. | | | Week 6 | Introduction to multimodal signal processing. | | | Week 7 | Explaining and applying the fundamental concepts of multimodal analysis. | | | Week 8 | Introduction to multimodal fusion methods: Sensor, feature, and decision level fusion. | | | Week 9 | Midterm Exam | | | Week 10 | Application of sensor, feature, and decision level fusion methods to multimodal problems. | | | Week 11 | Analysis and applications of multimodal fusion methods. | | | Week 12 | Multimodal data management: Explaining challenges and solutions. | | | Week 13 | Multimodal human-computer and human-human interactions. | | | Week 14 | Introduction to multimodal datasets and their usage in problem-solving. | | | Week 15 | Assignment presentations. | | | Week 16 | Final Exam | | | |
| 1 | Thiran, J. P., Marques, F., and Bourlard, H., 2009. Multimodal Signal Processing: Theory and applications for human-computer interaction. | | | |
| 1 | Jabbar, M. A., Kantipudi, M. P., Peng, S. L., Reaz, M. B. I. and Madureira, A. M., 2022, Machine Learning Methods for Signal, Image and Speech Processing. River Publishers. | | | |
| Method of Assessment | | Type of assessment | Week No | Date | Duration (hours) | Weight (%) | | Mid-term exam | 9 | | 2 | 20 | | Homework/Assignment/Term-paper | 15 | | 10 | 30 | | End-of-term exam | 16 | | 2 | 50 | | |
| Student Work Load and its Distribution | | Type of work | Duration (hours pw) | No of weeks / Number of activity | Hours in total per term | | | | | |
|