Course Description

In this course, students learn the information theory that is a theory of digital communications and storage which supports the information society of nowadays.

Information theory is a theory that deals with the theoretical bounds of encoding and concrete encoding algorithms. In this theory, encodings are classified into source coding to increase the efficiency of communications, and into channel coding to increase the reliability of communications.

In this course, students aim at being possible to discuss theoretically the infimum of the average code length of source coding, and the supremum of the code rate of channel coding without errors, giving the probabilistic model of a source or a channel. In addition, students aim at being able to decide which encoding algorithm is effective for a given purpose concretely.

Moreover, this course deals with analog source/channel, analog-to-digital or digital-to-analog conversion, the sampling theory, character encoding, the relation between information theory and cryptology.

Students acquire skills related to the diplomatic policy, DP4C.

Course Objectives

The goal of this course is that students master the following abilities;

Students can explain the relation of system model of communication, noise source, source coding and channel coding.
Students can explain the purpose of source/channel coding, the meanings of Shannon's source coding theorem and Shannon's noisy-channel coding theorem.
Students can explain the model of the memoryless source, source with memory, the memoryless channel, the burst channel.
Students can explain features which source coding algorithms should have by using code tree.
Students can process basic source coding/encoding algorithm as for basic source coding algorithms.
Students can explain the amount of information, entropy, mutual information, and can compute those values
as for some basic sources.
Students can explain the meanings of channel capacity, and compute that as for some basic channels.
Students can process basic binary channel coding algorithms as for some basic channel coding algorithms.
Students understand the sampling theorem and can obtain appropriate sampling frequency given maximum frequency of a signal.
Students can explain the necessity of character codes and the features of representative character codes.

Grading Policy

Grading policy:
Midterm report(50%), Examination(50%).

The way of feedback;
Answers for questions or feedback for the contents of class, worksheets, and examination will be given in a class, through LMS or during office hours.

Textbook and Reference

Requirements(Assignments)

Before each class, materials related to the class will be published through LMS. Students should download them to their own devices or print them to make it possible to refer to or to take notes.
Students should read these materials and grasp what they do not understand and they understand in an hour.
After each class, student should review the class through tests on the LMS in half an hour.

Note

Students can hardly earn credits not submitting the mid-term report. Thus, it is expected students to observe the deadline.
As for the self-learning support students are expected to utilize materials, such as slides, handouts and quizzes on the LMS.

Before taking this course, students should take the following courses;
Mathematical Logic, Linear Algebra, Discrete Mathematics, Applied Mathematics, Mathematical Statics, Digital Image Processing.

At the same semester with this course, students should take Multimedia Information Processing.
After taking this course, students should take the following courses;
Introduction to Stochastic Processes, Information Security, Communication, Communication Systems.

If a student has a question on quizzes or mid-term report or examinations, ask the question in the class or in office hours or through LMS.
This course is a required course, and relates to the mid term 4-1 of the attaining targets for learning and educating, in the JABEE program.

Schedule