Course Description

In this course, students learn the information theory that is a theory of digital communications and storage which supports the information society of nowadays.

Information theory is a theory that deals with the theoretical bounds of encoding and concrete encoding algorithms. In this theory, encodings are classified into source coding to increase the efficiency of communications, and into channel coding to increase the reliability of communications.

In this course, students aim at being possible to discuss theoretically the infimum of the average code length of source coding, and the supremum of the code rate of channel coding without errors giving the probabilistic model of a source or a channel. In addition, students aim at being able to decide which encoding algorithm is effective for a given purpose concretely.

Moreover, this course deals with analog source/channel, analog-to-digital or digital-to-analog conversion, the sampling theory, the character encoding, the relation between information theory and cryptology.

Students acquire skills related to the diplomatic policy 2 of Department of Information Science Correspondence Course.

Course Objectives

The goal of this course is that students master the following abilities;

Students can explain the relation of system model of communication, noise source, source coding and channel coding.
Students can explain the purpose of source/channel coding, the meanings of Shannon's source coding theorem and Shannon's noisy-channel coding theorem.
Students can explain the model of the memoryless source, source with memory, the memoryless channel, the burst channel.
Students can explain features which source coding algorithms should have by using code tree.
Students can process basic source coding/encoding algorithm as for basic source coding algorithms.
Students can explain the amount of information, entropy, mutual information, and can compute those values
as for some basic sources.
Students can explain the meanings of channel capacity, and compute that as for some basic channels.
Students can process basic binary source coding algorithms as for some basic channel coding algorithms.
Students understand the sampling theorem and can obtain appropriate sampling frequency given maximum frequency of a signal.
Students can explain the necessity of character encoding and the features of representative character encoding.

Grading Policy

Grading policy: Examination(100%).

The way of feedback;
Answers for questions or feedback for the contents of class and examination will be given in a class, through LMS or during office hours.

Textbook and Reference

Requirements(Assignments)

Before each class, materials related to the class will be published through LMS. Students should download them to their own devices or print them out to make it possible to refer to or to take notes.
Students should read these materials and grasp what they do not understand and they understand in an hour.
After each class, student should review the class through tests on the LMS in half an hour.

Note

In order to earn credits of this course, students must submit two reports and get 60% points for each report before taking an examination.

Before taking this course, students should take the following courses;
Linear Algebra, Mathematical Logic, Discrete Mathematics, Mathematical Statistics and Computer Networks.

At the same semester with this course, students should take the following courses;
Information Security, Digital Image Processing and Digital Communications.

After taking this source, students should take Information Security.
Digital Signal Processing 1 and DIgital Signal Processing 2.

Schedule