Entropy and information rates for hidden Markov models

Hanseok Ko; R. H. Baran

doi:10.1109/ISIT.1998.708979

Entropy and information rates for hidden Markov models

Hanseok Ko, R. H. Baran

School of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

A practical approach to statistical inference for hidden Markov models (HMMs) requires expressions for the mean and variance of the log-probability of an observed T-long sequence given the model parameters. From the viewpoint of Shannon theory, in the limit of large T, the expected value of the per step log-probability is minus one times the mean entropy rate at the output of a noisy channel driven by the Markov source. A novel procedure for finding the entropy rate is presented. The rate distortion function of the Markov source, subject to the requirement of instantaneous coding, is a by-product of the derivation.

Original language	English
Title of host publication	Proceedings - 1998 IEEE International Symposium on Information Theory, ISIT 1998
Number of pages	1
DOIs	https://doi.org/10.1109/ISIT.1998.708979
Publication status	Published - 1998
Event	1998 IEEE International Symposium on Information Theory, ISIT 1998 - Cambridge, MA, United States Duration: 1998 Aug 16 → 1998 Aug 21

Publication series

Name	IEEE International Symposium on Information Theory - Proceedings
ISSN (Print)	2157-8095

Other

Other	1998 IEEE International Symposium on Information Theory, ISIT 1998
Country/Territory	United States
City	Cambridge, MA
Period	98/8/16 → 98/8/21

ASJC Scopus subject areas

Theoretical Computer Science
Information Systems
Modelling and Simulation
Applied Mathematics

Access to Document

10.1109/ISIT.1998.708979

Cite this

Ko, H & Baran, RH 1998, Entropy and information rates for hidden Markov models. in Proceedings - 1998 IEEE International Symposium on Information Theory, ISIT 1998., 708979, IEEE International Symposium on Information Theory - Proceedings, 1998 IEEE International Symposium on Information Theory, ISIT 1998, Cambridge, MA, United States, 98/8/16. https://doi.org/10.1109/ISIT.1998.708979

@inproceedings{df00599b7d2b4ea29b7460980f165192,

title = "Entropy and information rates for hidden Markov models",

abstract = "A practical approach to statistical inference for hidden Markov models (HMMs) requires expressions for the mean and variance of the log-probability of an observed T-long sequence given the model parameters. From the viewpoint of Shannon theory, in the limit of large T, the expected value of the per step log-probability is minus one times the mean entropy rate at the output of a noisy channel driven by the Markov source. A novel procedure for finding the entropy rate is presented. The rate distortion function of the Markov source, subject to the requirement of instantaneous coding, is a by-product of the derivation.",

author = "Hanseok Ko and Baran, {R. H.}",

year = "1998",

doi = "10.1109/ISIT.1998.708979",

language = "English",

isbn = "0780350006",

series = "IEEE International Symposium on Information Theory - Proceedings",

booktitle = "Proceedings - 1998 IEEE International Symposium on Information Theory, ISIT 1998",

note = "1998 IEEE International Symposium on Information Theory, ISIT 1998 ; Conference date: 16-08-1998 Through 21-08-1998",

}

TY - GEN

T1 - Entropy and information rates for hidden Markov models

AU - Ko, Hanseok

AU - Baran, R. H.

PY - 1998

Y1 - 1998

N2 - A practical approach to statistical inference for hidden Markov models (HMMs) requires expressions for the mean and variance of the log-probability of an observed T-long sequence given the model parameters. From the viewpoint of Shannon theory, in the limit of large T, the expected value of the per step log-probability is minus one times the mean entropy rate at the output of a noisy channel driven by the Markov source. A novel procedure for finding the entropy rate is presented. The rate distortion function of the Markov source, subject to the requirement of instantaneous coding, is a by-product of the derivation.

AB - A practical approach to statistical inference for hidden Markov models (HMMs) requires expressions for the mean and variance of the log-probability of an observed T-long sequence given the model parameters. From the viewpoint of Shannon theory, in the limit of large T, the expected value of the per step log-probability is minus one times the mean entropy rate at the output of a noisy channel driven by the Markov source. A novel procedure for finding the entropy rate is presented. The rate distortion function of the Markov source, subject to the requirement of instantaneous coding, is a by-product of the derivation.

UR - http://www.scopus.com/inward/record.url?scp=84890370594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84890370594&partnerID=8YFLogxK

U2 - 10.1109/ISIT.1998.708979

DO - 10.1109/ISIT.1998.708979

M3 - Conference contribution

AN - SCOPUS:84890370594

SN - 0780350006

SN - 9780780350007

T3 - IEEE International Symposium on Information Theory - Proceedings

BT - Proceedings - 1998 IEEE International Symposium on Information Theory, ISIT 1998

T2 - 1998 IEEE International Symposium on Information Theory, ISIT 1998

Y2 - 16 August 1998 through 21 August 1998

ER -

Entropy and information rates for hidden Markov models

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this