> (Another exampe, the entropy of a
> physical system is calculated in terms of energy in joules or calories,
> while the entropy in information theory is in terms of bits).
Just by the by: physical entropy is measured in joules per kelvin. The
connections between physical entropy and information have been developed
by a bundle of people, with foundational work by Maxwell, Szilard (see
the book by Wheeler and Zurek, 1983), and Landauer (IBM Journal of Research
and Development, 1961). Bennett wrote a lovely article in Scientific
American on the subject in 1987; he and Landauer have written another Sci
Am article on the subject, although I don't recall the year (1985 or
circa 1990, I think).
Michael Nielsen
http://wwwcas.phys.unm.edu/~mnielsen/index.html