[Sigia-l] RE: Data vs. Information

[Christopher Fahey [askrom]]

Ed Housman wrote:
> bit (rejected as a Shannon concept)

Do you mean that you are rejecting the term "bit" because it is a
Shannon concept? It's a pretty good term. 

As I recall from Shannon, although information is measurable, it's a bit
tricky to define a standard "unit" of information since information is
so dependent on the mode of transmission, storage, and
encoding/compression. 

Here's a great example: Let's say I flip a coin eight times and write
down the results ("1:heads, 2:tails, 3:heads, 4:heads, 5:tails, 6:heads,
7:tails, 8:tails"). How many bits of bandwidth would it take for me to
telegraph this sequence to you? You might say eight at first, but
Shannon demonstrated that it can be done in fewer bits... as long as we
have a system in place. Let's say that the system is this: I will send
you 1 bit at 1-second intervals, but only if the bit is different from
the one before it. In the example above, you would receive this sequence
{1,0,1,_,0,1,0,_}. I have only sent you six bits, not eight.

One may argue that the act of *not* sending information at any given
interval is tantamount to sending information (a null bit), but there is
an important difference: the channel (the telegraph line) on which I
would have sent this information is now free for other information to
flow through (this is, of course, the groundwork for data compression).
This practical aspect of Shannon's work results, of course, from the
fact that his research was done on behalf of Bell Labs, not an academic
institution. 

The example above is a very simple model of Shannon's theory that
information is measurable... bits, words, or whatever. Shannon used the
term "symbols", I believe, but in general he used bits (base 2) because
it is the simplest symbolic language. It get even more interesting as
Shannon then begins to translate these ideas into mathematical formulas
based on probability and stuff. Again, if you want to "get" this stuff
and read some great stories about Bell Labs in its heyday, read Robert
Lucky's "Silicon Dreams":
  http://www.amazon.com/exec/obidos/tg/detail/-/031205517X/

-Cf

[christopher eli fahey]
art: http://www.graphpaper.com
sci: http://www.askrom.com
biz: http://www.behaviordesign.com