[rexlunae]

Originally Posted by Stripe

And you think that makes sense?

Yes, if you think about it. I think it is counter-intuitive though.

The problem with saying that it is a decrease in uncertainty in a receiver is that it implies that information is only information when it is being transmitted. When it's not being transmitted, it must be something else...in which case, what is it?

[Stripe]

Originally Posted by rexlunae

Yes, if you think about it. I think it is counter-intuitive though.

The problem with saying that it is a decrease in uncertainty in a receiver is that it implies that information is only information when it is being transmitted. When it's not being transmitted, it must be something else...in which case, what is it?

You're worried about what to call something that isn't happening?

Try, "nothing".

[rexlunae]

Originally Posted by Stripe

Don't worry. We'll wait for you to get your footing again.

Originally Posted by Stripe

You're worried about what to call something that isn't happening?

Try, "nothing".

How about a simple thought experiment then?

Suppose you have two containers filled with the same number of atoms. And suppose that in one container, the atoms were randomly arranged, and in the other container, the atoms were arranged in a 3-d grid. If you were to describe the exact positions of all of the atoms in each container, which one could you describe with less writing?

Interestingly, even using your "decrease in uncertainty" definition, it seems like there is more information in the randomly arranged container than the gridded one.

[Stripe]

Originally Posted by rexlunae

Suppose you have two containers filled with the same number of atoms. And suppose that in one container, the atoms were randomly arranged, and in the other container, the atoms were arranged in a 3-d grid. If you were to describe the exact positions of all of the atoms in each container, which one could you describe with less writing?

The decrease in uncertainty for the receiver would be greater for the ordered box upon seeing the same amount of data from both.

Interestingly, even using your "decrease in uncertainty" definition, it seems like there is more information in the randomly arranged container than the gridded one.

No, there isn't. There is the same amount of information in both. You need to measure any difference at the receiver end.

[rexlunae]

Originally Posted by Stripe

The decrease in uncertainty for the receiver would be greater for the ordered box upon seeing the same amount of data from both.

That's an interesting response, but it doesn't answer the question. Which one could be described with less writing?

[Stripe]

Originally Posted by rexlunae

That's an interesting response, but it doesn't answer the question. Which one could be described with less writing?

Oh, you want to talk compression now?

Because he knows the codec, sending the same amount of compressed data for both will see the receiver's uncertainty drop a lot more for the ordered box than in the uncompressed state.

[rexlunae]

Originally Posted by Stripe

Sending the same amount of compressed data will see the receiver's uncertainty drop a lot more for the ordered box than in the uncompressed state.

In my example, the change in uncertainty is the same. The receiver goes from having no knowledge of the arrangement of the atoms inside the container to having complete knowledge of the position of each atom (i.e, no uncertainty). However, one message is very compressible, and the other is not. And the reason that one message is more compressible is that it fundamentally contains less information.

[The Barbarian]

Barbarian observes:
It makes perfect sense. A message with zero uncertainty has zero information.

You're right. I am 100% certain about everything you said here and it is complete nonsense.

Someone's cranky, today. But you got the idea wrong again. The uncertainty is before the message is received. High uncertainty, means high information. Low uncertainty is low information. If you know in advance, exactly what the message will be, there is zero information.

Barbarian observes:
And, as you know, Shannon's approach (which your guy Schnieder finds incomprehensible) is used every day by communications engineers to maximize the amount of information that can go through limited channels.

I'm sure not-your-guy is well aware of this and fully capable of repeating such analysis.

I don't think so. He would build the channels exactly backwards. His complaint, as you see, is that the information people are using Shannon's discovery that high information means high uncertainty to do their work.

Pays to engage in the actual debate rather than looking desperately for a way out. If you're wrong, there's no harm in admitting such.

You say that, but then you act as though you immediately forgot it.

Barbarian observes:
Information is the amount of uncertainty in a message.

Utter gobbledegook!

Let's see... you have two messages coming in. One is from a channel that only outputs strings of "1" with the same number of digits every time. Another will print out a description of ways the uncertainty equation can be used in reducing noise in the channel. Which of those will have a higher uncertainty in the content of the message?

Which of those will have more information for you? If you can figure out that, you will understand why Schnieder is wrong, and the guys who manage flow of information on the internet are right.

If you have a device that can only send strings of "1111...", how much uncertainty will the message have? How much information will be in the message you get that way?

Depends.

The correct answer is "close to zero."

Before the message is sent, the receiver has some uncertainty about its timing and length.

If the length of the string is also constant, and the time is known, then the correct answer is "zero."

After the message is received, those questions are resolved.

Shannon's equation refers to the uncertainty in the message, not in the receiver after the message arrives. That's another goof Schnieder made.

[Stripe]

Originally Posted by rexlunae

In my example, the change in uncertainty is the same. The receiver goes from having no knowledge of the arrangement of the atoms inside the container to having complete knowledge of the position of each atom (i.e, no uncertainty). However, one message is very compressible, and the other is not. And the reason that one message is more compressible is that it fundamentally contains less information.

The receiver needs to know the codec. With the codec, he can garner more from a properly encoded message than in an uncompressed example.

There was no need for you to begin talking about compression. My explanation is the same.

[noguru]

Originally Posted by Stripe

The receiver needs to know the codec. With the codec, he can garner more from a properly encoded message than in an uncompressed example.

There was no need for you to begin talking about compression. My explanation is the same.

In the natural world of genetics the codec is inherent in the system. Otherwise it would never have gotten off the ground floor. The ability for genetic makeup to have a representation in the natural world is part and parcel the intitial system. Because such information is stringently based upon that relationship.

As we map the human genome we are discovering a codec that has been part of nature since the very beginning. Understanding this codec is a progressive pursuit for humans, though such discovery is anti-climactic to the natural world.