标签：information What Things uncertainty when entropy Shannon more

52 Things: Number 9: What are Shannon's definitions of entropy and information?

52 件事：数字 9：香农对熵和信息的定义是什么？ This is the latest in a series of blog posts to address the list of '52 Things Every PhD Student Should Know To Do Cryptography': a set of questions compiled to give PhD candidates a sense of what they should know by the end of their first year. This blog post tries to introduce some fundamental concepts in Information Theory: What are Shannon's definitions of entropy and information?
这是一系列博客文章中的最新一篇，旨在解决“每个博士生都应该知道的 52 件事来做密码学”列表：一组问题，旨在让博士生在第一年结束时了解他们应该知道什么。这篇博文试图介绍信息论中的一些基本概念：香农对熵和信息的定义是什么？ Information Theory was founded by Claude E. Shannon in 1948. It was originally developed to study signal processing but its application has been broadened to various disciplines through decades. This blog is intended to briefly introduce two fundamental concepts, entropy and information. If you are interested in more details, I personally suggest you to find more in [1].
信息论由 Claude E. Shannon 于 1948 年创立。它最初是为了研究信号处理而开发的，但几十年来，它的应用已经扩展到各个学科。本博客旨在简要介绍两个基本概念，熵和信息。如果你对更多细节感兴趣，我个人建议你在 [1] 中找到更多。 Entropy 熵 Entropy is a measurement to evaluate the uncertainty[3] of one or more variables.
熵是评估一个或多个变量的不确定性的度量[3]。 Assume we are investigating the first web page people visit when they open a web browser. We use samples from two groups of people: 4 cryptographer from Bristol Cryptogroup and 4 passenger grabbed at Bristol coach station. Let's make a more ambitious assumption that the cryptographers always visit http://bristolcrypto.blogspot.co.uk/ first when they open a web browser, while the others each visits a different portal.
假设我们正在调查人们打开 Web 浏览器时访问的第一个网页。我们使用来自两组人的样本：来自布里斯托尔加密组的 4 名密码学家和在布里斯托尔长途汽车站抓获的 4 名乘客。让我们做一个更雄心勃勃的假设，即密码学家在打开 Web 浏览器时总是首先访问 http://bristolcrypto.blogspot.co.uk/，而其他人则各自访问不同的门户。 Now let's evaluate the uncertainty of their answers: apparently, the answers from cryptographers are quite certain (low uncertainty) whilst it can hardly be guessed if an answer is from a passenger (high uncertainty). In other words, we say the answers in the cryptographers group has a low entropy but that in the passengers group has a high entropy.
现在让我们评估一下他们答案的不确定性：显然，密码学家的答案是相当确定的（低不确定性），而如果答案来自乘客，则很难猜测（高不确定性）。换句话说，我们说密码学家组的答案具有低熵，但乘客组的答案具有高熵。 So one of the most remarkable inventions of Shannon's is his definition of entropy(Shannon's Entropy):
因此，香农最引人注目的发明之一就是他对熵的定义（香农的熵）：
H=−∑ipilogbpi

where pi is the probability of a message (an answer in previous example) appears. In computer science, we usually use b=2 (bits).
其中 pi 是消息（上一示例中的答案）出现的概率。在计算机科学中，我们通常使用 b=2 （位）。
If we compute the entropy in our example, we will have:
如果我们在示例中计算熵，我们将得到：

Hcryptographer=−∑411log21=0Hpassenger=−∑4114log214=2
So the passengers' answer do have a higher entropy than the cryptographers'!
因此，乘客的答案确实比密码学家的熵更高！ Information 信息 Formally, the definition of Shannon's information is given in [2] as:
从形式上讲，香农信息的定义在[2]中给出为： " Information is a measure of one's freedom of choice when one selects a message."
“信息是衡量一个人在选择信息时选择自由度的指标。” To explain this, let's make a minor modification to our previous example. Let's grab another 4 passengers from Bristol train station and assume their answers are also random portals just like the passengers' in coach station.
为了解释这一点，让我们对前面的示例进行一些小的修改。让我们从布里斯托尔火车站抓起另外 4 名乘客，并假设他们的答案也是随机的门户，就像长途汽车站的乘客一样。 Here is the question: Given an answer y, can you tell which group the answer is from?
问题来了：给定一个答案 y ，你能说出答案来自哪个群体吗？ We can instantly tell that the answer is from our cryptographer group if y is http://bristolcrypto.blogspot.co.uk/. However, we will be struggling if y is a portal; therefore, we could say the message of http://bristolcrypto.blogspot.co.uk/ contains more information (less freedom) than a portal (more freedom).
我们可以立即判断答案来自我们的密码学家小组，如果 y http://bristolcrypto.blogspot.co.uk/。但是，如果 y 是一个门户，我们将苦苦挣扎;因此，我们可以说 http://bristolcrypto.blogspot.co.uk/ 的信息比门户（更多的自由）包含更多的信息（更少的自由）。 So how does it relate to entropy?
那么它与熵有什么关系呢？ Extending the definition of entropy, we define the Conditional Entropy as:
扩展熵的定义，我们将条件熵定义为： H(Y|X)=∑x∈Xp(x)H(Y|X=x) which describes the entropy of Y when given condition X=x. To make it more explicitly, since entropy is the uncertainty of a variable; hence the previous definition of conditional entropy is in fact the uncertainty of Y when given the "clue"(condition) X.
它描述了给定条件 X=x 时的 Y 熵。更明确地说，因为熵是变量的不确定性;因此，条件熵的先前定义实际上是给定“线索”（条件） X 时的 Y 不确定性。 Observation: consider two variables X and Y. If X contains only a minimal information of Y, then given an exact value of X should not help us much on deducing the value of Y, that is, it does not obviously reduce the uncertainty of Y; on the other hand, if X contains essential information of Y, then the entropy of Y is expected to be low when X is given. Therefore, the conditional entropy can be viewed as a rational measurement to the information X has of Y!
观察：考虑两个变量 X 和 Y 。如果 X 只包含的 Y 极小信息，那么给定一个精确值应该 X 对我们推导的 Y 值没有多大帮助，也就是说，它不会明显降低的 Y 不确定性;另一方面，如果 X 包含的基本 Y 信息，则在给定时 X 的 Y 熵预计较低。因此，条件熵可以看作是对的信息 X Y 的有理度量！ Another important measurement is called Mutual Information. It is a measure of mutual dependency among two variables. One way to define it is the loss of entropy(uncertainty) when given a condition:
另一个重要的衡量标准称为互信息。它是两个变量之间相互依赖性的度量。定义它的一种方法是给定条件时熵的损失（不确定性）：
I(X;Y)=H(X)−H(X|Y)=H(Y)−H(Y|X) Cryptographic Example 加密示例 The concepts of information theory is widely used in cryptography. A classic example is to view a cryptographic process as a channel with plaintext and ciphertext being input and output respectively. Research of side channel analysis also benefits from the usage of information theory.
信息论的概念在密码学中被广泛使用。一个典型的示例是将加密过程视为一个通道，其中明文和密文分别是输入和输出。侧信道分析的研究也受益于信息论的使用。

标签：information,What,Things,uncertainty,when,entropy,Shannon,more
From： https://www.cnblogs.com/3cH0-Nu1L/p/18104685

52 Things: Number 9: What are Shannon's definitions of entropy and information?

52 Things: Number 9: What are Shannon's definitions of entropy and information?

相关文章

赞助商

阅读排行