Explore
This article is from
Journal of Creation 26(3):115–119, December 2012

Browse our latest digital issue Subscribe

Information Theory—part 3: introduction to Coded Information Systems

by

The literature about information is confusing because so many properties are described for supposedly a singular entity. The discussion can be more fruitful once we realize we are studying systems with many components, one of which is a coded message. We introduce the notion of a Coded Information System (CIS) and can now pose an unambiguous question: “Where do CISs come from?”, which should be more precise than the vague alternative, “Where does information come from?” We can develop a model which is quantifiable by focusing on the effects a CIS has on organizing matter through a sequential set of refining steps.


Coded-Information

In part 1 of this series1 I demonstrated that there are many usages of the word information, with many specialists working on different notions. Dretske points out that “It is much easier to talk about information than it is to say what it is you are talking about … . It has come to be an all-purpose word, one with the suggestive power to fulfil a variety of descriptive tasks.”2 In part 2 of this series3 I drew attention to issues in various information theoretic models which seem problematic. There seems to be a common intuition that information leads to a desired outcome. But is information only vaguely (if at all) involved in attaining the intended goal (as implied by Shannon’s theory4) or fully, as Gitt maintains?5

Coded messages play a prominent role in Gitt’s framework,6,7 and are clearly indispensable for the first three levels of his model (statistics, cosyntics, and semantics) but it is not apparent how symbolic messages appear directly in the last two levels (pragmatics and apobetics). And what exactly is a coded message? The gun fired to start a race consists of only one symbol. Statistics and cosyntics are missing, but meaning (semantics) is present. Was a message sent? Is this information?

Schneider claims8 to show with a computer program that information can arise for free, autonomously, but Dembski argues decisively9 that the necessary resources were intelligently embedded into the program in different ways, and shows that information is provided whenever a suitable search algorithm is selected from among other possible ones.10–15 Can these ideas be reconciled to permit a coherent discussion?

Sometimes information is claimed to cause something via mechanical means. For example, the direction and force generated by a billiard cue has been said to provide the information to guide the ball. But all natural causes lead to some effect! So when is information involved? Surely information is more than mere cause–effect mechanics.

Now, machines are also used by living beings to achieve a goal. Some, like computers, work with coded messages. What about a watermill which grinds grain into meal? The water provides the energy needed for the machine to work. One could adjust the amount of force delivered upon the rotating wheel by changing the amount of water provided, and the drop height. But there is no coded message in this kind of machine.

Although the disagreements about how to define information are rampant, virtually no one would argue the subject matter is vacuous, a meaningless debate of empty words.

Pioneering thinker Norbert Wiener stated correctly that “Information is information, neither matter nor energy”, but this left unanswered what it is. And even experts vacillate between different meanings of the word, so readers might not know exactly what is implied in each case. The confusion arises from a multitude (of sometimes only weakly related) ideas applied to a single word, information.

To illustrate, Gitt assigns both statistics and apobetics to information in living systems, but how and where? On DNA? Statistics can indeed be discerned from gene sequences, but surely not from intended purpose (apobetics). The goal does not reside on DNA, fully nor implied. As we’ll see later, DNA is only one of multiple contributing factors to produce an intended outcome.

As a second example, one of Gitt’s Universal Laws of Information is: “SLI-4c. Every information transmission chain can be traced back to an intelligent sender”.16 In the same paper he also writes, “Remark R3: The storage and transmission of information requires a material medium.” It seems that transmission chain must be referring to coded messages, a stream of symbols on a physical medium. However, why must the other mandatory elements of his information or Universal Information (semantics, pragmatics, and apobetics) reside on, and be transmitted by, a material medium? Must an intelligent mind and all its parts be 100% material? Also for angels and for God?

I am not claiming there is contradiction in what Gitt writes. In fact, I edited and endorsed his last book.5 Careful consideration of his work reveals that ‘information’ is somehow distributed in separate, organized ensembles of matter, energy, and mind (e.g. the statistics vs the pragmatics portion) with different properties and functions. This makes an answer to “What is information?” almost impossible. And it leads to a struggle to find words with compound meanings to convey the multiplicity of functions assigned to information. His second law states, “SLI-2 Universal information is a non-material fundamental entity”.

Entity, in this statement, merely replaces Universal Information, and one does not know what it might mean. It reflects the search for a missing, suitable explanatory construct and therefore provides no additional insight beyond the phrase “Universal information is non-material”. But I believe the simple proposal introduced below will retain almost all his views in a coherent manner.

When someone asks, “Where does the information come from which causes a fertilized egg to become an adult?” it seems that a series of linked, guided processes are implied. Processes is plural, whereas a singular word, information, does not capture this intuition very well.

What needs to be explained?

Analysing the world around us, we note a family of phenomena which are not explained by deterministic law or random behaviour. Examples include:

  • Birds migrate to specific locations during certain time periods.
  • Thousands of proteins are formed each minute in a cell and their concentrations and locations are carefully regulated.
  • A few bacteria can reproduce into a large colony, metabolizing nutrients to survive.
  • A foetus develops into an adult.
  • Caterpillars metamorphose into butterflies.
  • Assembly lines produce hundreds of cars each day.
  • A few years after lava devastates a landscape, a new ecology develops.
  • Text on a computer screen can be transferred to a printed sheet of paper.
  • Deaf people communicate with a sign language.
  • Satellites are sent to a planet and back.
Figure1
Figure 1. Complex equipment sends symbols to a Receiver able to receive and process the coded message. The shapes between Message Sender and Message Receiver represent symbols of a coded message.

The above outcomes occur repeatedly, and what we observe does not follow naturalistic (mechanical) principles. Some observations become readily apparent.

Observation 1. A series of linked processes are involved.

Observation 2. Members of these processes sequentially refine and contribute towards a goal.

Observation 3. A coded message is used somewhere along the chain of processes. Complex equipment generates a series of symbols, usually embedded on a physical medium,17 which another piece of complex equipment receives, resulting in a measurable change in behaviour of a system attached to the Message Receiver (figure 1).

Observation 4. All these kinds of systems are associated with living organisms.

Making a fresh start

To uniquely specify our area of interest, we exclude all systems and machines which do not use a coded message somewhere in the process. We are left with phenomena which have something to do with ‘information’, and we wonder where such systems come from. But asking, “Where does information come from?” is too vague for our scientific enterprise. Clearly we are observing systems, with many independent, but linked, components. We need a definition for these message-based systems and then we need to consider how they could arise. Based strictly on observation, we make the following definition:

A Coded Information System (CIS) consists of linked tools or machines which refine outcomes to attain a specific goal. A coded message plays a prominent role between at least two members of this linked series.

CIS theory recognizes Gitt’s five sequential processes: statistics, cosyntics, semantics, pragmatics, and apobetics.5

Messages vs sensors in CIS theory

Coded messages are formed by ordered codewords,18 which themselves consist of symbols from a coding alphabet. Messages must conform to the grammatical rules devised for that coding system.

Cues or sensors are often found in a CIS but should not be considered coded messages. For example, a sensor could be composed of two metal parts, the volumes of which respond differently to temperature. When the temperature increases, selective expansion of one of the metals causes the construct to bend, bringing the tip of the sensor into contact with a critical element to trigger an action (such as by permitting a current to flow).

Taste and smell receptors are unique to specific chemical structures, and are also sensors. If interaction at a detector is a simple physical effect, and a signal is transmitted without an alphabet of symbols which are independent of the carrier, then we have a sensor and not a coded message.19 However, sensors are often valuable components of a CIS, and signals received by sensors could be converted into coded messages, as will be shown later.

As one example, barn owls use two methods to localize sounds: the time differential between the arrival of a sound at each ear (the interaural time) and the variance in the sound’s intensity as it arrives at each ear.20 Are these cues, interacting directly with the external physical factors, coded messages? Not at the point of external contact, which is based on strict physical relationships, with no alphabet, nor grammar.

As another example, a photoreceptor on a retina absorbs a photon, causing 11-cis-retinal to isomerize to 11-transretinal, which is followed by a signal cascade. This initially strictly physical behaviour is characteristic of sensors. The location at which the photon lands on the retina determines in most cases where the signal will be transferred to in the primary visual cortex of the occipital lobe.21 The cue is transmitted over a neural pathway, and eventually coded messages are involved to communicate with the occipital lobe. Why do we make this claim?

There are approximately 260 million photoreceptors on human retina. The initial signal gets transmitted a short distance, but these signals are subsequently distributed among only 2 million ganglion cells. “This compression of information suggests that higher-level visual centres should be efficient processors to recover the details of the visual world.”22

The signals originating from the retina are processed by specialized neurons which perform distributed processing to determine object attributes such as colour, location, and movement. Low-level algorithms are available, able to identify edges and corners.23,24 Somehow the whole needs to be combined into a coherent whole, taking context into account. The underlying language is not yet known, but rules are beginning to be identified, such as the use of AND operators.25

Coded messages could also precede and activate a specific sensor. And sometimes activation of a sensor can be supplemented with other contextual inputs which are subsequently coded into a message. To illustrate, a biochemical can dock onto a receptor (a sensor!) of a cell’s outer membrane, leading to a complex cascade of internal processes, culminating in regulation of several genes. The resulting process is part of a cellular language, the details of which are not fully elucidated.26 In this case, the signal from a sensor contributes input to a coded message.

The following example shows how sensors could be integrated into a CIS. Suppose four departments {A,B,C,D} at a university participate in races which occur hourly. During even-numbered hours men race; on odd-numbered hours the women do.

A scoreboard is divided into eight portions, representing the four departments and the gender. Each time a sensor on one of the eight squares is activated, the value displayed increases by one. This could be implemented in a mechanical, strictly cause-effect manner. The sensors are identical and so are the cues received. So far there is no alphabet of symbols or syntax. Therefore, a coded message was not received at the scoreboard, although something useful did result. Nevertheless, we’ll show that a coded message could precede or follow the work of the sensor-based equipment.

Figure2
Figure 2. Coded messages can precede or follow the use of sensors. The winners from four departments {A,B,C,D} could be communicated by an initial message, e.g. B C C B D A. The Decoder determines from the time (even or odd hours) whether each symbol received represents a man’s or woman’s race. Both facts permit activating one of eight boxes on a scoreboard. The eight sensors can transmit a signal elsewhere, and at the end of the transmission a new codeword unique to each sensor is generated. The new code, e.g. 0111011001… can then be transmitted or stored.

Let us assume the winning department for each hour is communicated by a judge, using a single symbol from the quaternary alphabet {A,B,C,D} which is transmitted towards the scoreboard (figure 2).27 The Decoder is also endowed with an internal clock, thereby permitting the winner’s gender to be identified. Now four departments × two genders, or eight outcomes, can be communicated, to one of the eight portions of the scoreboard,28 although each symbol alone can only provide two bits of data. The winner can be communicated by transmitting an electric signal through the relevant wire on to the correct one out of eight sensors on the scoreboard (figure 2).29

Suppose the winning department and gender are to be communicated to another location afterward. The back end of each of the eight boxes in the scoreboard first transmits a signal (not a message) along a cable. A coded message, unique to each original sensor, is then produced by encoders (the small round dots preceding the triangle in figure 2) using a new binary code, with codewords such as (0010), (1100), or (0111), unique to each wire. The new coded message identifies the same facts as the original quaternary one {A,B,C,D}, supplemented by the winner’s gender, and this message can now be transmitted far away or stored somewhere for future retrieval.

Note how the specific assignment of A, B, C, or D to either of two out of eight boxes was arbitrary and so was the assignment of specific triplet binary codewords to each sensor. The codes are independent of the physical infrastructure, as must always be true of informative codes.

I believe the simple example illustrates a general principle in cellular systems. Methylation at specific location on DNA or phosphorylation of portions of proteins are simple signals which get supplemented with other details and converted into coded messages.

Senders and receivers in CIS theory

The CIS model focuses on empirical measurements. A series of refining processes are observed, at least one of which results from receiving coded messages. Observations 1–3 above are illustrated in figure 3. Notice that after processing the message, additional refinements can occur, represented by the ever smaller contours in figure 3.

Figure3
Figure 3. Coded Information Systems sequentially refine behaviour through a series of processes. At least one process is guided by coded instructions. Each goal-directing refinement step could be influenced through coded messages, sensors, physical hardware, or pre-existing resources such as data or logic-processing algorithms.

The emphasis of the CIS approach is on observing the modified range of behaviour of the target system, unlike Shannon’s theory which analyzes the statistical features of messages. The effects caused by other sequential refinement components, which can precede or follow receipt of the message, are also evaluated based on resulting consequences. This will be elaborated on in part 4 of this series.

Apropos quantifying information, Shannon’s model is unsuitable to evaluate prescriptive instructions. Suppose a robot is to extract trees from a forest. An algorithmic message is sent, indicating how to find the largest tree within 50 metres of the Receiver, step by step. Statistical analysis of the series of 0s and 1s transmitted would be of little value, but the approach of CIS is to measure the resulting outcome empirically. It is the contribution to producing the correct outcome which matters, when compared to the (theoretical) reference state, that defines improvement, measured in bits.

The intention thus far is to introduce a more nuanced manner to discuss and measure information. The range of behaviour, weighted by observed probability, is compared for initial and a refined state, for each contour in figure 2. There can be many ways these improvements can be engineered, using software and hardware. Intelligent intervention, what some call smuggling information into a system, can now easily be taken into account. For example, any artificial guidance to select a genetic or other algorithm to attain a specific outcome is an input which achieves an impovement over the preceding, unguided state.

The precise, regulated designs used in biology and technology can be understood and quantified with this simple CIS approach. The details themselves, like gene expression or metabolic regulation,30 are often exquisitely sophisticated, but are in a sense ‘only’ details which can be understood by drilling down from the high-level concepts of the CIS model. We will defer a description of the many designs found in nature, the purpose of which is to ensure the right outcomes in a CIS.31

CISs are created to organize matter and energy in a precise manner at the correct time and location, a very dynamical challenge which requires sophisticated components. These integrated systems can typically be reused many times. The variety of unsuitable parts, which includes incorrect coded messages, greatly outweighs the functionally acceptable ones.

The motivation behind this analysis is to force researchers to consider everything involved to permit a message-processing system, such as cells, to work. One of Truman’s harshest critiques32 of the Avida setup and claims is that virtually everything necessary for the ‘simulation’ to work, such as physical replication of the electronic organisms, the energy source, physical transfer of data to the appropriate logic processing locations, and so on, were machines already made available. They made decisive contributions to ensure the desired outcomes. In nature all these components are coded for on DNA, and therefore subject to the ravages of random mutations. In Avida, mutations cannot destroy nor disrupt most of the fundamental system components. Virtually everything relevant to information was overlooked in the discussions. Forcing the participants to discuss the complete CIS should have prevented such foolishness.

One final notion in the CIS model is to distinguish between two kinds of receivers: mechanical receivers, which respond deterministically to the message’s instructions; and autonomously intelligent receivers, who first evaluate and decide how to respond. Between these extremes lie a range of intermediate possibilities, including programmed artificial intelligence programs designed to incorporate various forms of reasoning, and systems able to query for additional relevant details from environmental sources.

Part 4 will introduce the fundamental theorems associated with the CIS model, and show that this framework incorporates the insights from Shannon’s theory, Gitt’s model, Dembski’s contributions and other schemes. But consistent use of the CIS notions does lead to some different conclusions than those proposed by other frameworks.

Conclusion

The literature attempting to describe information is very broad. It is generally accepted to be non-material, and many attributes are assigned to it. But it seems that people are generally referring to a system which contains physical components, and not to a single entity. Analyzing components of a coded information system, such as coded messages, signals, and physical hardware separately, solves several conceptual difficulties. And as will be further elaborated on in part 4, the effects produced by a CIS as a whole offer a means to quantify what is accomplished by portions, or the complete CIS.

Posted on homepage: 14 February 2014

References and notes

  1. Truman, R., Information Theory part 1: overview of key ideas, J. Creation 26(3):101–106, 2012. Return to text.
  2. Dretske, F.I., Knowledge and the flow of information, CSLI Publications, Stanford, CA, p. ix, 1999. Return to text.
  3. Truman, R., Information Theory part 2: weaknesses in current conceptual frameworks, J. Creation 26(3):107–114, 2012. Return to text.
  4. “Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.” Shannon, C.E. and Weaver,W., The Mathematical Theory of Communication, University of Illinois Press, IL, p. 31, 1998. Return to text.
  5. Gitt, W., Compton, B. and Fernandez, J., Without Excuse, Creation Book Publishers, Atlanta, GA, 2011. Return to text.
  6. Gitt, W., Scientific laws of information and their implications part 1, J. Creation 23(2):96–102, 2009. Return to text.
  7. Gitt, W., Implications of the scientific laws of information part 2, J. Creation 23(2):103–109, 2009. Return to text.
  8. Schneider, T.D., Evolution of biological information, Nucleic Acids Res. 28(14): 2794–2799, 2000; www.ccrnp.ncifcrf.gov/~toms/paper/ev/ev.pdf. Return to text.
  9. Montañez, G., Ewert, W., Dembski, W.A. and Marks II, R.J., A vivisection of the ev computer organism: Identifying sources of active information, BIO-Complexity 3:1–6, 2010, doi:10.5048/BIO-C.2010.3; evoinfo.org/papers/vivisection_of_ev.pdf. The authors show how the ev search algorithm can be viewed as an inversion of a perceptron, and the algorithm provides immediate feedback every step like the child’s game ‘You are getting colder or warmer’ to guide to a target, even if the specific target is not fixed in advance. Return to text.
  10. Dembski, W.A., No Free Lunch, Rowman & Littlefield, Lanham, MD, 2002. Return to text.
  11. Dembski, W.A. and Marks II, R.J., Life’s Conservation Law: Why Darwinian Evolution Cannot Create Biological Information; in: Gordon, B. and William Dembski, W. (Eds), The Nature of Nature, ISI Books, Wilmington, DEL, 2009; evoinfo.org/papers/ConsInfo_NoN.pdf. Return to text.
  12. Dembski, W.A. and Marks, R.J. II, Bernoulli’s principle of Insufficient Reason and Conservation of Information in computer search, Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, pp. 2647–2652, October 2009; evoinfo.org/papers/2009_BernoullisPrinciple.pdf. Return to text.
  13. Winston, E., Dembski, W.A. and Marks, R.J. II, Evolutionary synthesis of Nand Logic: dissecting a digital organism, Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, pp. 3047–3053, October 2009; evoinfo.org/papers/2009_EvolutionarySynthesis.pdf. Return to text.
  14. Dembski, W.A. and Marks, R.J. II, The search for a search: measuring the information cost of higher level search, J. Advanced Computational Intelligence and Intelligent Informatics 14(5):475–486, 2010; evoinfo.org/papers/2010_TheSearchForASearch.pdf. Return to text.
  15. Dembski, W.A. and Marks, R.J. II, Conservation of information in search: measuring the cost of success, IEEE Transactions on Systems, Man and Cybernetics A, Systems & Humans 5(5):1051–1061, September 2009; evoinfo.org/papers/2009_ConservationOfInformationInSearch.pdf. Return to text.
  16. Gitt, W., Scientific laws of information and their implications part 1, J. Creation 23(2):96–102, 2009. Return to text.
  17. I say usually since I argued in part 2, ref. 3, that a huge amount of logic and computing occurs within the Mind, independent of the physical brain which acts as a biological database of multimedia data and abstract concepts. Return to text.
  18. Togneri, R. and deSilva, C.J.S., Fundamentals of Information Theory and Coding Design, Chapman & Hall/CRC, Boca Raton, FL, p. 106, 2003. Return to text.
  19. Hearing the crack of a starter’s gun to initiate a race is an example of a sensor and not a coded message. Return to text.
  20. Gazzaniga, M.S., Ivry, R.B. and Mangun, G.R., Cognitive Neuroscience, W.W. Norton, New York, 3rd ed., 2009. See pp. 168–170. Return to text.
  21. Gazzaniga et al., ref. 20, pp. 180–185. Return to text.
  22. Gazzaniga et al., ref. 20, p. 178. Return to text.
  23. Gazzaniga et al., ref. 20, pp. 177–199. Return to text.
  24. Pinker, S., How the Mind Works, W.W. Norton, New York, 1997. Return to text.
  25. Gazzaniga et al., ref. 20, pp. 169–170. Return to text.
  26. Several researchers find a clear analogy between regulatory and computational logic observed in cells and computer programs; e.g. Davidson, E.H., The Regulatory Genome, Elsevier, San Diego, CA, 2007; Shapiro, J.A., Evolution: A View from the 21st Century, FT Press Science, Saddle River, NJ, 2011. Return to text.
  27. The assignment of a letter to each team is arbitrary and unrelated to the transmission medium. This is a very simple, but legitimate code. Return to text.
  28. In part 2, ref. 3, we pointed out that for a four-symbol alphabet each symbol can transmit 2 bits of data. The fact that 22 = 4 possible outcomes are less than the eight which need to be specified is solved by using an additional informative source, the internal clock. Return to text.
  29. Each signal sent is identical, but which of the eight possible results you will get is determined by the choice of wire it is transmitted along. We begin to see an important principle: intelligent choices and designed engineered components can contribute to ensuring the right outcome. A common mistake addressed in parts 1 and 2 of this series is the claim that information arises easily, where the quantifiable contribution of intelligent interference was overlooked. Return to text.
  30. Fell, F., Understanding the Control of Metabolism, Portland Press, London, 1997. Return to text.
  31. For example, chains of linked enzymes produce useful bio-chemicals in cells. There are many feed-back inhibition designs in which an intermediate or final product interferes with the activity of an enzyme earlier in the chain, to unsure the correct concentration range of product. Sometimes a resulting protein can repress the expression of a gene necessary to produce an enzyme used to create that protein. The guiding instructions to perform such regulations are not directly guided by messages encoded on DNA. See almost any text on biochemistry or cell biology for examples. Return to text.
  32. Truman, R., Evaluation of neo-Darwinian Theory using the Avida Platform. Part 1, Progress in Complexity Information and Design 3.1, November, 2004; Truman, R., Evaluation of neo-Darwinian Theory using the Avida Platform. Part 2, Progress in Complexity Information and Design 3.1, November, 2004; www.iscid.org/pcid/2004/3/1/truman_avida_evaluation.php. Return to text.