Question Everything

The Science of Systems 11

Systems Science 11 — Bio-Systems Examples

What is Life?

Erwin Schrödinger asked that question in 1944, one year before my birth. Of course he was by no means the first person to ask it in scientific query. But he was the first to link living processes to the ideas of thermochemistry and our modern notions of organization and order. He called life negentropy or the negative of entropy, implying that living systems had the capacity to self organize and maintain states far from equilibrium in spite of the Second Law of Thermodynamics. What some scholars had, in previous times, called the 'vital force' Schrödinger correctly identified the basic principles that underlie the motivation of living systems, the flow of energy through the system.

Inspired by Schrödinger, years later (1968), Harold J. Morowitz (sadly no Wikipedia entry to my dismay) extended this thinking in what I personally consider a seminal contribution at least as important as that of Ilya Prigogene's with respect to chaos theory. Whereas Prigogine posited some emergence of order out of an otherwise chaotic process, Morowitz provided a precise mechanism for how this might happen. Far from deterministic, in the ordinary sense of that word, Morowitz provided the low level details of how a stochastic process could be driven far from equilibrium and provided a deeper understanding of the means by which Prigogene's dissipative systems could produce higher levels of organization and maintain a 'vital' process. It was not so mysterious after all. In the hands of Morowitz we see how the flow of energy through a well defined unorganized but bounded system will drive that system toward higher internal organization without it heating up to a point where whatever structure obtains is obliterated (via dissipation). This seminal work is Energy Flow in Biology, and in my personal opinion it ought to be required reading for every biologist who claims to understand evolution and organization (see this Amazon.com page for a list of Morowitz's other books).

Living organisms, from single celled to human beings, are nearly incomparable in terms of complex, organized systems that appear to be tractable in terms of what goes on within the observable boundary of that system. They are not closed to either energy or matter. Yet they are sufficiently like Mech-Systems that we are able to study the internal workings and hope to have some decent ideas about how they work. Nor are they closed to information flows and that too makes them more interesting than Mech-Systems, but not as ultra-complicated as an Eco-System. Indeed, individual organisms along with their populations comprise the living aspect of an Eco-System. But unlike the latter, a Bio-System has evolved with elaborate control structures, a hierarchical control system, that provides for stability internally that is not always present in an Eco-System.

In this installment, I want to explore, through some examples, the principles of organization and dynamism for Bio-Systems, showing where they inherit some characteristics of Mech-Systems (origin of life) and where they behave, at times, like Eco-Systems (neural Darwinism and learning). But mostly I hope to show how, via the mechanisms of hierarchical control, an organism achieves a relatively long life of stability after development and manages to survive and procreate as a result of the control of energy flow through the body (cell).

Where Did Life Come From?

One of the greatest questions of all time. At one point early in Earth's history, there were no living things. Then, about 3.8 billion years ago, primitive organisms emerged, swimming in ancient oceans and kicking off the Darwinian process of variation in form followed by selection for fitness. Today the origin of life (abiogenesis) is still shrouded in mystery, not because we haven't got an idea of how it might have come about so much as the fact that we have a number of competing ideas and are in the process of testing them.

There are several clear, key elements that must be present for a system to be alive. Typically life is defined by this list of characteristics (see Life). But Schrödinger, Morowitz, Prigogine and many more have wanted to define life as a unified process. One of the most ambitious attempts to describe this process in a concise, coherent way was developed by Chilean biologists Humberto Maturana and Francisco Varela. They introduced the concept they called autopoiesis in 1973. More on this later.

The key to understanding the origin of life is to describe how a prebiotic chemical mixture could give rise to something like an autopoietic system, in essence a cell. Morowitz has suggested that there are clues hidden in modern metabolism, sort of like fossil chemical reactions that have been conserved from the earliest times because they are so good at producing a stable but once primitive metabolism.

The first requirement for some aggregate of chemicals to be alive is that the system be bounded. Fatty membranes seem to form relatively easily when the right molecules are available in an aqueous solution. These membranes can act to contain larger, less portable molecules while being permeable to smaller, component molecules. The second requirement is for an appropriate energy gradient to exist wherein the system can capture high potential energy from the environment and use it internally to do the work of creating (and breaking) chemical bonds in the process of maintaining the dissipative structure, the structure that then emits low potential heat and waste products. One of the reasons I favor a metabolism-first model (see this section of the abiogenesis article) is that appropriately coupled energy flow mechanisms internal to the system have to have been in place in order for RNA synthesis, as well as other complex molecules, could be achieved. Someday I will take a closer look at this, but my hunch, for what it is worth, is that the origin of adenosine triphosphate came very early (or at least adenosine diphosphate). ATP is an energy currency (carrier) for cellular metabolism. The third phosphate bond is very high in energy and can easily be broken in reactions, releasing that energy to drive syntheses of various kinds. Moreover, adenosine is one of the four code molecules in RNA and DNA (the others being guanine, cytosine, thymadine, and uricil in RNA in place of thymadine). This looks suspicious to me! And, if it weren't enough that adenosine is a key player in energy flow and information coding in genes, it is the end molecule in the CCA terminal of transfer RNA the molecule responsible for bringing amino acid molecules to the ribosome where protein synthesis takes place. Adenosine seems to show up in some very interesting places in the metabolism of cells (see also cyclic adenosine monophosphate for yet another intriguing role for the adenosine molecule!)

A second requirement is that the aggregate of chemicals have achieved some form of collective autocatalysis, especially a web with internal autocatalytic cycles that forms the basis of autopoiesis. Again, this suggests to me that a primitive metabolism, bounded by some form of membrane, emerged early from the pre-biotic soup and gave us the origin of life. Once these pre-bacterial forms found ways to encode their own structure (perhaps in RNA first) and maintain that code in primitive chromosomes life was on its way in terms of Darwinian evolution. Yet another step on the way to eukaryotic cells is suggested by Lynn Margulis' theory of endosymbiosis, wherein early bacteria having various 'talents', like photosynthesis, formed symbiotic relations with other bacteria that had, essentially complimentary talents, such as digestion of large molecules, to form cells with organelles.

The main point, however, is that energy flows into the system are captured (by photosynthesis and respiration in modern plants and animals), disbursed to working areas (metabolism) and the system is able to move things around, maintain and repair itself, and, well, live. Some time after primitive RNAs that actually coded for functional components, proteins that worked as catalysts (enzymes), and in turn, for structural components such as organelles and membranes, Darwinian evolution got a purchase. The rest is biological history.

What stands out in the story of the origin of life is the way in which the right combinations of Mech-Systems, interacting among themselves, can produce a system capable of interacting with other similar systems. That combination, capable of a degree of adaptability (later) transitions from non-life to life, an emergent property of all such systems. We'll take a peek under the hood of a living system to see how these play out.

Homeostasis and Autopoiesis

Homeostatic processes are at the core of living systems. You may recall from Systems Science 7 — Cybernetics: The Science of Control that homeostasis is a first order cybernetic system, feedback control. Homeostasis is generally a little more complex than I described it there. In metabolism there are critical factors, like pH (acidity), calcium concentrations in the protoplasm and so on. In multicellular organisms like us, there are larger scale factors that require managing within critical ranges. For example body temperature for us hot blooded mammals can't be allowed to get too low, nor can it get too high. There is an allowable range of temperatures (or any of the factors) that the body (cell) can tolerate and adjust to. But then there is a range outside of that usually narrow range where action must be taken to counteract whatever is causing the deviation from normal, what I will later call a deformation. Then there is a slightly wider range of values in which damage can be caused, the deformation reaches a breaking point. The homeostatic mechanism is the cybernetic controller that works to keep the values within the safe range. Its job is to prevent damage and if the system gets too far outside the safe zone, the mechanism must be able to muster more resources to wrestle the system back into alignment. For example in the case of body heat, if you find yourself exposed to cold temperatures and your core temperature is starting to go down (even a little bit) your muscles are commanded to shiver to produce heat. Your behavioral system also takes note and you move around attempting to get out of the cold. The shivering is an automatic operational control. The getting out of the cold is a higher level tactical control.

The homeostat (short for homeostatic mechanism, the controller in Fig. 1 of SS-7) has to be constructed (from genetic information) and maintained. So other systems are responsible for keeping the homeostat charged and ready for action. In fact there is a whole complex web of mechanisms that are there to keep the system, all of its various homeostats, operating within their proper ranges. Success is health. Failure is disease and possibly death. This larger complex of mechanism includes subsystems whose job it is to repair damage done to the system as a result of a homeostat's inability to maintain proper values. These subsystems are held in reserve, so to speak, against the need to do repair work, but obviously the system would prefer (if it had a brain!) to not have to activate them.

This larger network is the self-maintaining autopoietic system described in the Wikipedia article. The autopoietic network surrounds the homeostats and acts generally as the logistical control for all the lower level operational subsystems (and their homeostats). Actually it is more complicated than that might sound since every subsystem has its own control parameters, even the autopoietic components, so in a sense every subsystem is supporting every other subsystem, even if indirectly. Nobody ever said understanding the details of a living system would be easy!

Fig. 1 shows a highly simplified diagram of a system with autopoietic maintenance surrounding a homeostatic core process.

Figure 1. A homeostatic core is surrounded by multiple mechanisms for maintaining and repairing structures. Each such component in a complex network has its own homeostatic control.

The "maintenance processor", actually many such processors might be involved, actually has an internal structure similar to the example homeostatic core process. However, in its case the environment IS the pictured homeostatic core process. A distortion or disturbance in the pictured homeostatic core process acts as the stimulus to the maintenance processor and its homeostat kicks in to cause the maintenance work to commence (shown as the black & white arrow pointing toward the homeostatic core) as its response. Inside a living cell and in your body there are literally thousands of low-level mechanisms interoperating to maintain the system in its function within its optimal ranges of factors.

Under nominal environmental circumstances (the right environment for the particular organism) every thing works in concert and the organism is "happy". But what happens if one or more environmental factor gets out of a comfortable range? This is going to trigger the appropriate control mechanism to counteract the disturbance. The autopoietic system will channel resources to meet the emergency and with luck and good behavior the system may be able to reestablish ideal conditions. However, should it fail to counter the effect before damage is done, then other mechanisms will be brought on-line to repair the damage done. If the damage overwhelms the system's capacity to repair, then, of course death follows. Interestingly, as we will see, in the non-death scenario this is the basis of adaptivity. Here is the argument.

The autopoietic system maintains an energy budget for directing internal energy flows to where they are needed to do work. This is part of its logistic control function. The budget is flexible in that flows can be adjusted on demand and within reasonable ranges, much like the ranges of deforming stimuli we discussed above. So, for example, if there is an increase in demand to move molecules within the cytoplasm, more ATP can be routed to the actin-based fibers responsible for the movement. Thus they can respond to the demand. This of course would be at the short-term expense of some other subsystems that might get less ATP as a result. More importantly, the autopoietic system can keep track of these shifting demands over time and make longer term adjustments based on longer term demand.

A good example of this adjustment based on longer term demand is your muscles. Suppose after a lifetime of being a couch potato you decide one day to get into shape by working out, maybe lifting weights. Initially your muscles are not competent to handle the weight lifting without some pain (and some damage), even if you start easy and work your way up. If you only tried the weight lifting once and then quit, your body would not do anything except complain with pain or stiffness the next day. It would have certainly routed energy into the used muscles to accomplish the goal at the time. But then it would go back to the 'regular' energy budget and that would be all there was to that.

But if you work out regularly, your muscles send signals back to your growth control centers expressing an on-going need for additional resources, both real time energy input and reinforcements in the form of more muscle cells. Your muscles actually retain a kind of memory of the effort you have been expecting over a longer period of time and training. So the body responds by coordinating several processes. Your diet may start to include more protein, for example. Your growth hormones for muscle bulk may signal muscle cells to replicate to bulk up your muscleclature. You basically build a physique that is capable of responding to the long term demands that have been put on it. You adapt structurally within the limits possible for doing so. The autopoietic system has responded to the longer term demand by marshalling resources to build more prepared structure. This so that the system is better able to respond in a faster time and avoid damage. Of course a new balance in the energy budget is needed; more energy will be routed to maintain the new, larger subsystem. But the tradeoff between energy needed to maintain the subsystem and the energy needed to repair damage, and even to respond to the demand is quite favorable.

Here is a paper I did some years ago, "Foraging Search: Prototypical Intelligence", which lays out the case for why learning evolved in the brain based on these energy budget arguments. If you look at the graphs you will see an interesting aspect of adaptivity (in the case of neurons, adaptivity is the change in synaptic efficacy that leads to changes in behavior). The first graph shows a non-adaptive stimulus and response along with accumulations of energy costs resulting from three different components. Costs, in this sense are based on the notion of opportunity costs. The energy is used to take care of the response, but it might have been used elsewhere had the stimulus not occurred. The first component is the energy cost to respond to the stimulus. Without going too deeply into the biochemical aspects of this, it is the case that the more prepared a response subsystem is, the less total energy it will require to respond to the stimulus! This is because of the lag time between the onset of the stimulus and the time at which the response becomes strong enough to begin counteracting the stimulus. A more prepared (stronger) response subsystem can react sooner and with greater force, diminishing the time lag and the total amount of work that needs to be don to respond. Thus, a minimally prepared subsystem will end up expending more energy for a given stimulus-response. The basic idea in structural adaptation is to minimize this cost.

The second component of energy costs is for repair of damage. In principle, if a subsystem is better prepared, not only will it minimize response costs, but also quell the stimulus sooner and avoid or minimize damage repair expenses. Finally there is the energy cost of maintaining the structure at an elevated level of preparedness. Bigger muscles require more food, even when you are not working them hard. However, this cost is only incrementally higher than at, say, a baseline level of preparedness. So while it is higher over time, it is marginally higher compared with the costs of response and damage repair.

A key idea here is that it is the long-term frequency of stimulus events that determines the degree to which the response subsystem needs to build up. If you were to lift weights only once per week, or stopped after working out every day for one week, your muscles would not bulk up to the Schwarzenegger level or even look that different from your couch potato days. But if you worked out, say, three times a week, for months on end, you would expect to see some more 'definition' in your muscleclature and muscle tone. If you worked out as much as Arnold did in his youth, maybe you too would look like the Governator (or more like the Terminator; age has its costs to pay!) The more frequently a stimulus occurs, and the longer the time frame over which it maintains that frequency determines how much adaptation is required to be 'ready' and 'able' to respond.

Anticipatory Systems

As I show in this paper, the benefits of a memory system based on building adaptive (bigger, more powerful) structures, is a system that can respond to the first hint of a stimulus and by earlier quelling of the stimulus, reduce costs considerably. But that isn't all that can be accomplished, at least in neural tissue.

Neurons, at their synapses, have evolved long-term associative memory networks (for more details see my paper, "Toward a Theory of Learning and Representing Causal Inferences in Neural Networks". Associative memory traces allow a neuron to create, essentially, a causal linkage between a non-deforming stimulus, such as the sight of a predator, and a deforming stimulus, such as pain from a previous, non-lethal encounter with that type of predator, so as to anticipate the outcome of letting the deforming stimulus occur. The third graph in the first paper shows that having such an anticipatory stimulus causes the system to preemptively respond such that repair costs are not incurred and response costs themselves, which are proportional to the stimulus strength, are very much more minimized. Thus, the system's total costs have been reduced at not much greater cost of maintaining the larger response subsystem and the associated anticipatory response subsystem.

While this is most visible in neurons, brains, and behavior control, the same general principle applies to all adaptivity in Bio-Systems. At the end of the Foraging paper I present the case for why Bio-Systems (animals in this case) need to evolve anticipatory adaptive response. The case hinges on the argument that the real world is non-stationary. That means that things are always changing. By this I don't just mean things are moving, or that the environment is subject to cyclical behavior (like the seasons). Rather this refers to the fact that totally new conditions can come into effect in an environment (when I get to Eco-Systems I will develop this concept much more thoroughly). Completely new stimuli may be encountered. But more often, new causal relations between old stimuli may arise. A Bio-System has to be able to adapt to these new arrangements and causal relations. It does this by always being able to flexibly adjust the energy budget and 'learn' the new associations. It must forever be able to readjust as needed over varying time frames.

The Limits of Adaptivity, Growth, and Development

One of the main things that differentiates Bio-Systems from Eco-Systems is that the former have internally imposed limits on their capacity to modify themselves in light of radically new stimuli. They are bounded by internal controls that limit their adaptivity as well as their capacity to grow or develop. While our genes provide us with considerable flexibility in terms of how variant we are in our development, and how much we can learn, they still put constraints on us in all of these ways. Of course, even Eco-Systems have limits when considered as a stable system. An old growth, climax forest can be utterly altered in hours from a bad fire. However, Eco-Systems are not closed to the invasion of other species and life on Earth has withstood numerous massive die-offs yet managed to adapt and life continued to evolve. You can't say the same for an individual, a Bio-System. [Note that individuals are very often colonized by all manner of parasites, forming a kind of Eco-System! But to the individual so colonized, the effects of the activities of these foreigners is part of the environmental forces acting on it.]

There are obviously good reasons why Bio-Systems are self-limiting. First there is no reason that a Bio-System should have to adapt radically and constantly, taking on virtually any form needed to survive. The environment doesn't change that fast. Individuals are part of a population and whatever changes do occur don't necessarily impact all individuals. Further, because of the way genetics works, there is generally a reasonable amount of variability of traits and behaviors within a population such that if some individuals are affected by an environmental change, not all will necessarily be negatively affected. That is natural selection at work. The one rule you can count on in Bio-Systems is that no such system will expend energy on some capability that isn't absolutely necessary for survival.

Energy arguments can be brought to bear on growth limits as well. Bio-Systems, with a few exceptions that are reasonably understood from an evolutionary point of view, are programmed to grow to a needed size relative to their eco-niche and then stop. The same does not apply to populations of Bio-Systems, however. Populations sizes are controlled, as a rule, by Eco-System internal dynamics that I will cover in that installment. But at the individual level, growth is carefully regulated by the logistical control system so as to keep any single individual within a population from dominating completely and possibly wiping out other members. Populations need to maintain variability and so domination by one fast growing individual (or even a few) would be counter to the long-term health of the species. Hence, individuals are restrained and kept within an admittedly variant range, but with a relatively small variance.

Finally, it would be completely self-defeating for an individual to undergo form and behavioral modifications that effectively made it unrecognizable to other members of its species. In the end, given that the only means of continuation of Bio-Systems' patterns is through reproduction, and that requires successful mating, con specifics have to be recognizable to one another. Neighboring Eco-Systems hardly care at all about what the others look like, or what their composition might be. But individuals need to mate with similar individuals in order for the species to continue. This might seem like a silly thing to be concerned about. But back in the beginning of life, it is very likely that the first proto-cells and very early cells that invented sex or sex-like reproductive strategies had more capacity to take on new forms since they had not yet evolved internal control systems to ensure species continuity. Of course, those that changed radically enough to not be successful in reproduction soon disappeared from the scene leaving those that could maintain species-specific forms and behaviors. Evolution is the great leveler.

Why Reductionist Methods Work on Bio-Systems (and are hard in Eco-Systems)

I'd like to end this installment with an observation about some methodological aspects of studying Bio-Systems, especially versus Eco-Systems. The relevance to the latter may need to wait until I've finished that installment to be fully appreciated. But I'll at least introduce the ideas here.

The fact is that reductionist science has been wildly successful at producing our knowledge about Bio-Systems and natural Mech-Systems (although engineering of man-made Mech-Systems may involve some reductionist thinking). The reason is that these systems tend to have more definable boundaries and boundary transfers with functional attributes that are relatively easy to measure. Put bluntly, you can dissect a representative system and find its parts and their relations to other parts. It may take advanced technology (e.g. electron microscopes) to get there, but, in principle, all functions are measurable.

This is a pretty strong statement, to be sure. But I think if you explore the history of biology you will see repeated examples of functional decomposition illuminating the inner workings of system after system. The use of model building as a way to study whole systems dynamics, has been less useful in understanding Bio-Systems as it has in Eco-Systems precisely because access to the system internals is so readily available given that you have the dissection tools to do it. And the history of biological science is that when you don't have the tools, you either put off the questions or look about to create the tools. The invention and development of imaging technologies for brains (and other working tissues inside live organisms) is a case in point.

It isn't uncommon to hear a critic of reductionist methods complain that reductionism fails to explain the whole organism. The mantra one most often hears is something like "The whole is greater than the sum of its parts." It all depends on what you mean by the word 'sum'. If you mean that if you just throw a bunch of parts together they will fail to produce a living system, then DUH! Bio-Systems and natural Mech-Systems are the sum of their parts AND the functional relations between the parts. And reductionist methods have done a splendid job of discovering and explaining all of that. [BTW: if you read the article, above linked, be sure to note the difference between reductionism in science and the philosophical notion that reductionism means 'reducing' explanations of complex systems to simpler principles (rather than parts and relations), e.g. reducing biology to "mere" chemistry. Of course, as we saw in the origin of life above, you have to be able to explain the biological phenomena BASED on chemistry, but this isn't the same as reducing a complex system to just its chemistry. I'll probably write more about that issue at some point in the future since it is such a common argument leveled at science, motivated, I think, by a desire to knock science off its high horse as the best method to obtain knowledge. As you might guess, I have very little patience for those arguments!]

With Eco-Systems the situation becomes incredibly much more difficult (and remember, Eco-Systems include societies and economies, etc.). The very fact that Eco-Systems are highly variant and non-stationary in nature makes attempts to understand relations between components by taking such a system apart problematic. Relations may be continually changing over time in non-stationary ways. Sampling the components itself is messy and hard work. The science of ecology is still grappling with reductionist methodology and probably always will. Where the Eco-System includes humans sampling becomes even more problematic for ethical reasons (of course this can apply to Bio-Systems studies that involve testing on animals). Sometimes it is the case that attempts to dissect parts of an Eco-System do destroy the sought relations and that fact may go undetected by the scientist. Fortunately, another methodology involving the construction of models (mostly computer-based) can be brought to bear on Eco-Systems. This is generally regarded as an integrative or whole-systems approach, essentially the reverse of reductionist approaches. But it is also true that models need to start with some basic understanding derived from field studies. The model must try to capture what is known about the system from reductionist studies, but at some point it can become a powerful tool, if used correctly, in positing hidden relations and suggesting places to look for previously hidden relations. I will likely do an installment devoted to computer modeling in the future to explain this tool.

While I said that modeling has been less used in Bio-Systems studies, that is starting to change. Today very powerful computer models are used to study genetics, metabolism, and neural networks just to name a few. These models are used in a manner similar to how they are used in studying Eco-Systems. That is they can be used to develop hypotheses that can then be tested in the reductionist sense. Thus there is another region of overlap between Eco-Systems and Bio-Systems. Next we'll take a look at this from the other side, that of Eco-Systems.

A paradox

A paradox, a paradox, a most intriguing paradox

      The Pirates of Penzance, Gilbert and Sullivan

---

Straight forward logic would seem to dictate that if you improve the efficiency of a given work process within a system then the whole system should enjoy the greater efficiency. Right?

Or we might reasonably assume that developing new ways to produce energy would on the whole increase the availability of energy.

These are slightly different ways of saying that a local optimal solution to a problem should contribute to the overall optimality of the global system. It turns out that this is mostly magical thinking.

In my last blog I said something to this effect:

As in all network of processes relationships, a local optimum does not imply a global optimum. What looks good at one node in the network, the choice to go to that other node instead of any other choices just because it looks like the best outcome can have disastrous impacts if all of the subsequent choices lead to extremely high costs.

I finally got around to reading Leonardo Maugeri's October, 2009 article in Scientific American called "Squeezing more oil from the ground" and I couldn't help but realize that this kind of thinking dominates the technological cornucopian perspective. In the article Mr. Maugeri promises us that there are much greater reserves of oil left in the ground than had previously been believed to be recoverable. And, that "new technology" will make that oil recoverable (real soon now). Hence, all this fuss about peak oil is something we can ignore for at least another couple of decades. While he does point out that these technologies are costly, what he never gets around to pointing out is what the actual total cost, in energy units, are realized in developing and deploying them. He leaves the reader to assume that because these technologies provide greater extraction efficiency (as defined by the recovered volume of oil) that we are all better off as a result. What he doesn't ask is: What is the net energy gain after all energy costs are taken into account. This wasn't an oversight or a belief that the average SA reader wouldn't be able to understand the nuances of energy return on energy invested. He simply doesn't think like a systems scientist.

It is extremely typical to apply the greedy method of thinking to what you believe is profit making decisions. You are at a node in a decision network. You can't see anything except the next possible moves you could make (decision taking) and have an estimate of the costs of each move. So you choose the one with the least cost. But suppose this is just one step in a complex web of possible (and possibly interconnected) steps along a path to a specific destination or specific result. If there is a longish chain of steps between you and your destination, and all you know is the cost of the available steps from where you currently sit, you could be tempted to apply the greedy method at each node in the network. But if you do, you are in for a nasty surprise.

This figure is a simple representation of a decision network. You are on the node with the 'X' and you have to figure out which way to go next; what the next node you should go to would be. All you have in the way of information about this network is the cost of each outward bound step. In the figure these are 5, 10, 50 and 20. What you don't know is what the costs out of the node that you end up choosing. Once you choose, you cannot go back. Hell, you don't even necessarily know if your choice will lead you to your desired end point (notice the upper left node?)

Notice that there is an actual cost associated with every link in all of the possible paths from where you are to where you want to go. But when you are standing at the 'X' node you can only see the costs of going to the next step. You have no god's eye view of this 'problem'. Unfortunately this is how most of our real world problems actually work. We have limited knowledge to work with, and have to make decisions with limited information. What do we do?

Clearly in this example picking the path with a cost of 5 is a disastrous choice. Picking the one with a cost of 10 is no better. The path starting with a cost of 20 looks undesirable from the location at 'X' but as can be seen from our special god's eye view, it turns out to be the best choice, cost wise.

A lot of energy efficiency problems have a similar structure. I mentioned my experience in the 80s in solar energy where the energy required to make the flat plate collectors could have easily been more than the energy gain for the local householder. The latter enjoyed a lower heating bill but at the expense of the rest of us in terms of total available energy. The problem there was the cost-price distortions imposed by governmental and market convolutions that decoupled energy costs from dollar costs. The latter did not accurately reflect the former, otherwise, no one would have bought flatplate collectors to heat their houses.

Just to give an example of what information distortions can do to decision making take a look at this next figure.

The bolder lines and numbers represent an information feedback chain that accumulates perceived costs according to the number of steps in the path. Think of these as 'prices'. Note that they accumulate at each step and that they are artificially maintained at 10 added per step. Suppose you now had the information shown (only two examples) in bold numbers at the 'X' node. Which path would you choose? Once again you would be fooled into taking the wrong path even though it is cheaper in terms of feedback numbers, i.e. price. As far as you can discern, that price is really less than the one for 60 and since this represents prices accumulated all the way back to the destination, you are perfectly rational to assume this is the best, cheapest path. But the underlying costs are what really count. Believe it or not, there is a lot of this dollar price distortion versus energy costs so that our local buying decisions do not fairly reflect the true costs being borne.

In the case of increasing, or at least maintaining, oil flows by adopting new technologies we have a similar problem. Secondary extraction techniques, for example, involve pumping CO₂ or CH₄ under high pressure down into the wells to increase flows. Something similar is done by pumping water (sea water in Saudi Arabia) into the well. Oil floats on water so this boosts production. What the oil engineers report as the costs of doing this is the dollar amounts used to pay for the gasses and electricity to pump. This works out OK as long as the gasses don't have to be shipped long distances. None of the oil pumping operations puts a price on externalities, like toxic spills. In fact much more energy (hidden in the assets section of the balance sheet!) was used up just to set up the infrastructure needed to perform these operations. It would take a small army of accountants backtracking through the capital equipment assets to derive what energy was actually consumed, but I assure you it would amount to a great deal more than shows up in the reported costs of operations of secondary extraction.

Also consider the energy costs of exploration. How much of that shows up in the real costs of keeping the oil flowing? Or the drilling in deep water or frozen tundra? They say it is expensive, sure, but we keep the same number of barrels of oil flowing. We never get an accounting of the net energy gained.

Another case in point is that of the use of cadmium telluride in a 'newer' technology for photovoltaics. What the enthusiastic purveyors of this technology will gush about is the increased efficiency at collection and conversion, at, now get this, lower prices than conventional polycrystaline silicon cells. What is left out of the equation is the fact that tellurium is rare and will eventually prove to be extremely energy costly to produce, or that cadmium is an extreme poison that needs very careful processes for recycling to prevent toxic accidents. All of this takes a lot of energy but is not currently reflected in the traditional economic sense in the prices.

A large part of the seeming cost savings come from the process efficiencies of manufacturing the cells (supposedly). But those efficiencies were gained by developing newer plants and equipment. All very good on the surface, but what lies beneath. How much additional energy went into the capital goods so that we could get a cheaper production process?

Supposedly, over a very long time scale, when all of the capital has been properly amortized and the competitive pressures on pricing have been stabilized, eventually the long-term price of a product like this should reflect all but the true externalities (the costs of environmental damage resulting from operations), that is the costs the companies can avoid paying [There is a move afoot in accounting to report what is called in financial statements. The purpose of this is to account for potential liabilities companies accrue and might end up having to pay for, like fines to the EPA. Well, that will be the day!]

Unfortunately it isn't just energy problems that are of this local optimum, global sub-optimum ilk. They seem to come up over and over again when someone attempts to solve a local problem but ends up having global consequences. Food aid might fit in this category according to some critics. For example consider the case where grain surpluses are given to African villages who come to rely on the donations (especially if development efforts are hampered by poor soils or lack of water for irrigation). Then one day along comes corn ethanol, profit motive and all, and the aid gets cut because we Americans want to keep gas prices low.

Let's face it. We want to eat our cake and have it at the same time. We will almost always opt for the greedy choice when we have to decide from a local (non-systems science) perspective. And as far as economics is concerned, that is always.

We will try to build high-speed rail systems because that will get us to where we want to go faster (time efficiency). But it never occurs to us that moving mass at a higher speed uses exponentially more energy. We would do well to just fix and improve our existing rail system. But from the non-systems perspective, that doesn't solve our immediate problem. So...

This is one of the (many) reasons I have little hope for Homo sapiens as currently configured. Systems thinking is long-term and strategic. It isn't the greedy solution to a logistics problem. We're good at the latter and piss poor at the former. I still think its a matter of evolution of eusapience.

The Science of Systems 10

Systems Science 10 — Mech-Systems Examples

The Boundary Problem

Before we get started on some real world examples there is a small matter to warn you about. In the third installment, 3. Organization and Its Conceptualization, I noted that systems have boundaries, even if they are hard to distinguish. For the purposes of that subject I left it at that. But as we start to study real world systems we will find very quickly that there is a problem associated with the identification of a boundary that arises not from the notion of "systemness", but rather from the very act of studying or observing a system from the perspective of a conscious, thinking being. The problem arises because systems are always subsystems of some higher-order system, hence embedded within an environment containing other systems. Indeed, some of these other systems might have some close association with the system under study in such a way that they cannot be easily distinguished from it. Hence, we might have some difficulty isolating 'our' system for study. A lot of work is involved in resolving this issue.

It gets even more complicated when a system is highly interactive with many of its environmental components. If the system's boundary is indistinct it may not be possible to say definitively what should be included in the system and what should not. We run the risk of choosing a boundary that might include some components that appear to be members of our system of interest but are not actually essential functional members. They could have just as easily been left out, the boundary drawn between the components of our system and the other one correctly. But how do we know? The boundaries we "choose" consciously are just that — choices, not necessarily physical dividers. Or at least it seems that way.

The real problem is that the whole idea of systemness starts to look a bit arbitrary when we run into these kinds of situations and, in the end, choose a boundary based on our preferences or biases. If systemhood is an arbitrary category, decided one way by one investigator, and another way by a different investigator, our whole basis of systems science could start to look useless! Indeed the main criticism of systems science is that too often boundaries are chosen arbitrarily, thus negating the whole validity of the systems concept.

Our intuitions, not always a great guide in science unfortunately, tell us that there is a reality to the notion of a system, and fortunately, in this case, they serve us well enough. Aside from the many clear examples of systems with boundaries that are easily discernable and provide a unique individuality to a system under study, there are methods of analysis that allow us to resolve the boundary problem in those cases where we are having difficulty. As we explore various kinds of systems in these next few installments, we will come across examples of ill-defined boundaries that will give us pause. But we will also see that some of these methods can be used to help out. In one of the last installments I will actually focus on some formal methods that are used in systems analysis and provide a treatment of how this boundary problem can be solved there. For now, when we come up to the problem in these examples, we'll use a less formal approach that will anticipate the formal methods later on.

Starting Simple

It's best to start out with some simple examples that capture the main principles of systems science and then work our way toward greater complexity. We'll start with Mech-Systems, both human-designed and naturally occurring systems since these are among the simplest of all 'interesting' systems. It is fitting to start with human-built systems since some of our earliest recognitions of systems principles started with studying our own machines, usually with the intent of improving designs.

Below I will start with a very simple object that has 'behavior' and delineate the basic principles that I've written about previously. Additionally, we will see how this machine has been 'modularized' and incorporated into larger, more complicated machines, demonstrating the principle of emergence even in human-built systems.

Deterministic Black Boxes

Suppose we have a box sitting on a table with three wires sticking out of it [in truth there will be two more wires sticking out to supply operating current to the device. These are the energy flow input and output while the three wires are signals — more on this later]. Two wires are input, in that they receive a voltage from some other source (two sources generally), and the other wire is an output. Figure 1 shows such a box in four different states of behavior given the indicated inputs. A zero indicates no voltage input while a one indicates some voltage input. Similarly a zero output means the box emits no voltage while a one means it is emitting a voltage.

Figure 1. A black box with two inputs and one output. This device is performing a logical AND operation. a) two zero inputs produce zero output; b, c) a single input of one on either input lines still produces a zero output; d) two inputs of one produce a one output.

In a real sense it doesn't matter what is inside this box, or even that the inputs and outputs are necessarily voltages. This could just as easily be done with water pipes or mechanical levers as long as the inner workings of the box are compatible with the nature of the inputs. This black box performs a simple Boolean logical operation called AND, and the device is called an AND gate. The AND behavior rule is this: the only time the output will be something rather than nothing (e.g. 1 instead of 0, or also couched in terms of 'true' rather than 'false', simply a convention of semantics) is when both inputs are something (not nothing) at the same time; at all other times the output is nothing. Or in terms of voltages on wires, if both inputs carry positive voltages than the output will be a positive voltage; otherwise the output will be no voltage.

The four situations cover the entire range of input conditions and output results. If we were to observe such a box for a very long time, testing all of the four input choices many times, we would come to the conclusion that this device, however it operated internally, was an AND gate.

We might have a similar looking box and run the same set of input experiments but get the following results: 0,0 -> 0; 0,1 -> 1; 1,0 -> 1; 1,1 -> 1. Note that the first and last output behaviors are the same as for the AND gate, but the two middle behaviors are different. What is this box doing? It is a logical OR gate, producing a 1 output if either one or both of the inputs is set to 1. There is also a gate that performs an EXCLUSIVE OR, which simply means that one or the other of the inputs is a 1, but if both are set to 1 the output is 0.

Now lets say we have yet another black box with only one input wire and one output wire. This device has an odd behavior. When the input is a 1 the output goes to 0, and vice versa. This device changes the sense of the input and is called a NOT gate.

In all three of these boxes we've assumed more or less continuous operations with inputs switching from 1 to 0 or 0 to 1. The boxes themselves retain no memory of their prior states. It isn't in their job description. But another kind of box we have has a peculiar behavior in that it is capable of retaining a memory of the last state it was put into on its output. This box is actually a combination of two of the above box types wired together to produce a nifty effect. If we take an OR gate and wire its output to a NOT gate we get what is called a NOR gate (see the Wikipedia link for the symbol for this device)! Now we take two of these NOR gates and wire them in a most peculiar way. Please take a look at the diagram in this Wikipedia article on the SR LATCH. This one is a bit difficult to explain but the behavior is not. This device uses internal feedback between the two NOR gates to capture and hold a state that was set into it by the values on the two input wires. The state behavior is given in a table in the article. The two output lines are labeled Q and a Q with a bar over it (meaning NOT Q, which means it will have the opposite output from whatever Q has). Generally speaking we will only use the Q output. The way this device works is that if both input lines, labeled S and R, are set to zero, the output at Q remains what ever value it had previously been set to. In other words, it represents the memory of the last transition (from 1 to 0 or from 0 to 1). The operation table shows you what you have to do to S and R to get the Q line to be either a 1 or a 0. The fourth situation, where both S and R are 1s is not a viable condition, it is ambiguous, so this input situation is not like the AND or OR gates.

What is important to note in this example is how we built up a slightly more complicated box from combining several less complicated boxes — wiring their outputs to inputs. Also note that we have done this by design so we can't call this device a black box because we know how it works. This is, instead, a white box — its workings are fully disclosed (we might have called it a transparent box, but that is too many syllables!). One can transform a black box into a white box if one can dissect the black box without destroying the workings. For machines this is trivial and possible. Note that certain dissections of other kinds of systems may destroy its capacity to work and we will lose information about it by the act of taking it apart.

Putting things together in cool way

At this point we can start to do some very interesting things. Boolean logic, while interesting to a point, is very simple stuff. But being able to do arithmetic with an automated machine is starting to get really interesting (OK, so today we take calculators and computers for granted, but that doesn't mean people know much about how they work inside!) We have all the elements we need to do this with the above logic gates and the memory device, the latch. We can put them together to create a device that can add two single bit binary numbers together. Binary arithmetic is based on the ability to represent binary numbers (where the digits are only 0 and 1). A small example may suffice if you already understand positional value notation, as in the decimal system where the digit to the left is worth ten times the one to the right. It works the same in binary except that the digit to the left is double the one to the right. So in binary we have the following numbers:

Binary	Decimal Value
0	0
1	1
10	2
11	3
100	4
101	5
110	6
111	7
1000	8

Zero and one are pretty obvious. The decimal number 2 if formed in the same way you form the number 10 in decimal, by adding 1 to 9 (which gives a 0 in that column) and carrying the 1 to the next column to the left. For a binary representation of 2 we simply add 1 and 1, leaving a 0 in that column and carrying a 1 to the next column. Here is an example of adding decimal 3 and 4 together in binary format.
    0011
+  0100
---------
    0111

Starting from the right-most column, 1 plus 0 = 1, no carry; same in next column; and 0 plus 1 also = 1. Hence the answer, 0111, is decimal 7 as expected. By the way the leading zero has a purpose I'll get to next.

Now lets do something a bit more interesting. Add decimal five and decimal four, that is:
    0101
+  0100
---------
    1000
which is decimal eight.

The difference here is that when you add a binary 1 to another 1 the answer is 10, or you might say 0 carry a 1 to the next column.

So what does all this mean? We can put the component boxes that we've developed so far together in a way that allows us to add two binary numbers with four column just using logic gates along with a set of memory cells (latches) wired together to form what we call a register (a four-bit register). We need three of these, two to hold the operands and one to hold the result. The description of how this is accomplished is more than I want to get into here, so I will let the interested reader refer to the Wikipedia article on adder circuits. The main point I want to get across is that starting with some reasonably simple components we have built a device that can add two numbers together. The number of bits in a register is rather arbitrary. We showed a four-bit value, but in real computers the registers are generally 16, 32, or 64 (some 128) bits wide. As homework you should try to find out how big a decimal integer can be represented in a 64 bit register. There is also an interesting fact about how negative integers can be represented, but this isn't really a course in computer architecture so we will leave it at that.

But with the assumption that you can represent and add negative integers you suddenly have subtraction from a simple ability to add. And once you get subtraction you have all of arithmetic. Multiplication is just repeated additions: 2 x 3 is just 3 added to itself. Division is repeated subtraction, with the caveat that you need to make arrangements to handle a remainder value.

By now, though, I think you get the picture that more complex machines are built up from simpler machines, combined and recombined in ways that increase the kinds of functions that they can perform. An adder circuit, plus some additional logic gates can produce a device that we call an 'arithmetic-logic unit', which is at the heart of all computational devices. Along with a bunch more registers to store intermediary results and keep track of operating instructions and where we are in the program, and a set of control logic circuits to control sequencing through the program, you have a 'Central Processing Unit', the brain of a computer

Add huge banks of registers, called memory, wires to get the bits in the CPU to and from the memory, and some specialized logic/register devices for input and output and you have the modern computer. Simple, no?

A Computer

OK, maybe it isn't so simple when you look at the big picture (or inside a modern computer). At least it doesn't look simple. But in fact it is fundamentally very simple when you realize how it is built up by combining these few simple circuits in a hierarchically increasingly organized way. Note that the complexity of a computer comes not from having a vast number of different component types (unlike chemistry that has 92 naturally occurring elements to play with), but with the way the few components are recombined into a hierarchical structure. Each new level provides seemingly new functionality.

The computer is a wonderful example of both hierarchical complexity and evolution! Most of you can remember when the home computer was a largish box with small memories and primitive input/output options. Remember the days before the Internet? But engineers are always tinkering and thinking up new ways to recombine components to get more speed and function. Some of those designs work well and are picked up by the marketplace. Others wither and die, even though they worked in a technical sense, they weren't fit to survive the rigors of the market.

The computer and its related technologies illustrate even more interesting aspects of systems. The computer is a 'perfectly' deterministic machine (as long as the parts don't fail). If even one tiny bit, especially in the CPU, goes out, the whole machine crashes, even though that bit only talks to a few other bits in the machine (remember, mech-systems have much sparser interconnections among parts). Computers are examples of brittle machines. Engineers are beginning to learn from nature, biology specifically, and starting to look for ways to reduce this brittleness. For example, having multiple 'cores' or more than one CPU working in concert helps. If one CPU fails, the other can keep going. It may slow things down but there won't be a catastrophic failure of the whole computer. This is the result of redundancy. Each component is still brittle, but by having several available, if one goes down another can pick up the work load. In the future, engineers hope to develop less brittle, more 'adaptive' components. Redundancy can be quite costly, so having a component that may get sick but still perform acceptably by adapting to its hurt (like limping) could be much better.

Finally, while a computer might be a very deterministic machine, it turns out that we can now build elaborate, complex systems out of myriad computers, like the Internet, where overall behavior becomes stochastic. That is it becomes subject to the laws of probability. This is especially true for computer systems that interface with the real world through sensors and effectors, like robots. The real world is highly stochastic, so the states of inputs can never be sufficiently determined for the computer. This is how Mech-Systems can actually have more life-like behavior, or at least probabilistic behaviors. Even man-made machines are getting so complex that we cannot always be sure of their performance. It is an example of how perfectly deterministic subsystems can, when interacting through a noisy medium or interacting with external stochastic processes, become non deterministic in their overall behavior. It is not at all unlike the way non-living atoms, when combined in particular configurations and interacting with one another in a warm watery bath can produce the phenomenon we call life.

A Stream^*

If you've ever been trout fishing or even just hiking by a mountain stream you may have noticed that despite the swirling and splashing, first running fast, then slowing in tranquil pools, the stream has a very interesting organization. The energy flow that propels the water downward is the gravitational gradient. Water condenses high in the mountain, in the watershed, and collects in a low lying crevice in the topography. It then builds momentum flowing down the gullies and valley ways. Gravity and surface tension of water, along with the banks, help form the boundaries. They are not impermeable as water evaporates or splashes up on the rocks. Animals drink from the stream and carry the water away.

Energy also enters the stream in the form of nutrient runoff from the watershed. Falling tree leaves and other debris that serves as food for insects and they for fish. Clearly the stream's energy is dissipated in flowing eventually to a lake or river or to the ocean.

With the exception of the life that the stream may harbor, it is basically a Mech-System in terms of the mechanical and chemical interactions it has with the immediate environment. Over time the stream's course will change as it carves new channels or reshapes the banks. During heavy spring snow melts the banks may overrun accelerating this process, but there is a limit to the extent of overflow and the integrity of the stream system is maintained over time, as long as precipitation continues annually. The stream overflowing is an example of the difficulty in choosing boundaries mentioned at the very beginning of this installment. If you were to see the stream only during one of these events, you might choose the boundary as high above the 'normal' bank. Contrariwise, if you visited in late summer or early autumn you might choose a narrower boundary as the flow rate has shrunk to a trickle by comparison. This is why systems require careful observation over long stretches of time and under many different environmental conditions so that you can take all of these phenomena into account when choosing a boundary.

While it is easy to think of a stream as 'only' a Mech-System, it is an important example of one that forms the environment of living systems, Bio- and Eco-Systems. Indeed the interactions between the biome and the stream can have tremendous impacts on the evolution of the stream itself. For example, salmon streams are heavily influenced by the annual migration of spawning salmon up to the higher elevations where they die after spawning. This has an incredible impact on the chemistry of the stream as their bodies decay. So, while the running water itself is a Mech-System, taking a wider boundary view can readily convert the system of interest into a full blown Eco-System. This is an example of why there must be an on-going effort to observe all interactions at your chosen boundary to make sure you have captured all of (what for you) is relevant dynamics. This is not particularly easy to do for very complex systems, but it is the only way you will know. In a sense, we adopt the black box philosophy. We watch the chosen boundary very carefully and measure everything that goes in and comes out (Fig. 2). Careful metering of the inflows and outflows will tell us a lot about the choice of boundary we've made. But then, to really know, we need to take the white box approach to make sure we know that the system we are studying has the interesting internal dynamics as well.

Figure 2. The only way to be sure of the correctness of your choice for a boundary is to monitor it closely and instrument (if you can) the inflows and outflows. The little 'meters' set on the boundary to measure all of the relevant flows can give a quantitative method for determining the black box analysis of the system. The arrows leading out of the meters are data to be collected and analyzed.

A Hurricane

As a final example of a natural Mech-System, consider a hurricane or typhoon. These circular patterns of high winds are actually vortices in which there is a vertical temperature gradient from the warm ocean surface water to the upper cooler atmosphere. Once a tropical storm system forms, and there is also a pressure gradient (horizontal) heat can be conducted upward through the storm system in rising concentrations of moisture. The laws of physics regarding the formation of whirlpools, vertical fluidics acceleration, cause the system to form a vortex producing the winds. At the surface this increased wind speed enhances the evaporation and heat conduction even more. The heat is now being removed by convection from the ocean surface and convection is a much more rapid means to carry heat along a gradient.

The process is self-reinforcing as long as a temperature gradient exists (this is why hurricanes fall apart once they are over cooler land). Hurricanes are short-lived phenomena, compared with streams, but during their time, they develop considerably complex structures, such as the eye where no wind blows.

Note that the boundaries of a hurricane are actually harder to define than for a stream. One way, commonly used, is to mark the periphery where the wind velocity falls below a certain threshold. That is somewhat arbitrary, but it is based on years of observation and determination of just at what speed the winds are considered destructive. The air/water boundary seems easier to declare but with the wave action and evaporation pumping water into the air, it isn't all that easy to instrument!

Energy Flows and Dissipative Structures

These Mech-Systems, and all active systems, demonstrate a key fundamental principle in how systems organize and work. In every case the system exists within an energy gradient that "drives" the dynamics of the system. The components interact is constrained, that is lawful, ways, but it is the flow of energy through the system that makes things happen. Energy flows involve temporary residence, as it were, in internal micro-structures within the system's component interactions — the subsystems — and this energy, in potential form, can do very specific work before dissipating as waste heat. This is how the internal structure of systems is formed, how they become organized. In the case of the computer, or any manmade machine, new organizations come about through invention and product development rather than in situ. But in the case of a hurricane, organization develops as a result of energy flowing in real time.

As long as energy flows from a high potential source, through the system, to a low potential sink (the gradient) the system will tend toward greater organization of microstructures that best (are most fit for) dissipating heat by doing internal work. This will continue from the time the initially disorganized system first finds itself within the gradient, until a time when no better dissipative structures can develop. All the options have been tried by trial and error and no new structures emerge. This is the steady-state, which will be maintained so long as the same energy gradient applies. If the gradient intensifies or diminishes the system will respond accordingly. There will be much more to say about this energy flow phenomenon in the next two installments on Bio-Systems and Eco-Systems where the effects are truly astounding. For now, just recognize that dynamic Mech-Systems are just as subject to the organizing principle of energy flow as all other systems.

---

^* Thanks to Charlie Hall, at SUNY-ESF for the inspiration for this example. He is teaching a Systems Ecology course (I'm sitting in on it) in which the students took a field trip to a local stream where they took samples of all sorts of parameters. They brought their samples and data back to the lab where they are now in the process of building a systems dynamics model of the stream.

The Science of Systems 9

Systems Science 9 — Major Classes of Systems

I find it useful to consider systems in three major classes or types. There is a fourth type but I think of it more as a kind of an abstraction than a real system, at least at this stage of the universe's life. I'll explain that in a bit.

The three non-abstract classes are:

• Mech-Systems
• Bio-Systems
• Eco-Systems

Below I will explain each of these, but a rough idea here will lay some groundwork. First, the "abstract" type of system can be called the Chaos-System, or Unorganized-System. Think of a completely insulated flask containing gas molecules, say hydrogen as H₂. Allow the system to age for a really long time and according to thermodynamic theory inside the flask the temperature is the same everywhere and things are just bouncing around. Very boring! The reason this is abstract (even though you can imagine such a flask existing in reality) is that any attempt to peek inside to see if things are really chaotic would introduce an observer which would be at a different temperature that would cause the system to undergo changes (even a really minute measuring device would still cause a change!) Thus we can only use such systems as an abstract 'ground state' in a mathematical sense. Still it has its practical uses in being able to characterize real systems in relation to their abstract (theoretical) ground state. While an important issue, for purposes of this short version, I will leave it at that. Plenty of books on thermodynamics, statistical mechanics, thermochemistry, etc. lay it out in excruciating detail, so interested readers should refer to them.

My classification scheme (of what we should think of as 'interesting' systems) is based on several parameters or dimensions derived from the previous installment on complexity, etc. Leaving aside thermodynamic order, which is highest when the temperature is lowest, I consider four basic dimensions that help to classify systems. These are actually a pair of two dimensional spaces: competitive vs. cooperative component interactions and complexity vs. organization. I should hasten to point out that this is not the only two spaces one could conjure to elucidate system types. Indeed, it would be useful to consider multi-dimensional (hyper-)spaces to see if the analysis bears consistency when many factors are taken into account. But these two spaces should prove instructive for my present purposes. In the figures below you will see ovals representing the three types of systems. These are conceptual aggregations. The idea is if you were to take a survey of real systems according to some measures on these dimensions they would be placed within (mostly fuzzy) boundaries within the larger space. This is, after all, how we define classifications. Seemingly different systems might share similar measures on these dimensions and hence end up clustered within the 'glob' represented. I must emphasize that this is a hypothetical framework. As far as I know no one has done such an analysis rigorously. What I offer here is based on a very anecdotal survey of my own. But I think you will see the general merit of classifying systems in order to bring some organization to systems science, just as classifying plants and animals (systemics) helps organize our understanding of phylogenetic, behavioral, and physiological relationships between different flora and fauna.

Figure 1 shows a two dimensional space based on complexity (vertical axis) and organization (horizontal axis). The origin at lower left is labelled low for both dimensions, while the top and far right represent high levels of these dimensions. The three types of systems are shown as rough areas occupying somewhat overlapping regions of the space. I've included the Proto-universe (after the Big Bang), galaxies, and solar systems as kinds of Mech-Systems, which I will explain below. As represented here, Mech-Systems are toward the low end in complexity, having few components and few interactions. They can, however, be very high on organization. As we will see, human engineered machines fit this category nicely. Bio-Systems occupy the region in the middle, having medium to high organization and complexity both. Eco-Systems are generally very high in complexity, but not necessarily very ordered (we now know that even so-called 'climax' ecosystems are not long-term steady-state systems).

Figure 1. Complexity versus Organization as a way to classify system types. The overlapping regions implies that there are no 'clean' boundaries. Eco-Systems contain Bio-Systems, for example, a bee colony has aspects of both ecosystem and macro-bio-system.

The second two dimensional space relates the dominance in the tradeoff between competitive processes (component interactions) and cooperation. Bio-Systems rely more on cooperation than competition, though the latter is essential for certain biological functions. Mech-Systems are characterized by higher degrees of cooperation among the parts. Human engineered machines are the quintessence of such systems where absolute cooperation between the parts are essential. But we can also consider non-engineered, but mechanical systems, such as the Earth's crust or a mountain stream as a Mech-System with some amount of competition going on between the parts.

Eco-Systems tend to have more competitive interactions between 'components'. At the low end of an Eco-System we find populations of the same species, where individuals are in competition with one another for food and mates, but still manage to cooperate for the good of the species. At the high competition end we have a typical rocky intertidal zone where numerous species and individuals within a species are continually competing for real estate (which usually translates into food and mates!). A critical question that we need to address is: where are human societies placed in this figure? As I will argue, there are societies (examples) that run the range from relatively high internal competition to relatively high cooperation. But in general humans seem to center somewhere in the overlapping region of Bio- and Eco-Systems. I will address human society as a special case of Eco-System that may need to tend toward Bio-System if we are to survive in the future.

Figure 2. Cooperation versus Competition as a way to classify system types. Some kinds of systems based on this scheme are infeasible, they cannot have both high competition and high cooperation simultaneously. On the other hand, minimal interactions with some low cooperative and some low competitive suggests simply a disorganized, possibly chaotic system, as our abstract system mentioned in the text above.

Mech-Systems^*

This class of systems is characterized by the preponderance of cooperative interactions among the component parts. Additionally most components interact with very few other components directly. The most obvious members of this class are man-built machines. Machines are designed to fulfill very specific functions and their working components are designed to work together, cooperatively, to produce that function. However, if we are not too strict on the absence of competitive relations some natural systems can be fit into this class, especially atomic and molecular-scale chemical systems where quantum effects are not too noticeable. Even though DNA molecules are associated with living systems, the molecule itself is not living. It is, however, fairly stable (thermally) and rigidly structured in the right temperature and pH aqueous solution. It can be manipulated chemically in very predictable ways so that pico-machines can actually be built using molecules of DNA.

Machines are generally characterized by their degree of deterministic behavior. They can be counted on to do what they are supposed to do. The better the design and fit of all the parts, the more durable the materials used, the more reliable the machine is for a longer duration. But entropic processes still get their due. Parts wear out eventually and unevenly and can alter the performance of the machine in time. Energy is required to replace or repair the workings.

Machines are a good example of the difference between organization (in the sense of complexity) and thermodynamic order (third law of thermodynamics). Because machines are more likely to do what they are supposed to do, they can be said to have a high level of order, as if their parts are 'frozen' in their relationships to one another. In this regard they are like a crystal in which the bonds between components are 'frozen' and the number of states of the system are restricted.

Figure 3. A Mech-System is characterized by components with few interrelations with other components and those relations are "frozen" such that the machine produces predictable outputs. Here the whole machine is represented by a rounded-corner rectangle. Each component is a circle (thin line circumference). Lines between components (interactions/interrelations) are few and the overall structure is rigid, like a crystal.

As mentioned above we should not restrict the notion of Mech-Systems just to machines. Many natural systems have machine-like properties in that they have fewer interactions between components and those interactions are stable over time. In natural systems, however, many times these interactions are stochastic, that is they are probabilistic over time. One example is a waterfall. Over time the water pouring over rocks generally follows a pattern, even if the details are not specific. Waterfalls are chaotic systems with no drops of water following preordained pathways. The interactions between water flow and rock is that the latter gets gradually eroded so as to change the general pattern of the flow. So such Mech-Systems do evolve over time (actually not unlike the wearing out of mechanical parts in a human-built machine). Nevertheless, the system is fairly simple and organized. One might also argue that there is a balance between competition (erosion) and cooperation (channeling the general water flow).

Bio-Systems

Bio-Systems can be thought of as more complicated networks of quasi Mech-Systems (like the DNA example above) in which there is greater flexibility in the network of connections; they can vary over time. Clearly individual living organisms are the paradigm example of Bio-Systems, hence the name. As we will see, Bio-Systems are somewhere between machines and ecologies. In the latter case, relationships are constantly in flux and systems can only be characterized by changing network structures. In Bio-Systems there is less rigidity in interrelationships than in Mech-Systems. There are generally many more connections between components providing multiple redundant pathways for interactions. A key characteristic of Bio-Systems is the trade-off between cooperation of components (as in the machine) and competition between them under critical conditions.

Individuals, and in the case of eusocial animals, colonies, are composed of components that generally cooperate, or at least the dominant mode of operation is cooperation. But this is not a strict rule. During development there is often competition between cells as they differentiate and vie for location. In the cortical tissues of more advanced vertebrate brains, even in adulthood, there can be an on-going competition for processing power and physical space between modular functions. This is demonstrated most readily in the motor cortex where plasticity of the neural units hinges on use-it-or-lose-it rules and in the case of loss of a digit or limb results in the areas once dedicated to controlling that appendage's movements will be colonized by nearby areas controlling similar appendages.

Competition comes into play especially when the system is under stress from resource constraints or disease. Some non-critical subsystems are shut down or at least down modulated so as to allow resources to be routed to critical functions. This is true at the single cell level and at the whole individual level in multicellular organisms. In this case a logistical control subsystem is responsible for mediating the competition (regulation) to ensure that the critical function wins.

There are several other examples of competitive processes operating under normal circumstances within an individual's body but where logistic regulation prevents destructive competition. One example is opponent paired muscle sets that move joints or maintain posture. Muscles like the biceps and triceps operate in a dynamic tension to control the motion of the forearm. Neural controls, which include cross inhibition, make sure that these two muscle sets don't over-flex and produce damaging tensions around the joint or tear each other apart. Without such logistic regulation, controlled fine motion would not be possible. Another example is related to the neural circuit just mentioned. Throughout the nervous system (though generally more prevalent in brain stems) there are examples of circuits called 'central pattern generators'. There are many different versions and these circuits are responsible for rhythmic motions like walking/running gait control. They work by sets of neurons, generally through a sequence of other neurons, mutually inhibiting or exciting one another. Their outputs tend to resemble oscillatory excitations that are sent to specific effectors, like the logistic regulators for opponent paired muscles. They are especially useful for undulatory motion such as peristalsis in the digestive tract. In these circuits the neurons appear to be competing with one another for excitation (say through cross inhibition), but the phase differences created by the effective distance between cells and their terminating synapses allow the circuits to produce a cooperative output. [see also: Mobus, G.E. and Paul S. Fisher, "Foraging Search at the Edge of Chaos", in D.S. Levine, V. R. Brown and V. T. Shirey (Eds.), Oscillations in Neural Systems, Lawrence Erlbaum Associates, Publishers, Mahwah, New Jersey.]

Evolution has worked to produce individuals in which cells and tissues generally cooperate with one another in a well choreographed dance. Complexity is quite high compared with machines since many component parts interact in multiple ways with many other parts, the interactions are most often context dependent. The range of possible states is huge by comparison with machines, but it is not, technically, infinite. States are defined by molecular-level interactions within cells, so at a certain level of observation the system appears to be continuous rather than discrete. But if one examines the micro-states of cells, one discovers more distinct transitions as biochemical processes proceed. Still, for generally practical purposes the systems can be treated as continuous without (usually) loss of information. However, caution is advised in treating this too literally. A single micro-mutation in a DNA molecule in a stem cell or a germ cell can lead to considerable shifts in the future states of individuals.

Competition between certain components in a Bio-System are conserved through evolution because competition is useful at critical times in the life of individuals. Cooperative processes, via logistic regulation, dominate so that the individual acts much like a machine most of the time, at least during its adult, sexual stage. The parts work in concert until age or disease cause them to falter or fail. But, at the same time, their normal function is stochastic in nature, meaning that the variability of behavior of parts is higher than in a strict machine. Such variability stems from retention of the plasticity of sub-cellular and tissue-level components. This plasticity is what allows Bio-Systems to adapt to non-standard environmental conditions when necessary (and in brain tissue it is the basis for learning). The downside of maintaining such plasticity is that sometimes parts don't behave exactly as they should and because they are interconnected to so many other parts this can cause intrinsic (that is, not from the outside) malfunction. The parts may get 'out of synch' with one another leading to oscillatory fluctuations. Generally this is temporary and correctable. But with more complex Bio-Systems, more parts means more potential for things going wrong. Many genetic-based diseases fall in this category.

Bio-Systems are, thus, highly organized and the parts are highly interconnected making life a very complex kind of system. At the same time they are intermediate in terms of thermodynamic order between machines and Eco-Systems. Evolution of Bio-Systems has been described as keeping species on the boundary between order and chaos. They are just organized enough, and sufficiently controlled so that they are successful in meeting the challenges of living and procreating. But they are flexible enough and subject to just enough intrinsic fluctuation that they are not strict automata. They can deviate from the norm as a form of 'exploration' of the environment and of useful behavior space.

Figure 4. A Bio-System is characterized by more interconnectivity (greater organization) between components. Here components are represented as Mech-Systems but real Bio-Systems are more often hierarchies of Bio-Systems within Bio-Systems. The whole Bio-System is here represented by a thick line circle. More cogent characteristics, not shown in this figure, are the relationships in time between competitive and cooperative interactions.

Eco-Systems

The third class of systems is characterized by a high degree of complexity (generally) and a low degree of thermodynamic order. These systems are aggregates of multiple Bio-Systems from multiple 'species', often multiple Mech-Systems, and non-living components, having many more interactions, with much higher variation in both kinds of interactions and degrees of coupling strength in those interactions. More importantly there is often a much higher degree of competitive types of interactions, even between conspecifics, relative to cooperative ones.

Figure 5. An Eco-System is characterized by somewhat less, but highly dynamic and variable interconnectivity between components. Components are of every other type of system, Mech- and Bio-, and can also include nested Eco-Systems.

Here I look at two basic kinds of Eco-Systems, the standard idea of an ecosystem as studied in ecology, and a Social System, an ecology of social organisms. Of course the one we tend to be most interested in is the human social system.

Biological Ecosystems

Ecology is the study of natural environments and their component biological subsystems. In fact many ecologists have eschewed the systems concept in describing their units of interest because the term too often is associated with Mech-Systems (machines) and ecological systems are anything but like a Mech-System. However, as this scheme of classification implies, ecological systems include Mech-Systems, the non-living substrate and general physical environment, as well as Bio-Systems from viruses to megafauna and plants.

Ecologies are often studied from the standpoint of systemics (what species are present), dynamics (how they interact with one another — behavior), population dynamics (how species grow and contract), evolution (how some species diminish over time and other species invade, changing the dynamics), and, very importantly, the thermodynamics or energy flows from inputs (via primary producers) through trophic webs (primary consumers, carnivores, etc.) to microbial recyclers.

I don't propose to go into any greater detail in this section since so much has been written about ecology and there are still many fine systems theoretical perspectives in play in the field. If this ever becomes a book on systems science I would probably provide some of these examples plus relate various principles from prior installments to specific ecological principles to show how systems science (and not just Mech-Systems science) should still be the overarching framework for this field. This is especially important because of the relationship between the non-human ecology and human social systems. Though many wise people throughout recent history have pointed out that the human ecology has to be regarded as a subset of the larger ecology (of the Ecos) it is only now that we are about to disrupt that ecology by not recognizing this fact that truth dawns.

Social Systems

My argument is simple. Human societies and cultures, including all artifacts, behavioral conventions, institutions, and especially agriculture, are just as natural as any wild forest or coral reef. The only difference is that humans are clever enough to overcome any of the constraints of the larger Ecos that might have kept us from overrunning the world. In the process of expanding our territory out of Africa we have exploited other ecologies, other resources, and often destroyed them in the process. Humans have now succeeded in appropriating over 40% of the primary productivity of the planet for their own trophic uses, and there are no natural predators waiting to check our population.

Social Eco-Systems depend on the strong cognitive and emotional interactions between individuals making up the social group. Ultimately that is what binds us. As in a general Eco-System these interactions may be restricted to a few other individuals (at least at a time), but through chains of interactions with other groups, and elements of our culture (e.g. TV) we have evolved an extremely complex and generally organized ecology.

Eco-Systems are, however characterized as much by competitive processes as cooperative ones. In the case of biological ecologies, the shifting dynamics of a region allow for an evolution of natural feedback loops that constrain the components from 'taking over'. Human societies have also evolved to introduce some elements of regulatory constraints to help coordinate activities and keep competition from causing too much damage. But this evolution is really just begun. Governance and economic market activities are still relatively young inventions and even though we would like to believe we have developed the perfected form of both, in the form of democratic capitalism, there are still many flaws and unrestrained aspects as we have recently witnessed in the economic crises we find ourselves in now.

Humans continue to evolve, and assuming that there are human survivors from any potential cataclysm due to climate change and net energy decline I fully expect that it will be in the area of strategic thinking capacity (an actual development in the brain). Along with that evolution I expect that the social structures involving logistical and tactical coordination (in other words governance and markets) will also develop. For one thing we have learned that political process breaks down into ideology and dishonesty (with money involved). Politics has turned out to be a horrible way to form and run a government. And markets without appropriate regulatory oversight (along with foolish market participants) are disasters in the making.

The human social Eco-System is actually evolving toward being more of a Bio-System in which coordination resolves the issues of competition that cause Eco-Systems to be so dynamic and in a state of constant flux. Bio-Systems are stable over time whereas Eco-Systems can go through drastic changes from time to time. As an individual component in either of these two kinds of systems, which would you rather be? A Bio-System provides greater predictability, stability, assurance, etc. An Eco-System is a grab bag. As human evolve to have greater moral fortitude, altruism, empathy, and strategic perspective, they will work to evolve social systems that function more like Bio-Systems than Eco-Systems. But that is a long way off. The current species of human has only just awoken to the desire for Bio-System status. It will take a long time for this species to give way to the next, more sapient species that can complete that process.

---

^* I use this hyphenated form in Mech-, Bio-, and Eco- systems to denote a generalized type of system. In fact other non-living systems might fit into the Bio-System class, and social systems can be analyzed as a form of ecosystem, hence the Eco-System class.

The Science of Systems 8

Systems Science 8 — Complexity, Organization, and Order

We've looked at the boundary of a system as a way in which one system is delineated or differentiated from the environment. The boundary 'contains' the elements or components of a system. The word 'system' itself implies some kind of internal organization or structure along with function. In this installment we will take a look at the internal milieu of a system and make some general claims about organization that seem to hold for many different kinds of systems at all scales and composition.

What we are trying to get a handle on is the concept of complexity as it applies to the internals of systems, and by our recursive definition of systems the complexity of relations between multiple systems. We have already examined some aspects of how systems evolve from a lesser organized state to one of greater organization. We've also hinted at how subsystems organize and give rise to wholly new functions and interactions with other subsystems. The latter is what we have called emergence in that new structures and functions emerge from lower level subsystems interacting in new ways that become stable, consistent with the energy flows through the overall system.

'Complexity' is a tricky word. Even among scientists who study complexity in its many guises there is not always agreement about what it means or implies. So I want to be careful to delineate how I am using the term and why. Specifically I do not want the word to carry too much burden of meaning. Some people look at highly organized systems with many interacting parts and call that complex. They are not wrong, but the word then implies not only numerous parts of numerous kinds, but also highly specific and possibly stable interrelationships between the parts. The problem arises when one looks at a collection of many component parts of many types and may not see any pattern of relationships that remain stable over time. Is this a lack of organization in the system or a lack of perception and understanding on the part of the observer? In Installment 9, Classes of Systems, I will explain this more thoroughly.

A similar problem exists with the term 'order'. This implies a stable, consistent, perhaps even rigid structuring of the component parts. Complexity, in the sense of organized structure, is sometimes equated with order, but this is not quite right. Order, at least in the sense of the Third Law of Thermodynamics, is of the type of a crystallized system. There are a few well defined structural relations that have become literally frozen in time and space. This can be achieved by lowering the system's temperature to a point where the internal thermal motion is minimized and the components settle into their minimal energy configuration with respect to one another. Or at least they are trapped in a local minimum, with a global structure that minimizes total entropy. The point, however, is that any complex system might be frozen into an ordered state but this wouldn't tell us much about how interesting the system is.

And I think the issue is one of how interesting a system might be. Crystals might be pretty, like a diamond, but they are not terribly interesting as objects go. At the other extreme a gas phase mixture of many kinds of atoms is similarly not very interesting if held at a temperature that prevents interesting chemistry from taking hold. In the first case of the crystal entropy is at a minimum but nothing is happening. In the second case entropy is at a maximum with too much happening, but none of it interesting. What we find interesting is the somewhere in between phase where things are happening, but more than just atoms colliding with one another. What we are really interested in is dynamic organization, that is, where structures can emerge and interact in new ways.

Complexity Defined

Lets start with a straightforward definition of complexity. Then we will relate it to the ideas of energy flows and emerging organizations. In the end, we will see that this is precisely the basis for what we call universal evolution.

I will start by describing a multi-dimensional space in which each dimension measures an attribute of complexity. The first dimension is simply the number of kinds of atomic components. The word 'atomic' here means components that cannot (or need not) be broken down to simpler forms, as in the original Greek usage — atoma. The number of kinds can easily be finite, as in the number of kinds of naturally occurring atoms in the universe. In fact, I sometimes think that an infinite number of kinds would be a meaningless concept. Of course the number of kinds might be extensible just as we have extended the number of kinds of atoms in the periodic table by nuclear chemistry. But this isn't essential to the basic definition.

A second dimension would be the numbers of atoms of each kind — their relative abundance in the system. In the case of the universe, for example, the atoms of hydrogen and helium are by far the most abundant proportional to carbon, nitrogen, oxygen, etc. This dimension is important, but by no means crucial to the potential that complexity offers. We know that the existence of carbon, for example, counts for a great deal when it comes to organization.

A third dimension measures the number of pair wise interactions that are possible between components. This dimension is somewhat more problematic definitionally. In regular chemistry, for instance, interactions are never purely pair wise (e.g. reactions that take place in solution). The best that we can say for this dimension is that it measures the number of rules by which two atomic components might interact in as pure a context as possible. For example one might define a set of 'standard' conditions and then enumerate the ways in which any two components might interact. For example, in chemistry and subatomic particle physics there are definite rules of binding the components together (e.g. number of electrons in the outer shell of an atom determines the kinds of chemical bonds). The need for standard conditions introduces one or more additional dimensions that measure the various aspects of the environment which 'condition' the interactions. Here we see that complexity itself becomes complex to define!

This four⁺-dimensional approach to measuring complexity is very general. It should be applicable to any level in the ontological hierarchy (quantum physics, chemistry, biology, etc.). The measure of complexity is thus not a single number but a n-tuple (e.g. a list of numbers and other m-tuples separated by commas, where the length of the list is the number of dimensions and the numbers are appropriately scaled). The actual value of n will depend on how many environmental conditions are involved in describing the interactions. A list containing any tuple with low values is likely to describe a relatively non-complex system. For example, the four-tuple, (2, (10^100,000, 10^20,000), 1, 5), would represent a rather impoverished system with only two component types, one interaction between them and five conditions, even though it has a huge number of components of the two types (the 2-tuple as the second item in the four-tuple). [Note: I cheated a bit here. The last number should actually be a 5-tuple itself, with each number identifying the range of condition parameters, but I thought that might be overkill for this example!]

The salient point is that it is not very useful to think of complexity in too vague terms. Or put another way, complexity is complex! The reason we want more precision in thinking about what it means to be complex is that we can then start to think about what sorts of interesting things a complex system can do under the right circumstances. To be definite, the circumstances I refer to is the flow of energy through the system. Depending on the situation at the boundary and the internal complexity of a system various conditions of energy flow can produce very interesting results. In fact, at some sufficient level of complexity and at just the right flow rate of just the right quality of energies, very interesting structures can form and move and further interact with one another. Emergence can take place. Selection can take place. And organization can take place.

The surface of the Earth receiving the influx of solar energy is such a system. As the planet formed from rocky, dusty, and gaseous debris circling Sol there was much disorganization but great potential in terms of the complexity of the system. Through the combined energy fluxes of sunlight and gravitational forces things happened. The Earth got interesting.

Energy Flow Jostles Components

Energy flow is, itself, a somewhat problematic concept. Energy isn't exactly something you can see or feel, except by its effects on matter. Perhaps the quintessence of pure energy is the ,a href="http://en.wikipedia.org/wiki/Photon">photon. The photon is neither a particle nor a wave, but both, more or less simultaneously. Photons move and mediate the electromagnetic force. They travel at the speed of light through a vacuum and interact with electronic particles like electrons and protons. In doing so they can 'move' the particle and this effect is what we mean by energy doing a kind of work.

Sensible heat (what you feel as heat) is a very low energy form of photon, in the infrared radiation range, jostling the electrons and atoms in your skin, in the molecules of water in your cells, mostly. So heat, the lowest grade of energy, has an ability to interact with matter, but depending on the relative temperatures of the heat source and the heat sink it may or may not do any work to speak of. The higher the temperature of the source, relative to the sink, the more 'pressure' will exist to push the energy flow through the intermediate system. The same principle applies to other forms of energy, e.g. when electrons themselves move and mediate the electromagnetic force (i.e. electricity). In that latter example the pressure is called voltage.

In a reasonably complex but not necessarily organized system with an external source of energy, say heat at a high temperature, and an external sink at a low temperature, energy will flow through the system (if the boundary conditions permit!) from the source side to the sink side, jostling any matter that is in the way. The important point here is that the jostling causes our 'atoms' to bang into one another as they become particles mediating a mechanical force. They also may have the capacity to absorb a bit of the higher potential energy in the form of potential to 'stick' together. Chemical bonds created by jostling are an example. But when the atoms stick together, they form more complicated, organized objects. More importantly, these objects may interact with one another in more complicated interrelations.

In addition to 'chemical' interactions the material is mechanically pushed, so to speak, toward the low temperature sink side. In the strictly thermal model we've been using the act of heating particles at the high temperature source causes the particles to initially spread out creating a high pressure zone at that end. At the low temperature end, the particles are cooled and tend to cluster closer together creating a low pressure zone. Matter moves from regions of high pressure to regions of low pressure. That is why we have wind! So the mass starts a more organized movement from the source end to the sink end. At that end, the cooler particles are displaced by incoming hotter particles and tend to move to the outside (not outside the boundary) of the column of hot-to-cold region moving particles. They, in turn continue to move along the periphery toward the hot end where they will get re-heated, setting up a cycle called a convection cycle. Of course a lot depends on the overall geometry of the system, but some form of organized motion is set up by this energy flow through.

So just with the flow through of energy from a high potential source to a low potential sink, and given a sufficient level of general complexity we have an increase in organization, both mechanical and chemical. And that organization will persist so long as energy flows. Note that the entropy of the whole arrangement, the meta-system of energy source, system (of interest), and energy sink is actually increasing as the potential between source and sink tends toward equilibrium. But locally, within the system (of interest) order increases and entropy declines. Turn off the energy flow, however, and the system will return to its original disorganized amorphous state.

Organization Emerges

A lot now depends on the types of components, their relative abundance, and the kinds of interactions that can take place between them. And some luck helps!

The process described above is what we call stochastic. The actual interactions that take place are a result of chance meetings. However, given that the process is strong and the complexity high, the chances for very interesting interactions is high. Some particle interactions or combinations are more stable under the given conditions of the system with its flow through of energy. It will have a characteristic temperature gradient from source to sink. The same applies to pressures and composition gradients. The stable combinations will tend to persist, to be robust along the gradient and in being so change the relative concentrations of 'atoms' within. Combinations that are not stable will end up coming apart and their components will return to the population of particles. Though even the most stable combinations will eventually come apart just due to chance jostling of the wrong kind. However, this happens much less frequently than the coming apart of not-so-stable species. The internal environment and dynamics of the system tend to select for certain stable forms.

Now, again if the system is sufficiently complex, it will happen that some of these stable forms interact with one another in very interesting ways. They can mutually reinforce or benefit one another. For example species A might 'help' species B by having a strong affinity for one or more of the component parts of B but is able to give the parts to the formation of B without itself being destroyed. A simple example is that of a catalyst in chemical reactions. A particularly important form of catalysis is cyclical where A helps B and B helps A. Such a reaction can easily dominate the system leading to a predominance of A and B. More complex loops of mutually reinforcing species are possible. Indeed the origin of life is now believed to be wrapped up in these mutual 'autocatalytic' loops involving precursors of RNA and proteins.

The important thing to recognize in all of this is that the generation of greater organization is driven by the flow through of energy but is also dependent on some elements of chance. For example of two equally potent autocatalytic processes that might have emerged, would be competitive for components, and are equiprobably, chance will decide which one forms first. And that chance formation might make a great difference in what evolves next.

Evolution

So here we are at the beginning of something really interesting. The system (the Earth) is driven by several energy flows. The one with the most evident impact on the emergence and evolution of higher and higher structure is the Sun. Light energy drives life, for the most part. Gravitational energy in the form of creating high temperatures at the core of the planet (and in the mantle) and the tidal dynamics caused by our moon, have certainly contributed to helping organize the way things developed. The fact that the Earth rotates daily means that the high energy input source and the sink (deep space) alternate so that energy flow is a kind of pulsed (sinusoidal) affair at points below the arctic and antarctic circles. At lower latitudes near either pole the pulses are seasonal rather than daily! Nevertheless, life has evolved on this planet in a drive toward increasing organization and order changing the high entropy complexity into low entropy complexity over the past 4.5 billion years.

Another way to reflect on the nature of organized complexity is our old friends information and knowledge. Going back to "Systems Science 5 — Information and Knowledge" we can see that the structure (knowledge) and the generation of messages by the dynamics of those structures (information) are another way to measure complexity.

We've now built a basis for understanding general systems. We have several 'tools' for describing systems and their dynamics and evolution, at least qualitatively. The reader, I hope, appreciates that there are more formal and mathematically sophisticated ways to handle these concepts. But for purposes of introducing the general reader to the concepts of systems we have made some good progress.

In the next installment I will be turning to the taxonomy of systems in preparation for starting to apply systems theory to the real world. We have to be able to differentiate aspects of real-world systems in order to understand some additional important phenomena, but also to apply our tools to systems we encounter in life. My intent is to show that a grounding in systems science will allow anyone to tackle understanding diverse and seemingly disparate objects and processes in this world.

What is "sustainability"?

The Magic Word

I have been thinking about this word, sustainability, a lot lately. How could one not? It is used everywhere these days. Sustainable development, sustainable energy, sustainable growth! Everyone, it seems, is on the sustainability bandwagon, and I have been no exception.

I developed a model which I call that sustainability criterion for renewable energy production. The criterion is simple enough, a renewable energy production facility (e.g. solar, wind, etc.) is sustainable only if it produces enough excess energy above that required for economic work to replicate itself before its useful life is ended. Of course this doesn't meant that every solar panel has to somehow reproduce another solar panel every, say, 20 years. What it does mean is that in the aggregate of all renewable energy producers and the physical plants that produce those producers, there has to be enough energy produced to produce the next generation of energy producers. On the whole, then, the combined energy outputs of all renewable sources has to serve the economic work process plus provide energy to run the processes that continue to produce the energy capture and conversion equipment.

Currently solar panels and wind mills/generators are manufactured using existing energy infrastructure which mostly comes from fossil fuels. Eventually, when the fossil fuels are no longer available all of these energy producers will need to supply their own reproduction energy just as biological systems must do. To be sustainable, in the sense that life has sustained itself on this planet for billions of years by capturing more energy than just what was needed for maintenance, our invented energy capture/production equipment will have to do the same.

This is a systems-based definition of sustainability. Virtually every profession and perspective will define the term somewhat differently based on their particular view of things. For example, a sustainable business model will reflect the values and beliefs held by capitalists who foresee sales and profits going on and on into the far future. You know, like General Motors! I've even lately heard the term used in the context of education — sustainable education invokes the notion that we can keep the machinery of school-based education (teachers and students) going on forever, with good 'outcomes'.

'Sustainable' has become one of those words that can mean pretty much whatever we want it to depending on who is using it and under what context they're doing so. Too many of these implied meanings seem contradictory at times. At some point it becomes a meaningless word.

The dictionary definition basically says that it means to support or keep going. No real time frame is mentioned. I think most people apply this general concept to whatever is being claimed is sustainable. We'll keep the business going. We'll keep the education system going. And so on.

But there is also a hint of the notion that we won't be able to keep the business going unless we do something special to the model (process) that makes it sustainable. We need to change something about education in order that it becomes sustainable. Our energy systems and economy need to be altered so as to be sustainable long into the future. And that, it seems to me, is where things get pretty fuzzy.

I'd suggest we take the systems perspective on this issue. In this approach we view that which is to be sustained as a process. It has boundaries through which inputs and outputs travel. It gets energy, materials, and information from its environment and pumps out heat and wastes and 'products'. In most cases the question resolves to: "can this process continue to pump out that product as long into the future as we wish?" From this perspective it is clear that three things need to be settled before an answer can be gotten. First, the inputs are resources and we need to know if there are essentially endless supplies of these. Second, are the internal workings of the process such that the ratio of product to waste (and energy to heat) is maintainable for all time? Are there possibly flaws in the inner workings that will cause a steady degradation of output over time? Third, will the environment continue to carry off the product(s) and wastes so that they do not build up and clog the flow through?

This may seem like an oversimplification of the whole problem, but it is only a matter of accounting and measuring in reasonable units of time. Every process has these characteristics even if there are millions of inputs and outputs; even if the inner workings are mind bogglingly complex and stochastic. All of that may make the question harder to answer, but it does not alter the fact that an answer is possible. And, necessary.

Several things can impact the long-term sustainability of a process. If the resources are finite then the draw down of them clearly makes the process unsustainable. In some cases, alternative inputs might be substituted, but if they too are finite then this just delays the inevitable. If the process is one that grows, then several things can go wrong. The growth itself may involve slight errors in replicating key sub-processes causing the internal workings to go awry. If the rate of influx of even an infinite resource is fixed then at some point the growth will outstrip the resource input and the system will neither be able to continue to grow, nor will it be able to have any kind of margin of error, so to speak, should something go wrong. A related problem is if the growth of a process means an increase in product/wastes output relative to the absorbing rate of the environment then the process can be choked in its own output.

If we take this systems approach to characterizing the meaning of sustainability then it becomes clear that claims of sustainable this or sustainable that need to be verified by demonstrating that all of the factors above are met. This will take research, careful observation, careful measurements, and careful use of language! Of course the bandwagon is rolling already. The word, like the word "green" has taken on a life of its own devoid of any thoughtfulness and so it is unlikely that the powers that be will insist on some kind of standard usage that includes validation of claims made. Sustainability is a fad, so the likelihood that it will be a terribly useful guide to policy decisions is rather small. Just label something as sustainable and everyone will applaud.

Of course, in the really long perspective nothing is sustainable! If sustainable means keeping something the same forever (or a really long time) it is actually an unachievable status in the real world. Everything is changing and evolving. I mentioned that life has sustained itself through the trick of capturing more energy than it needs to just maintain. That is how it grows. But the life on Earth today is vastly different from life a half billion years ago. What has endured is the energy flow through driving the internal workings. Those, in turn have been subject to copy errors and because all processes (cells, for example) are embedded in complicated environments that put stresses on them, they are subject to selection for those errors that happily confer some advantage. Life is sustained as long as energy flows, but species, and higher taxonomic categories as well, are not sustainable. Indeed they should not be sustainable in the very long run.

Our current social systems, our economic and political models, even our organizational forms and family units, are presently undergoing huge stresses from an environment we ironically had a great deal to do with creating. Nothing that we have or are is truly sustainable. Everything, including us, will change or go extinct in the long run. And the long run may not be as long as most of us think.

We cannot sustain our current version of social order and economic activity. We look for ways to fix the inherent flaws (like increasing regulation over capital markets) to make what we do now sustainable. But it is largely a vain effort. We are all inherently conservatives who want the world we have known to stay the same. We want to go on building so-called wealth and having kids to give it to. We want everything to be as it was for our lives. We believe our experiences were valid and should be sustained for all time. But we are wrong.

We should not be looking to sustain what we have, just for the sake of keeping it going. We should look for ways to stabilize the parts that work, to buy time and keep the change from being overly destructive. But we need to embrace evolution. We need to let go of institutions that are dilapidated, that are actually causing us more pain than good. What we need to do is look at our processes as described above and make wise decisions about resource flows based on what is and is not renewable. We need to evaluate the internal workings, our economic models and our population size, relative to both the resource issues and the piling up of wastes at our feet and in our water.

Life abides. It is sustainable. Processes change. We should recognize this and live by it. We should not believe that we are going to keep everything exactly the same for the indefinite future. That doesn't mean it is OK to be a destructive force or cause damaging changes (as with our over production of CO₂). But it does mean we have to think through our institutions, our processes and our values guided by nature's design and be able to change those as seems wise, rather than slavishly follow some vague concept called sustainability out of a foolish desire to preserve a status quo. Right now people are wasting valuable time trying to figure out how we can generate liquid fuels out of vegetation in order to keep driving our cars. They talk about sustainable fuel production trying to make the idea of maintaining our lifestyles seem reasonable. They want to build giant solar arrays that will rob the Ecos of some of its fair share of light so that we can maintain our consumption of electricity (indeed without addressing the population growth this is more than just maintenance). Human life, in this mode of thinking, is not sustainable. And if we persist, neither might be a livable planet.

Boundary Conditions

There is considerable effort underway to figure out what is going on in the economy. Everybody, rightly, wants to understand why the economy is tanking and why the financial system is out of control. And many economics analysts are working assiduously to answer these questions.

Unfortunately most of them will not be able to see the forest for the bark on the particular tree that they happen to specialize in. Understanding what is happening to the human economy is actually fundamentally simple. We need only look at the ultimate boundary conditions of the human economy to grasp what is happening.

The boundary of the human economy (as distinct from the whole Ecos) is comprised of the interface between the economy and the sources of energy and material and the sinks for wastes. Figure 1 shows the whole economy and this interface.

Figure 1. The boundary conditions for the human economy. Energy influxes from real-time and fossil sunlight drive the economic engine. Energy is needed to obtain usable energy as well as usable materials. Both the material resources and the fossil fuel resources are finite. Some material may be recycled internally but energy cannot. All energy influx eventually exits the system as waste heat. Much of the material similarly exits the system as waste (some being toxic). The major elements of the economic engine are shown internal to the boundary.

What is instructive in this viewpoint is that everything that can happen in terms of work and the production of human-defined wealth is limited by the flow of energy and its availability to do human-defined useful work. The system is limited, no matter how efficiently energy and materials are used, by the total flow rate of energy. As the figure suggests, the influx of real-time solar energy (driving photosynthesis, the hydrological cycle—hydroelectric power, and the atmospheric and oceanic dynamics—wind and currents) is approximately stationary over time. Hence, while we may be able to increase some of the capacity to convert raw sunlight into useful form (e.g. photovoltaics, wind turbines) the scaling up of this approach is problematic.

Energy obtained from fossil sunlight is many times more potent than what can be gotten from direct conversion of real-time sunlight. It is concentrated. Unfortunately it is also a finite resource. The evidence has been mounting that shows that we have reached a peak of feasible production of conventional oil. Thus energy gain from oil reserves (the bank) will begin to decline. Compounding the problem is that it takes energy investment to obtain energy returns. The only returns that count are the net energy available to do useful work. As the finite resource diminishes it also gets harder to extract — it takes more energy invested to get the same unit of energy return. This precludes growth in energy available meaning that less and less work will be done per unit time in the future. Less work, less wealth.

As long as the human economy could extract energy from the fossil fuel bank in ever increasing amounts, year over year, all of the internal subsystems in the economy, the machines, the structures, the photosynthesis (agriculture and forest products), the solar power (hydroelectric dams), and even the biomass of human and animal labor could grow and develop. Similarly, the institutions, the control hierarchy, and other social systems could expand to meet the needs of maintaining organization within the economy. But as we enter an era of contracting supplies of net energy available to do all that work of expanding the economy the economy itself will have to contract. Subsystems within the economy, those listed and in the figure, will need to contract in proportion to the whole and variously relative to one another in order to maintain some kind of orderly process. That will depend on how rational and wise we humans are going to be in approaching this condition.

I have already written about one formula for achieving this end. That would include a serious reduction in producing those machines and structures that serve only luxury and aggrandizement functions along with increased attention to increasing efficiencies in both of those areas. In addition we will need to restructure our institutions to be both more efficient and more effective in managing our resources. We will need to channel more of our energy saved from these efforts into the construction of solar power conversion equipment (augmented I suspect by nuclear power) and the better management of photosynthesis (e.g. more efficient use of water). On that last point, this does NOT include using photosynthesis in a vain attempt to keep our transportation fuels at a level necessary to support the unlimited personal transportation system, which, in turn, supports the unsustainable living arrangement known as suburbia.

And that brings me to the 'labor' subsystem. Of course I mean the population of people (and farm animals). Contraction of energy implies contraction of the population as well as contraction of the material consumption standard per capita. Trying to maintain the current population size while the machines, agriculture, structures, and institutions (which supply jobs) contract means that every individual will become poorer over time. Unless we address the size and scaling issues of population along with the other components, and set about judicious (and ethical) population reduction measures we will simply be inviting social strife and upheaval as people perceive their losses and get angry.

An economy that is in contraction is said to be in depression. We've had ample examples of uncontrolled contraction throughout history. The big difference between these previous cyclical depressions (and recessions) and our current situation has been that previously there was always more useable energy to be gotten easily. Eventually, when investment and consumer confidence started to build, the energy to do the work was there for the taking. The current situation is fundamentally different. Net energy for useful work is starting to decline meaning that less work can be accomplished. While in an era of profligate energy increasing we could readily expand and invest in development. The economy could generally grow. We could, in effect, borrow against expected future energy increases to build stuff that we didn't really need (but wanted) and did not see that the bank would eventually run out.

Now it is time to pay the piper. With each unit of energy from whatever source requiring more energy to obtain with each passing year, our net useable energy is headed downward. That alone forces us into contraction. Since we are so used to thinking in terms of progress and growth, it will be hard to mentally accept this completely new situation. We could easily fall into despair and view our condition as something like a permanent depression. But it doesn't have to play out that way. We could look upon this as an opportunity. We already know that our economic activities, especially the burning of fossil fuels, is threatening our world. Global warming, declining soils and water tables, waste chemicals that are toxic and have unknown long-term consequences, and many more human-caused problems are upon us. We are seriously considering the collapse of civilization on a global scale, with nowhere else to escape. So a forced contraction of economic activity would seem to be a good thing for the entire Ecos, including ourselves. We need not be depressed. The contraction could be managed if we are willing to take the necessary actions and act wisely. Part of that wisdom includes acting in a timely manner. The global threats seem to be accelerating so time is of the essence.

There will be no increase in the influx of solar energy to the earth. There will be a continuing diminishment of energy from fossil solar energy as the oil, coal, and gas supplies are depleted. We may be able to increase our supply of electricity marginally with increases in solar conversion capital and infrastructure, but doing so will be tricky if we are to avoid robbing the rest of the Ecos of its fair share. Until and unless we can harness nuclear fusion, it is looking more and more like our future will be one of contraction until we find the right steady-state equilibrium point where our energy uses exactly match our renewable supplies. Only then will we have a long-term sustainable economy.

Humans are not guilty of sinning just because we've had a helluva party. We have been doing exactly what any living thing would do under the same circumstances. We've expanded to fill the world with minds and gadgets and, most importantly, knowledge. We are biological after all. Life is mandated to grow and complexify. There is no need to feel guilt or sorrow about the mistakes we've made in judgment, so long as we have learned from those mistakes and are now ready to act like adults, take responsibility, and do what is necessary to restore balance between ourselves and our world. The world of tomorrow will not be the same as it has been for our history. We have already wrought irreversible changes that will require us to adapt. We have already altered the biota and we don't know what the impact will ultimately be for life on earth. But one thing we do know is that such changes have happened before on this planet. And each time life has emerged with some new, more complex and organized structure. I suspect that absent a planet killing meteor or catastrophic solar event evolution will continue in this pattern. There will be a better tomorrow.

Meanwhile we need to suck it up and get to work preserving the worthwhile and learning to live within our means (energy flow). I don't mind telling you I have my concerns. My work on sapience suggests that wisdom is in very short supply on this planet. We may not be able to muster the judgment and moral reasoning needed to make good decisions and get the majority to recognize them as such. In the end we as a species may just be too selfish and self-serving to do the right things. In which case, that better tomorrow will be without human representation.

The Science of Systems 7

Systems Science 7 — Cybernetics: The Science of Control

In the prior post I asserted that what exists are systems, or unities, entities, networks, comprised of aggregates of components. Over time, and under the influence of energy flow, these components tend to self-organize into structures and processes in which energy flows in and dissipates out while doing work to maintain the internal organization. Living systems are the epitome of this system-as-process ontology.

One might well ask how a process, a dynamic structure, can maintain its integrity over time, especially when energy flows might vacillate (such as in the diurnal cycle) and material flows, for resources, are uneven. A system becomes long-term steady-state or stable when part of the internal organization acts to control the overall function of the process, to keep it doing what it should be doing in order to remain stable and adapted to the external conditions.

Cybernetics is the science of control of systems, processes, such that they continue to function in consonance with the environment in which they evolved. Once a network of functional relationships has been evolved, and under conditions of reasonably stable (if slowly varying) energy flow, a control subsystem can act to maintain functional stability over the long haul.

Within-process control is achieved by the use of the feedback of messages to a control actuator that can have causal influence over some key elements of the internal system network, usually through amplification, such that the system as a whole changes its internal flow rates and/or configurations in order to maintain stability. Once evolved, a process' internal network represents the expected state of affairs, under normal operating conditions. When, however, conditions change enough to make a difference in the overall performance, relative to that standard, then the messages being fed back to the system are informational. That means the system has to change something in order to get back into stable operation.

Feedback and Closed-loop Control

Remember that one way to define information is "...news of difference that makes a difference." That is exactly the way it is used to manage processes. In a simple cybernetic system a transducer measures some quality parameter of the process output (see Fig. 1). The resulting message signal is compared with a reference standard or 'set point' value for the parameter. Whatever difference exists, positive or negative, between the set point and the actual real-time value constitutes a signal of error. In other words, the resulting message tells the system news of difference from the expected, zero error. To complete the loop and provide control, a controller must act, as noted above through amplification, to counter the error.

Figure 1. In this view an entity is comprised of a work process that converts inputs into outputs, one of which is considered 'valuable' (possibly by another entity) or represents the long-term health of the process. The stability of the process is accomplished by measuring the output value and comparing it to the value of the desired or target value (also called a set point value). Any error in the actual measured value is fed back through a controller, a subsystem that decides on a counter action that is initiated through some kind of amplifier.

The core adaptivity of a living system is actually its ability to control the internal structure of its basic work processes (e.g. physiology) through homeostasis. This is achieved by feedback control as shown above.

Coordination Control

Figure 2, below, shows a more complex situation where three entities are organized such that the product outputs from two of them are resource inputs to the third. The latter uses these inputs to produce a 'final' product output. All three entities need to cooperate in order that the final product is of optimal quality. Internally, cooperation is aided by a new layer of control using essentially the same principle of feedback control but with a twist. A new kind of controller, a coordinator, is introduced. This controller uses error information from all three entities along with a general model of how to coordinate the behaviors of the three to achieve the minimum error from the final product.

I am not introducing a mysterious object here. The coordinator controller actually developed from another entity that simply specialized in the task of processing messages — a kind of computer, in essence (see below for discussion of specialization). Messages, remember, are just very low power flows of energy or energy and small bits of matter. They are modulated into a communications channel between two entities and convey information when the receiver has a low expectation of the precise message form received. The signals (thin arrows) in Fig. 1 represent transduced messages. The error signal provides the controller with information when there IS an error.

Figure 2. Coordination between multiple entities by a higher level controller, a coordinator. This controller uses a model of the optimal performance of all three entities to generate control signals that (as shown) can modify the target values of each entity in such a way as to force the individual controllers to adjust their entity's process.

The trick, as shown in the figure, is to adjust the target values slightly in order to take advantage of the existing individual controllers working on their own process. This works by changing the error signals and prompting the controllers to adjust. Of course this scheme only works if the target value can fall within a range of acceptable values (see reference to homeostasis above). It also depends on a reasonable amount of stability (predictability) in the initial input resources. There must exist a 'feasible' model solution to variations in these inputs that will maintain an acceptable range of output values.

The model used by the coordinator fills a similar role as the targets did for the individual controllers. And, just as the targets were 'given', the model is pre-established and essentially fixed. As the complexity of the network (number of cooperating entities) grows the model given must be more complex as well (sometimes even more complex than imagined). Simple networks such as depicted in Fig. 2 are generally quite brittle. That is there are few feasible solutions to optimize the output for wider ranges of distortions in the inputs. Such systems would be completely at the mercy of the environment with minimal capacity to adapt to these variations.

Note that the system is portrayed as inputs coming in from the left and products and wastes exiting on the right. While this might seem stylistic, in fact all systems can be organized as networks with this general arrangement. We can see, in this arrangement, that opportunities for increasing complexity, adding more entities, exist in terms of increasing breadth (appending new entities in parallel with existing ones) or increasing length (appending new entities at the input or output ends). Several advantages accrue with both of these possibilities. For one thing, it is possible to introduce redundancy. Suppose we simply replicate one of the input entities and add it below one of the existing input entities routing its product to the output entity as a backup input. Now if its companion input entity should have difficulties obtaining acceptable resources, perhaps the new one can make up the difference. See Fig. 3 below.

Figure 3. A system can become more complex by adding entities to the breadth. In this example an input entity has been simply 'copied' and used as a redundant or additional input processor. The coordinator must be extended to handle the new addition, which will change and complexify the model.

The system can also extend or lengthen by virtue either of copying something (either input or output) as in the broadening example, or by capturing an entity that was previously independent that produces one of the inputs, say, that the system needs (Figure 4). Of course capturing a new entity on the output side can effectively change the nature of the whole system.

In both broadening and lengthening a system it is necessary to extend the coordinator and its model to cover the new functionality.

Figure 4. In this example the system is lengthened by 'capturing' a previously independent entity that produces one of the critical inputs to the system, thus helping to ensure stability of that input. As above, the coordinator has to be modified to incorporate this new configuration into the model and control processing.

The living systems version of lengthening is exemplified by the endosymbiotic theory, which, for example, provides an explanation of how eukaryotic cells emerged by symbiosis among bacteria-like simple cells.

Imagine that each entity here represents some relatively simple bit of metabolic activity, say, for example, early bits of what we know now as the Kreb cycle. Various bits might have evolved chemically semi-independently. They would have been loosely coupled in the primordial soup, perhaps each with their own unique primitive membrane acting as a structural substrate. At some point the now ubiquitous phospholipid bipolar membrane may have allowed the uniting of these bits and pieces producing a tighter coupling and allowing the cycle to form and evolve.

Coordination control involves two basic types of control processes. One is the coordination of the entity with its environment, that is, its sources of inputs and output sinks. In the example above of lengthening the input chain by 'capturing' a source, by establishing a tighter coupling between the source entity and the coordination control of the main entity, the entity is changing the ways in which it acquires resources. This is a tactical move to ensure resource availability. Tactical control involves maneuvering with respect to the environmental entities so as to ensure resource availability (in a timely manner) and to ensure products and wastes are appropriately transferred to the environment. This latter is every bit as important as ensuring resources since the buildup, even of products but especially wastes, can be as detrimental to the entity as loss of resources might be.

The other form of coordination control that works in tandem with tactical control is logistics. Logistic control involves internal routing of materials and energies to ensure that each operating unit is adequately supplied at the right time and wastes are eliminated. In a sense, logistical control takes over the management of what had been (in earlier versions of the entity) the effective tactical control employed by each operational unit when they were not as tightly coupled. The logistical controller is the mechanism that promotes the tighter coupling and relieves the operational units from needing to supply as much energy to independent tactical control.

Both tactical and logistical controls operate over somewhat longer time spans than operational or real-time control. That is they tend to aggregate error data over time and use time-averaged values for computation of their control outputs to the operational units. This is primarily because it takes time for trends to evolve that would mandate control decisions. Acting too quickly to short timescale errors could have a destabilizing effect on the operational units.

In living systems, especially cells, very complex networks of sub-processes require intricate coordination controls for both getting food and other resources (tactics) and maintaining internal flows. Some operational-level units are tasked with repairing other units that tend to degrade due to thermal effects (entropy). They require energy and raw materials to maintain the integrity of the entire entity. There are also operational-level processes that construct other processes (another example of autocatalysis when one process constructs another process that, in turn, constructs new copies of the first!). The logistical control of such complexity is substantial, involving distributed, numerous feedback loops. The entire workings have been captured in the concept of autopoiesis conceived by biologists Humberto Maturana and Francisco Varela.

Planning Control

The highest level of control in the hierarchical scheme is planning. This kind of control requires that entities have the capacity to construct internal models that are changeable to reflect changing conditions. Derived evolutionarily from the model structures used in coordination control, planning models involve not only a model of internal process, but also a model (or models) of the external world. That is, the entity must be able to represent the relevant parts (other entities and physical conditions) of the environment and have a way to represent changes in those as a function of time. In deed, in order to make plans, for the future, an entity must be able to anticipate how the environment might change. The planning then involves taking anticipatory actions that will increase the likelihood of the entity thriving or avoiding damage.

Of course we are now fully in the realm of living beings. And I will restrict the discussion to animals since whatever 'planning' might be done by plants is rudimentary at best. Animals have achieved a high form of agency. That is, they are able to produce effects on the environment as much as be affected by it. They have varying degrees of autonomy in making decisions to act. Some decisions are purely reactive, based on evolutionarily hardwired responses. Others are more flexible in terms of taking more situational conditions into account before taking an action decision. Still others, in animals with more evolved nervous systems, take far more conditions into account, including historically learned propensities/exigencies, and internally generated motivations to deliberate prior to taking a decision. Humans, of course, seem to have reached the epitome of such decision processing.

When there is a dedicated, distinct third level of control in the hierarchy specifically for planning the long term behavior of the entity we call it strategic. Strategic means that the plans take into account the likely scenarios of what will happen in the environment, how other entities will behave and what conditions will prevail at some temporal horizon, scenarios of what will happen internally based on adaptive competencies, resources stored, possible weaknesses, etc, and the panoply of motivations, desires, requirements, and such. The strategic planning process is to take all of this into account and develop a program of goals to be used by all of the various tactical and logistical controllers.

Devilish Complexity

Complex organization (like a living entity) involves elaborate, intricate networks of many kinds of components interacting in multiple ways to produce a whole, temporally stable, system. And as I have pointed out in previous installments, systems are really systems of subsystems or networks of networks (of networks ...). The three level hierarchical control model described above can be applied to many subsystem levels within a very complex entity. That is, for example, all cells in the body have fairly elaborate coordination controls, as discussed above, but also some very primitive planning control! Every cell devotes some of its internal processing to anticipating changes in its environment in the form of adaptive responses with memory. Consider muscle cells, for example. They will adapt to frequent and persistent demand for twitching by 'bulking up'. When a person works out routinely, their muscle strength and endurance increases over what it would be for a couch potato. Moreover, the muscles retain a readiness to react to demands based on this training for some time into the future. In fact, even if one stops working out the muscles will remain at the ready for quite a while. Only after several weeks or months of failing to work out will they start to loose their readiness.

Living tissues have a built-in ability to retain a 'trace' of capacity to respond to environmental conditions. When put under demand, the tissues (cells) build up the capacity under a schedule (essentially a logistic curve). This is a primitive form of 'learning' and acts to anticipate future demand. So, in that sense, it is a primitive version of planning for the future.

Thus it can be seen that every complex subsystem of a larger, more complex system, can have its own hierarchical control structure. Perhaps a much easier to visualize example of this is a typical large business organization. Such organizations have different departments, like production, marketing, accounting, etc. Take accounting as an example subsystem. Within the accounting department, which functions as one of the main tactical (financial accounting) and logistical (managerial accounting) control systems for the entire organization there will be operational sub-units (e.g. bookkeepers), logistical sub-units (middle managers over bookkeepers), tactical sub-units (supplies requisitioners), and strategic planners (controllers, CFOs). And every department in the organization has a similar (self-similar) internal structure. Even the production department has its chief operating officer who does strategic planning for the production facilities.

With this kind of devilish complexity is it any wonder that individuals, who frequently are called upon to make strategic, tactical, logistical, and operational decisions both for their organizations but also for themselves, are too often caught in being confused about what kind of decision they should be making at what time? You see this confusion (and its aftermath) all the time in typical organizations. In general most people don't even explicitly understand what differentiates the various kinds of decisions and the timeframe over which they should be made (or what information goes into formulating a decision). This is one of those areas that I strongly feel systems science could positively affect. If people understood the differences in each of the types of control decisions they would be better prepared to make appropriate decisions. I have personally been involved in strategic planning committees where the participants too often strayed away from thinking about strategic issues and got into logistics or tactics without realizing that they were doing so. It is hard to make distinctions, especially when all subsystems involve some amount of all three kinds of control.

And, of course, there is the problem with governance in general. Governments are hierarchical control structures (see my blogs on Sapience, esp. page 2 where I ran a series on Sapient Governance discussing this). Most of the world democracies have achieved some amount of differentiation into the various controls. But too often confusion reigns. I would argue that even in the US, for example, the notion of truly strategic planning (foreign policies, environment, energy) is sorely lacking or very weak at best. In part this is because the political system itself gets in the way of doing meaningful strategic planning (the politicians need to be more mindful of short-term issues that will get them re-elected). But it is also due to both politicos and citizens failing to adequately distinguish the kinds of decisions and time scales involved.

Perhaps a better educated citizenry, with respect to at least this part of systems science, would help move us toward better articulated decisions that address all of the relevant control aspects of modern civilization. It certainly couldn't hurt.

The Science of Systems 6

Systems Science 6 — Networks, Structure, and Function

Now that we have some idea of what exists in the universe and how the stuffs interact with one another we can take a closer look at how these stuffs form systems, how systems give rise to meta-systems and new levels of organization. In short, we can begin to understand how our world came to be from the interplay of matter, energy, information, and knowledge.

Networks

The Universe is a

Literally, all matter and energy in the Universe form a weave of networks within networks. Aggregations of matter form nodes within a larger network. The connections between nodes, formed from energies, are the pathways along which information travels. And the inner network of components forming the aggregate hold a structure that represents the knowledge of that node.

Energy, and perhaps a bit of matter now and then, flow through the larger system from subsystem to subsystem via sparse links between those subsystems. Energy flows into and out of subsystems affecting the internal organizations of those subsystems depending on the constituents of each. We have seen that the pre-existing rules of interaction between components determine the kinds of self-organizing structures that obtain from this energy flow. At the simplest, and lowest level of organization, energy flow largely acts to jostle components about (raises their thermal modes of energy) so that they have opportunities to interact with larger numbers of other components. Some interactions result in tighter coupling between components and a stable organization. Such configurations are energetically favored and persist, possibly giving rise to new possibilities for coupling with additional components.

Every once in a while, a configuration may emerge that autocatalyzes, that is, it promotes the development of more of itself. In a sense it becomes self-replicating, competing with non-autocatalytic configurations that would otherwise use the same components.

Such an event is important for the further evolution of organization within the larger system. Given sufficiently many degrees of freedom (number of kinds of components, number of kinds of interactions of components, etc.) and energy flow to move things around, systems can undergo self organization. This pattern is evident in every level of organization in nature. It holds for the Earth as a whole system with sunlight providing the input of energy and deep space acting as the sink for heat radiation. Biological evolution is a special case of the organization and reorganization of complex sub-networks. Living systems from bacteria to humans are, themselves, networks of networks, organized to use energy to self replicate. Fortunately the sun has supplied an adequate flow of energy to drive this process through the development of humanity. Our societies are networks of networks of people and food stuffs and artifacts through which energy flows.

Figure 1. Systems are networks of networks. Here three sub-nets with tight coupling between multiple components also have weak coupling between them. The tightly coupled sub-nets form entities that are components at a higher level of organization. This pattern of networks of components forming higher order nets is found at all levels of organization in nature.

Structures

As mentioned, a network of tightly coupled components tends to settle into a few 'comfortable' configurations. Comfortable means that they are all energetically stable over an extended time. Some configurations are so comfortable that they take more energy to disrupt than they would give up if the couplings were broken. Others can be broken more easily and tend to relinquish larger amounts of energy as the configuration falls apart. In the chemistry world this is known as endothermic and exothermic bonds, respectively.

The way networks of components form and create whole structures is largely an historical process. That is, the specific compendium of components (numbers of kinds incorporated and numbers of interactions possible) and the ordering in which they are brought into the mix determines the specific form and function of the structure. Depending then on the history of any specific sub-net, what components were in its immediate vicinity as it condensed, etc. these network/entities will have differing details. Some may be at the low end of complexity and have very few, rigidly determined forms, such as molecules of gases or simple hydrocarbons. More complex entities can take on high degrees of variability based on their history of formation, like DNA or RNA condensates (without templates to form on). Still other, even more complex entities, like the neural nets of brains, are highly variable as a result of the environments in which they form.

The more successful entities in the living world, however, have found the right mix of pre-determined structural form and developmental adaptiveness to variable conditions. The core form (e.g. morphology and physiology) is necessary for producing core behaviors (functions) that provide long-term stability to the entity. But there is enough room for variation due to chance encounters along the way with environmental factors to provide variation in the organization. Variations on a theme is the modus operandi of living systems.

A necessary condition for networks that form entities to maintain stability in a world of fluctuating energy flows is that they must be able to use the energy to maintain internal organization and then dissipate the waste heat and 'left over' energies. They pass the former to the environment via radiation and convection (where unattached components are free to absorb the energy as an elevated thermal mode). The latter may be transferred to another entity with a lower potential in that specific energy form and where a channel (a weak coupling) exists.

Adaptive Structures

When entities absorb energy or incorporate new material components they can undergo a conformational change in structure. For most simple entities this will result in disruption unless the original structure was particularly 'comfortable'. Such latter structures will simply dissipate the energy or kick out the new component(s). But for more complex entities another possibility exists. They can incorporate the material or internally channel the energy such that their structure is changed such that they again are able to dissipate waste heat. Such systems can grow and evolve in form. They become better suited to their environmental conditions.

A change of structure resulting from incorporating some new component(s) or adjusting to a new energy flow that persists over time is a memory of the events that drove the adaptation. The entity has 'learned' something about its environment. This is what it means to convert information (the NEW component or energy) into knowledge (new structural arrangement that does not degrade dissipation).

Living systems are, of course, the paradigmatic example of adaptive structures. Simple life, like bacteria and single celled eukaryotes, are minimally adaptive in that they have little behavioral control over their environment and must rely on various forms of taxes to find food or escape destructive environments. Life subsystems have to stay within a relatively narrow range of values for characteristic physical and chemical aspects of their environments, such as temperature, salinity, and pH (acidity). In the next post I will be exploring the methods of control (cybernetics) of these subsystems that further refines the roles of information and knowledge in managing the flows of energy and materials.

Multicellular systems, networks of cells, have increased degrees of freedom as well as built-in redundancies. Some of these cells can be altered by genetic mutation such that they form a new sub-network that performs a slightly different function. It specializes in some new capability that increases the fitness of the whole organism. The difference between somatic cells and germ cells may have been one of the earliest differentiations to take place in multicellular evolution.

In all cases living systems must continually import low entropy material and raw energy while exporting waste heat and high entropy material. A living structure has achieved a steady-state condition when the structure has achieved a stable flow through of material and energy while maintaining a a stable network of components internally. The attainment of such a state not only depends on low level operational control, but on multiple levels of control — hierarchical control systems — that organize the internal flows of energy and materials (logistics), the obtaining of those resources while avoiding threats (tactics), and overall planning of future behaviors based on the current and anticipated future state of the environment (strategic). Look for another expication of the importance of hierarchical control in a future post in this series.

Function

When a highly organized adaptive network is able to absorb material, incorporate energy to maintain the structure, and dissipate wastes, as a living system does, then it can be said to be functional. That is the outputs (wastes as far as the entity is concerned) are a function of the processing of the inputs by the network. Such a network could be, in principle, decomposed to find its sub-networks and each of these might be shown to have their own specific functionality with respect to channeling matter and energy fluxes internally. The function of the whole is a result of the functioning of all of its sub-networks (components) along with the overall structure coupling these sub-nets into the whole.

As it turns out frequently in the living world, the output wastes from one entity can become a resource for another entity, especially if the latter has an independent source of energy to synthesize those resources into lower entropy form (new networks). Thus a coupling is formed between entities as part of a yet larger web of organization (see figure below).

The notion of functions of entities or processes supports much of what we think of as organization. When each entity (strongly coupled network, component, process) in a larger network processes inputs to produce outputs that are either used by other entities in the network as inputs or are expelled into the environment, we observe them as performing useful functions.

Figure 2. Multiple cooperating processes each perform a function, turning inputs into outputs. Several processes (P1 and P2) receive inputs of energy, matter, and messages from the environment. They process these in some fashion passing their own outputs on to other processes as inputs to them. P3 is completely 'embedded' in this network. P4 and P5 produce outputs that are emitted to the environment. Note, messages can be passed back and forth between processes, hence the bi-directional arrows.

In the above figure these processes must produce exactly the kinds and quality of outputs needed by the other, downstream processes. This means there must be a stable set of inputs from the environment to begin with. Outputs to the environment (from P4 and P5) are just outputs. There is no implied obligation that this entity (all of the cooperating processes together) produces outputs needed by some other entity. However, should those outputs turn out to be useful to another entity then we have the basis for a higher yet level of organization.

This figure shows a stationary view of an entity. That is, even though materials and energies and messages flow through the entity, it remains the same over time. Sufficiently complex entities, operating in a more dynamic and non-stationary environment, where the inputs are not stable and, indeed, can vary in kind and quality over time, may be able to change their internal structure. Processes may be able to adapt to these changing conditions and still meet their obligations to the other processes. This, in fact, is what we mean by adaptation.

Also not shown is the case where some of the internal processes act as copiers of the structure itself. These processes take in matter and energy, having knowledge (e.g. DNA) of the entire structure of all processes, and use their inputs to replicate the entire entity, or enough of a seed such that the new entity can develop under its interactions with the same environment. Once again, life and beyond.

Analysis

That the entire universe can be seen as this network of networks (of networks, etc.) suggests some powerful tools for analyzing structure and function that might help us better understand. A basic mathematical tool is graph theory. A graph, in this sense, is just what we have been describing as a network. It is composed of nodes (components) and edges or links (couplings). Some of those links might be represented with arrows indicating a causal relation. An even more useful version of graph theory used to study the flows of stuffs through networks is called flow network theory. This is a form of directed graph where the arrows point in the direction of flows from node to node. This version can be used to study such things as traffic flows and network traffic in the Internet.

Another useful tool, one that can be used with graph theory, is systems dynamics modeling. A system represented as flows and stocks (couplings and components) can be simulated to discover aspects of its behavior that do not show up just by looking at a static diagram of the network.These tools are increasingly being used to explicate the nature of systems. An especially important analysis of the world system called Limits to Growth by Donella H. Meadows, Dennis L. Meadows, Jørgen Randers, and William W. Behrens III. This model demonstrated quite graphically that the Earth system stands in danger of disruption owing to the growth of human populations and consumption. The latter was made physically possible by the advent of easy to acquire fossil fuel energy. The human economy is a network of networks, etc. in which energy flow has allowed the construction of more complex forms of organization (our cultures). The model suggests that this system reaches a point of maladaptive restructuring in which the human built world cannot grow further without destroying itself and much of the natural environment.

It is clear that the analysis and understanding of complex networks is essential to our coming to grips with our place in the Ecos. This alone makes systems science a critical endeavor.

The Science of Systems 5

Systems Science 5 — Information and Knowledge

In the last post in this series, I introduced the notion of organization as a fundamental concept in systems science. A key question in science in general and in systems science in particular is where does organization come from? How does the world we see today, with life and all the complexities of climate, oceans, mountains, and mineral deposits within the crustal layer of the earth, come to be?

Ontology Revisited and Some Associated Metaphysics

What exists? In Systems Science 4 I indicated that at a fundamental level, we have energy, which in a concentrated form is matter. But this is only half of the story. In this installment I want to introduce another consideration of what constitutes reality. Not everyone is going to accept this view, it is still in the realm of philosophy more than in science. However, I will provide arguments that derive from our scientific understanding of what might seem like an ethereal substance but is, in fact, quite real. The other half of the story is the Information/Knowledge duality. It is related to the Energy/Matter duality in a very deep way, as I will explain.

In Figure 1 below we see a diagrammatic representation of the structure of reality at the most fundamental level. Having some fun with lexical play, I call this the Ontos, what exists in physical reality. By this I mean to distinguish between physical reality and an imagined, or claimed reality where other things like spirits and unicorns might exist. Imagined concepts certainly 'exist' in the Ontos, but as knowledge structures within heads of people and as symbolic representations (e.g. dragons flying about in a video game or movie). Someday I need to explore the idea that humans have achieved a level of imagination that creates sometimes impossible things and events (in the sense of physical embodiment) while grounded in physical reality with all of its inherent constraints (like gravity). What does it mean????

The center of the diagram shows the basic existence of what we might call the 'stuff' of the universe. There are four quadrants, energy, matter, information, and knowledge. Matter and energy are in the left two quadrants, collectively the 'Physical' side of reality. These are the stuffs that we can observe and of which we are made. On the right side, the two quadrants labeled Information and Knowledge are, in my terminology, called the 'Ethereal' side of reality. There is noting material about information or knowledge, yet these 'stuffs' exist and are very real.

The upper half of the quadrants represent the dynamic or 'Movement' aspect of reality. Energy and Information are the stuffs that travel. But they respectively drive Matter, through forces, and Knowledge, through influencing construction (below). The lower half represents the actual 'Structure' of reality, how it is organized materially. Note that there are several relations, with arrows in the diagram, that interrelate these four aspects. Information is transferred by the flow of energy (with or without matter, i.e. as in light). Knowledge is composed in matter, i.e. the arrangement of matter in a system is a representation of that system's knowledge (also below). The last relation is an upward causal one from the lower half, 'Structure' to the upper half, 'Movement'. Namely the actual arrangements of matter, and hence knowledge, channel or constrain the flow of energy and, hence, information. In other words, there is a causal feedback loop from Movement to Structure back to Movement that recurrently creates a generative force involving all four stuffs. More on this later.

All of these are observable at the human scale of reality. Indeed it is these stuffs and their relations which form the self-similar, recursive patterns discussed in my last systems science post. But the universe scales across many orders of magnitude, from the smallest (right side), where reality is dominated by quantum laws, to the largest (left side), where gravity rules and the bulk of the universe seems to be made up of dark energy and matter. I have whimsically called this larger framework of reality the 'Mysterium', partly because the two ends of these scales are only dimly understood by human science, and partly to give a nod to mysterianism, which holds that consciousness will never be understood by the very mind that experiences it. Indeed, if the latter philosophy holds up, they may never be fully understood! But the relationship between dark energy and dark matter and the physical side of human scale reality seems straightforward, even if mysterious [incidentally, dark energy may yet prove to be a misnomer or a result of misunderstanding in cosmology, so don't take this too seriously — I just like the symmetry!] On the small side quantum effects that may be the source of things like measurement uncertainty (e.g. Heisenberg's Uncertainty Principle) may turn out to be the ultimate source of information, which is, after all, a measure of improbability (below). Quantum weirdness may lie at the root of our inability to have perfect knowledge. Uncertainty and weirdness are thus related to our remains of ignorance.

Figure 1. Reality and what exists — the Ontos. Everything that exists can be categorized as energy, matter, information, or knowledge. However, all of these are related at a deep level such that all of the structure and movement of the Universe that man can observe involve these four 'stuffs'. The Mysterium represents the scale boundaries of human understanding, at least at present.

Just as with the duality relation between energy and matter (remember Einstein?) information and knowledge are, in a sense, interconvertable. That is, information can give rise to knowledge, and knowledge can give rise to information. Unlike matter/energy which is ultimately constrained by a conservation principle (neither created nor destroyed), the information/knowledge duo appears to expand owing to the positive feedback nature of the information → knowledge → information loop. My suspicion is that the expansion is bounded ultimately by the rate of energy flow, which, when the universe comes to maximum entropy — all the stars are burned out — will bring the loop to a halt.

To explain the I/K duality I will have to treat each separately first but then delve into the interrelationship since that is the essence of this thesis. There are, however, a few additional principles to recognize as they will play into the descriptions below.

The Principle of Flow

The first principle to recognize is that of flow, or moving from one position in spacetime to a different position. In material/energy systems this is achieved when there is a region of spacetime in which there is a concentration of material/energy and another region in which there is not. The former is called a high potential region, with respect to the material/energy, the latter is called a low potential region. The rule is very simple. Material/energy flows from the region of high potential (called a source) to the region of low potential (called a sink) at a rate that is proportional to the numeric difference in their respective concentrations and inversely proportional to any "resistance" to flow through the interceding region or channel. The force exerted, pushing material/energy from source to sink, diminishes as the concentration at the sink becomes equal to that at the source.

Of course this simple description leaves out a tremendous amount of detail, such as boundaries and boundary conditions, channels and their internal interactions with the flowing material/energy, etc. As I said earlier, this is not meant to be a physics text and so the reader who is not familiar with these concepts from physics will need to take some time to brush up on them. This principle, of flow, the basis for the Second Law of Thermodynamics, is essential to understanding the Ontos.

The Principle of Dissipation

Energy can flow through material structures, appropriately arranged. In other words it is possible to find some arrangement of matter (say a configuration of atoms in a molecule) that will absorb energy in some appropriate form, but quickly be given off without doing any substantial work on the matter, i.e., change the configuration in the long run. Molecules can absorb a photon of light, maybe an electron is raised to a higher orbital energy for a pico second, and then when the electron falls back to its original (lower energy) state, a photon is emitted. The molecule may have gone through a temporary configuration change but nothing substantive changed.

This is as opposed to situations where the input of energy results in some substantial physical (mechanical or chemical) work being done to change the configuration of the matter. Energy, for example, might be stored in chemical bonds creating new molecules. Depending on how thermally stable they are, they might continue in that configuration even as additional energy is input. Such systems are further from an equilibrium state and additional input of energy may not do any additional work, being dissipated as in the above paragraph.

Systems like this might be thought of as being 'pumped' up. They reach a point at which additional input has no effect on the structure. Any additional energy input simply 'maintains' the pumped up structure.

The Principle of Amplification

Doing work involves controlling the flow of energy through a system. It involves applying the correct amount of energy at the appropriate time. Amplifiers and their binary cousins, switches, are material devices through which energy can be made to flow under the influence of a second, usually lower-level or lower power, input of energy. The transistor is a paradigm example. A high-level voltage is applied to one end of a wire that is broken in the center of the device and the gap filled with a semi conducting material. The wire exits the other side and eventually terminates in a low-level voltage (ground) but under ordinary conditions, the energy flow (the current) cannot get through the gap so that no energy flows through and out the exit. A third wire is used to control the flow of energy. This wire can carry a low-energy signal to the gap where it's current can turn the semiconductor into a conductor. The result is that the high-energy flow can cross the gap and exit to ground. Thus, a small or weak signal can be used to control a large or strong signal. In an ideal amplifier, the amount of weak signal is proportionally represented in the amount of strong signal. This means that the strong signal essentially mimics the weak one in its ups and downs, but just has more umph (a technical term meaning strength or capacity to do work!).

Figure 2 shows a generalized representation of amplification. Figure 2A shows a device that is "off" in that no low-level energy is flowing so no high-level energy can flow either. In Figure 2B, the low-level energy flow is on, which then turns on the high-energy flow, in proportion to the amount of low-level energy. [Note: Transistors used in digital devices do not actually function as amplifiers. The low-energy input is actually at the same potential as the high-energy input. The device then acts as just a switch, it is either on or off, hence the use of the term "binary".]

Figure 2. An amplifier is any physical device in which a small flux of energy (thin arrow) is used to modulate a large flux of energy (thick arrow). Application of a low-level energy flow through the thin arrow (B) opens the pathway for the high-level energy flow through the amplifying device. Such devices can be used to boost the signal of a low-level energy flow, as when an audio amplifier converts a weak radio signal into a signal strong enough to drive a speaker coil. Or it can be used to switch a flow on or off, as in the case of transistors used in digital circuits.

Amplification is what makes work potentially useful. We will revisit this principle again in the general theory of the tool. But for now let us just note that amplification is the phenomenon that makes what follows make sense. Information is conveyed in energy flows. The control of an energy flow makes it possible to encode information in the form of fluctuations in that flow. An information process is one that modulates the flow of energy in a way that results in the accomplishment of useful work. And useful work results in the modification of a physical system — in the configuration of material structures. This latter, we will argue below, is the basis of what we call knowledge.

The Principle of Transduction

If amplification is the modulation of a high-level energy signal by means of a low-level signal, then transduction is essentially its opposite. That is, transduction is the process of modulating a low-level energy signal in proportion to a high-level signal. This is the means by which sensors of various kinds produce signals related to the physical forces they respond to. A sensor is any device that converts a large input signal into a small output signal. As a general rule the input signal is due to an energy flow in one form, say mechanical, and the output signal is in another form, say electrical.

In most respects, transduction is physically similar to amplification. It is just one energy flow providing a sufficient amount of free energy to the physical device to do some work resulting in the outflow of another flow of energy. In some devices there doesn't even need to be a distinction between high-level and low-level signals, as in a simple switching device. All that is necessary is that some of the energy inflow is available to do the work of modulating the resulting signal outflow. Nevertheless, it is instructive to consider amplification and transduction as essential physical processes in order to understand the basis for the "ethereal" stuffs. Information and knowledge are dependent on the ability of systems to encode messages into communications channels and extract those messages at the receiving end.

The Principle of Buffering (Storage)

There are special devices that transduce energy flows and amplify energy flows. Energy flow is across space and time. Often time it involves a flow of matter as well. Because physical channels have finite capacities, in order to have anything like coordination between the outflow from one system to the inflow of another there needs to be some intermediary device that can temporarily store accumulations of the energy (and matter). If, for example, a receiving system is not ready to accept an inflow of energy there needs to be some intermediary buffer that can hold the energy until the receiver is ready.

The buffer is, of necessity, at a lower potential than the source of the flow, so acts as a finite sink that, as it fills up, comes to the same potential level as the source. Hence it will act as a source to the ultimate destination sink — the receiver system.

The Principle of the Quantum

One final observation about the physical stuff is in order. I believe the evidence is sufficiently strong to suggest that matter-energy structures are, at base, quantized. That is at the smallest scales of matter and energy these come in discrete packets or quanta. This certainly seems to be the case at Planck-scales of time and space. The importance of this principle is that when we look at a number of systems we see that what appears to be continuous at one scale is readily decomposed to discrete quantized steps at a smaller scale. The most obvious example of this is the digitization of analog signals that is used in high quality electronic systems such as DVDs. Any analog signal can be discretized and quantized, that is, it can be broken into equi-interval pieces with each piece given a fixed (often) integer value. This process is based on taking a periodic measurement of the analog signal at a rate that is rapid compared with the normal fluctuations in the signal. The transducer (sensor) described above can be made to do this interval sampling and is, in fact, the basis of digital electronics systems.

The importance of this dual nature of matter/energy — describable in both continuous and discrete quantal terms — is that we can treat these descriptions as essentially equal. If it turns out that, at bottom, Planck-scale, all things are discrete it doesn't matter. That scale is so infinitesimally small compared to processes of interest that everything is essentially continuous. This is just like the nature of water. It is composed of discrete molecules of H₂O. Yet at our ordinary scale of perception water appears to be a continuous fluid (at room temperature). We can describe it as such, but, if needs be, also choose a description based on its discrete nature. [As an aside, the latter is possible in principle but proves damned hard to do in practice since the scale of description of the discrete will always be much smaller than the scale of description of the continuous. We still are finding out things about the nature of water at the molecular level that surprise us!]

The Principle of Random Events

At a level of description that is continuous we recognize variations in the process that seem to deviate from the expected. That is, things take a turn that cannot be predicted from our knowledge of the behavior of the process, except in some statistical sense. If we were to be able to drill down to the underlying discrete substrate (the molecular level in water) we might find perfectly deterministic explanations for these seemingly random fluctuations. Such explanation for apparent randomness has been advanced both philosophically and in some instances of physics (e.g. chaos theory applied to nonlinear, dynamical systems). The important point is that for all real processes, described as continuous, there is some, however slight, random quality that makes perfect predictions impossible.

An example of this principle can be seen in random mutations, necessary for the generation of novelty in genetic material upon which natural selection can operate in biological evolution. Taken at the level of description of the population we can only provide a statistical picture of mutations and subsequent inheritance. We cannot predict what mutations will take place or when. At the molecular level there are causal sequences that lead up to the replacement, say, of a specific nucleotide in the gene. Because of the sheer complexity (number of variables) of the situation it is impossible, even in principle, to measure each and every particle, its position and velocity, that would be needed to explain the results. We need to be satisfied with our statistical descriptions based on a theory of probability that serves us well in the macrosphere. Hence, we treat randomicity as effectively real.

These are some of the important principles that appear to be operative at all levels of organization of reality. They describe the basis for the relationships that can be found at all levels between the four stuffs in our ontology. Now I can delve into information and knowledge as the second great duality of reality — the Ethereal realm.

Information

The problem with the term information is that it has been used for many purposes. It is so common that it is in danger of losing its most valuable kernel of meaning. I say this as a computer scientist who has witnessed its usage by fellow academics who play fast and loose with it. For example, how often have we heard the phrase " information management" only to find that what is really being described is the management of data? Yes, there is an important distinction to be made between information and data. But there are more distinctions still. In this section I will spend no small amount of effort to make those distinctions so that you will (I hope) understand how information is such a fundamental stuff.

In particular I will make a most important distinction between information and knowledge. It is amazing to me how often these two terms are interchanged. On the other hand, I suppose I shouldn't be too surprised. As I will explain, there is a deep connection between information and knowledge that can blur the distinction. Knowledge is a result of information acting on a physical system which is capable of undergoing a change in configuration. Thus, it is sometimes said that knowledge is embodied information. It is true, in some sense, but we should be very careful not to think that they are therefore the same. For example, it is wrong to say (in my Ontos!) that a teacher imparts knowledge to a student. As we will see, it is even wrong to say that a teacher provides a student with information - but that is a little trickier. It is the student who constructs knowledge (in the brain) as a result of the information she receives from the world, which could include a teacher.

What then is information? Gregory Bateson had an apt definition; "...information is news of difference that makes a difference". By this he meant that information is that which tells a receiver (also called an observer) that something is different from what the receiver expected and that is the basis for an active receiver to do something different than what it would have done otherwise. Claude Shannon developed the communications theory of information which comes very close to this idea of difference from expectation. He defined information in terms of a probability function. First he had to distinguish the roles of various players in the act of communicating information.

As already mentioned, you need a receiver or observer. This entity receives messages from a sender. A message is conveyed along a channel and is comprised of a coded configuration of matter propelled by energy. The channel is anything that acts as a medium for the message. So, for example, a message may be an organized set of voltage pulses in a copper wire. The wire is the channel and the organized pulses are pieces of the message called symbols. A famous example of a coded message process is the Morse code of telegraph fame.

Figure 3. A communications model. The definition of information depends on this model.

The symbols are thought of as being sent and received in a serial fashion with varying timing between. It is this temporal arrangement of symbols which constitutes the message. Before getting into the details of messages and their meaning, I want to provide a small example of information communication that will be useful for further development of these ideas and especially with the relationship between information and knowledge.

Let's suppose we have a system comprised of a sender, channel and receiver as in Figure 3. Let's further suppose that there are just three symbols possible for the sender to send and receiver to receive. In other words, both sender and receiver must a priori be "designed" to "handle" these symbols. I will explain more about the designed part and what it means to handle symbols shortly. For now please just accept this as a given. Also accept that for simplicity sake we will state that the symbol generation by the sender (that is the creation and insertion into the channel of a symbol) is clocked at discrete intervals.

Let the symbols be represented by the characters a, b and c. We would say that these are symbols in a fixed set called S, and represent this set so:

S = {a, b, c}

At each interval of time, Δt, the sender selects one of the symbols in the set and inserts it into the channel. At some time later, depending on the speed of transmission and length of the channel (details that won't concern us here), the symbol arrives at the receiver. Symbols arrive with the same Δt interval, just offset to some later time by the transmission delay. So at each interval, the receiver gets a symbol from the set S. The question Shannon asked (and answered) was: How much information is conveyed by each symbol received. To be fair, Shannon tackled much more complicated versions of this question, some of which we will go into later. But for now lets follow the reasoning and see what we mean by an "amount of information."

Shannon, and others who were interested in measuring the value of messages, reasoned that each symbol must convey an amount of information that is inversely proportional to the likelihood of its arrival. What this means is simply this: Information is a measure of surprise to the receiver! The more a receiver expects to get a specific message at a specific time, the less informed they are. You can relate this to your own common experience. If you don't know what to expect from observing some event, say who will win a Superbowl game, then the result of the event will inform you that one team was "better" than the other (I should caution you not to think of this as the same as the meaning of a message — we will cover that later). Even more to the point, if you did expect team A to win and team B did, then you would be surprised (and maybe a bit disappointed if you had bet money on A). The level of surprise is the measure of information.

In some ways this is not intuitive because we ordinarily attach the idea of meaning to the common use of information. But this is not what is meant by the measure of information. That measure, I, is a number between infinitesimally close to zero to infinity! Here is how Shannon set it up. If I is inversely proportional to the expectation of an event (the receipt of a specific symbol at a specific time), this means that the more probable the event is, the less information is conveyed. In math talk:

I ∝ P_t(x←{a, b, c})^-1

which, translated to English means that information is proportional to the inverse of the probability of the event being the arrival of one of the symbols from the symbol set. [OK, maybe that isn't regular English! So say, if a symbol is highly probable (expected) then it produces little information; if it is improbable, then it produces a lot of information.]

Here, the probability P_t(x) is defined by the likelihood of x being one of a, b or c at time t. And what is tremendously important to grasp is that the likelihood is based on the receiver's expectations.

What Shannon did, a stroke of genius, was to see that the relationship could not be a simple one. Probabilities range from zero (absolutely impossible) to one (absolutely certain). Information, intuitively, ranges from minuscule (I just knew that would happen) to humongous (wow, I didn't see that one coming!). Furthermore, it should not scale linearly because, again intuitively, a large change in probability when you are above a 50-50 chance should not convey that much more (or less) information as compared with a small change when the probability is near unlikely, which should tell you a lot.

A function that has these properties is the negative logarithmic function shown in Graph 1. Shannon chose the base 2 log function since it has the nice property of producing one unit of information for a probability of 0.5. This unit, Shannon named the "bit" for binary digit (or binary digit if you prefer!). It represents the amount of information that one gets from flipping a fair coin. A priori there is a 50-50 chance of getting either heads or tails — they are equiprobable. The flip results in one or the other, producing just one bit of information, which, if you think about it, isn't much.

Graph 1. Relation between probability and measure of information. This graph shows information as being equal to the negative logarithm (base 2) of the probability.

The bit works for binary choices — on-off, one-zero, yes-no, etc. As a result it is the basic unit of information used in computational engines. Numbers are represented in computer memories by arrays of binary values, the ones and zeros. A single memory cell is called a bit, and a string of bits is used to represent a base 2 number. We will have more to say later about computation and representation, but we will leave off talking about computer numbers for the moment since there is nothing more that can be said of binary choices that helps further understand information.

The real nature of information shows up in more complex decision situations. We can start to see this with our model communications system above. Remember that the system only had three symbols. A question we might ask is, how much information each symbol conveys when it arrives at the receiver? This question is motivated by a prior agreement as to the meaning of each symbol. For purposes of our example, and to keep it simple, lets say that the symbols have been assigned the following meanings.

a → BAD, HURT, PAIN!
b → NOTHING, NEUTRAL
c → GOOD, REWARD!

That is, the arrival of an a means either something bad is going to happen to the receiver at some time in the near future, or it could mean that something bad has happened to the sender at some time in the past — it all depends on what the designer intended the system to be used for. Similarly, the arrival of a c means something good. Now the information value of the message takes on some significance. Clearly if the receiver is getting a lot of bad symbols, it should be doing something with that information that would be different from what it might do if it were getting good symbols. And if it were getting the neutral b it wouldn't need to do anything. This may seem a little abstract right now, but you will see it come into play when I describe how the receipt of an informational message 'causes' changes in the receiver below.

Now lets start the system (start the clock ticking). What is the information value of each message? One way to think about this is that the receiver maintains an array of three probability values, one for each symbol type in the set. If we had to guess what any symbol's information value would be at the outset, we would guess they were equally probable. That is, initially we would assume that the P(a) = P(b) = P(c) = 0.3333... But there is no reason to assume these are the "real" probabilities, representing the long-term experience of the sender. Imagine that something really bad is happening to the sender who wants to convey this message to the receiver. Then the sender would start sending out a series of as. The receipt of the first a would produce an amount of information equal to the -log₂(0.333), which is about 1.586 bits. Not a spectacular amount of information. Not much of a surprise. The conclusion one should draw from this is, well, not much. We don't have enough information to determine if something bad is happening or not.

But now lets say the receiver has an ability to use the information received to modify its own probability array. We need a function that will relate the information received to a value that we can add to the current probability to get a new probability for the next occurrence. For example we could simply divide the information received by 10 (this assumes that if we were to receive 10 bits of information we would be so overloaded that the whole system would blow up!) and add that to the current value. This new probability would be the basis for computing the next informational value of the receipt of that symbol.

To see this work, say the probability for receiving an a starts out at 0.333. Now, for the next 15 clock ticks the receiver gets an a. This is telling the receiver that something bad is happening, but with each receipt the message is less and less surprising ("tell me something I didn't already know!"). That is, there is less and less information generated with each new receipt. Table 1, below, shows the schedule for probability and information with each message (symbol received) for 15 ticks. Notice how the probability climbs, but at a decreasing rate, and the actual information value declines. Less information results in less increase in the probability which leads to less information in the next iteration.

Tick	Probability	Information
0	0.333	1.586405918
1	0.491640592	1.02432406
2	0.594072998	0.751287879
3	0.669201786	0.5794868
4	0.727150466	0.45967417
5	0.773117883	0.371239686
6	0.810241851	0.303575489
7	0.8405994	0.250509668
8	0.865650367	0.208143652
9	0.886464732	0.17386486
10	0.903851218	0.145842783
11	0.918435496	0.122749693
12	0.930710466	0.103595665
13	0.941070032	0.087626006
14	0.949832633	0.074254772

Table 1. The change in probability resulting from a steady receipt of the same symbol. As the probability increases, the amount of information declines. The receiver comes to 'expect' that the next symbol to be received will be an a!

There is, of course one caveat. As the probability of the next received symbol being an a increases, we must decrease the probabilities of the other two symbols. This is just one of the rules of probability theory for mutually exclusive events. The sum of all of the probabilities of all of the symbols must be 1.0 and not a jot more (or less). So, if the probability of receipt of a goes up, because now we can see that with each such receipt we are coming to "expect" a, the probabilities of the other symbols must go down accordingly. I won't bother with the details of how this might be accomplished here. Suffice it to say that we need to proportionally reduce the other two probabilities by an amount 1.0 - P(a_new). In other words we take that difference and reduce the other two to portions of the difference so that the total sum still equals 1.0. The results would look like Table 2, representing the array of probabilities after the 15th tick from above.

P(a)	P(b)	P(c)	Total P
0.9498...	0.0251..	0.0251..	1.0000

Table 2. The memory array in the receiver after 15 ticks. The probabilities of b and c have declined

To carry this a step further, suppose that after the 15th tick the receiver gets a c. What does this mean? Since the probability of receiving a c is only 0.0251, the information generated is 5.316 bits. This is a lot! It means that the receipt of c was relatively unexpected. It's receipt was a big surprise. What should happen now?

By our procedure above we need to increase the new probability of c to 0.5891 and reduce the probabilities of a and b by amounts proportional to their current values and to 1 - P(c_new). Shortcutting the math the new array would look like:

P(a)	P(b)	P(c)	Total P
0.3902...	0.0206..	0.5891..	1.0000

Table 3. The memory array after the receipt of c (after the string of a s received previously). This single event was unexpected, resulting in a substantial shift in the probabilities of both a and c. The probability of b is barely affected.
[In real systems it is more likely that a radical shift like this is not the result of a single symbol receipt. More generally a more graded response is the result. That is, the receiving system will respond only minimally to the shift from a to c, treating this shift as potentially due to noise — a random event. If the receiver continues to get c after the initial one, then the receiver will respond more strongly. In other words, the system should employ filters that moderate the information encoding so that the receiver does not make such radical internal changes due to noise.]

If the receiver continues to get a symbols for several more ticks, then the probability of receiving a goes back up, just as it did with c. Any series of symbols will cause the values to rise and decline accordingly. It should be easy to see that if the symbols arrive in a random fashion, with a uniform distribution, then the probability values in the array will tend to equilibrate again to 0.333...

Over time, the values associated with each symbol will represent the receiver's expectations for what is happening in its world — the sender. As time goes on the receiver can expect bad things to happen, if mostly as are received. It could come to expect good things, if mostly cs are received. Or it could come to expect that things are neutral for the sender if mostly bs are received.

This example is very simple, yet you can see that even it has its complexities. The more symbols that a system has, the more complex the computation of probabilities becomes. Though the method used here is not mathematically rigorous, I took some unexplained shortcuts, it illustrates the basic concept of information leading to a change in the receiver that then results in a change in the receiver's expectations for future events. The fact is that the array of probabilities, in a very real sense, represents the receiver's knowledge of the sender's state. If you had to say what the general state of the sender was you could look at the array and make a prediction. And this is exactly what we mean by knowledge. We will take a closer look below.

There is yet one more point to make about information before moving on. We should note that there is a profound relationship between information and energy. It takes energy and work to send messages, so the whole system uses some energy in the act of communicating. It also takes energy and work to compute the new values of probabilities. The energy applied to the computation, in general, comes from other sources and the information input operates through amplification (or transduction) to make physical changes in the receiving systems (see Knowledge below). That is, information receipt — receiving an unexpected quantum of energy encoding a message, triggers work to be accomplished in the receiver such that a similar quantum received in the future will be more likely dissipated, producing less work to modify the system next time.

Actually, it takes energy just to maintain the storage of these values (as for example the electricity needed to maintain a memory cell in a computer). There is a very fundamental relationship between energy and information in the sense that there is always some energetic cost (work must be accomplished) to convert information into a change in configuration representing a change in probability for future similar messages. In another way of putting this, the entropy of a system is increased with every message! This deep relationship can be seen in looking at the similarity of the formulation for average information in a message and that for the statistical mechanics formulation of entropy. Indeed, one may hear someone refer to information as the entropy in a message.

This has been a brief look at the nature of information as what I have called a 'stuff' in my Ontos. Information is a fundamental category of what exists. It is real even if ethereal-like. It has a real effect on physical systems, otherwise computation and thinking would not be possible. Yet we are only now coming to recognize it for what it is and to realize what its role is in the functioning of the Universe.

Knowledge

The more you know (about a message source) the less surprised you are by the receipt of a message, in general from that source. The less you know, the easier it is to be surprised. In this sense, knowledge is the inverse of information. Figure 4, below, shows the relationship between information and knowledge from this perspective. What is being measured, as above, is the probability of an event, the receipt of a specific symbol, say. Having a higher expectation of the event means its occurrence is not very informative. But having that expectation means that the receiver must have accumulated experiences with the sender such that, as described above, the array of expectations represents the fact that that event has occurred frequently in the past. Similarly, an event that has not occurred frequently will have a low expectancy and thereby, when it occurs, generate more information for the receiver.

Figure 4. Knowledge can be thought of as the reciprocal (inverse) of information. Knowing less means that messages will be more highly informational. Note one of the consequences of this view is that there is no such thing as perfect or complete knowledge, i.e., since the function approaches probability 1 asymptotically there will always be some residual ignorance!

Both receivers and senders are physical systems, connected by a physical channel through which energy or energy-driven matter can flow. The energy moves, as the principle of flow demands, from a high potential source in the sender, to a low potential sink in the receiver. That energy is transduced by a physical aspect of the receiver and, in turn, can be amplified (using energy flows inherent in the receiver) to do work on the internal physical configuration of the receiver. In other words, the receiver's internal organization can be changed by that work so as to represent the message receipt.

But by the principle of dissipation (above) we can see how a sufficiently complex organization within a receiver can become 'pumped' up by prior messages such that future similar messages will have no impact on changes to that structure. The receiver's internal structure that embodies its encoded history of messages represents its knowledge of the sender! This is what I meant above by an array of expectations. A configuration of matter that can be reorganized by the receipt of a message (transduced and then, through amplification, doing work on that matter) can be seen to be 'encoding' that message. The extent to which such a configuration approaches a pumped up saturation, and subsequent dissipation of energy, determines how much capacity for storing expectations a system has. Systems that dissipate too rapidly cannot retain a 'memory' of prior events, whereas a system that can jump to a fully dissipational state and maintains stability for too long, cannot learn new relationships between message states. Systems that have workable memories have to be able to encode informational messages but also forget (reduce their expectations of) messages that were encoded but are no longer arriving regularly.

Sometime in the not-too-distant future I will write about how the human brain (neural networks) fulfill this function; how neurons and synapses do work to encode messages that are unexpected and dissipate the energy of messages that are expected. We need to make the connection between this general model of information and knowledge with what goes on in the brain in order to more fully appreciate the principles involved.

So knowledge and information form a duality just as matter and energy do. Moreover, knowledge is embodied in the configuration of matter, but the latter is determined by the flow and interactions of matter with energy. The improbability of the latter, in any given time frame, is gauged by how responsive the material configuration is to the flow. Responsiveness, followed by some form of longer-term storage of energy, constitutes a memory of the energy flow. If the configuration is then able to dissipate additional flows without significant re-configuration, then the system can be said to be NOT surprised(!) by that additional flow. Thus the four stuffs of the Ontos are deeply interrelated to one another.

Whereas the matter/energy duality is constrained by the law of conservation, it is not clear that such a law pertains to knowledge/information. The latter is characterized by the fact that information begets knowledge and knowledge changes the behavior of the knowing system. This change of behavior is the source of information for other observers that perceive our knower as a sender! Thus information increases in the universe as a result. That is the causal loop I mentioned above.

Look for future posts to use this Ontos model and the relationships just described to explicate evolution and organization. I hope to show, through specific examples, how this applies to any system we might look at, thus strengthening my argument that systems science is the meta-science I claimed.