The work of ET Jaynes strikes me as that of the supreme logician - essentially the epitome of the AI advantage which represents the exhaustive exploration of every combination possible (ergodic thoroughness). My chief criticism of this capability introduces two criticisms:-
- that of selection (the brilliance of the productivity is counterbalanced by the task of sorting the wheat from the chaff)
- that of conceptual vacuity (the mathematical or probabilistic approach inherent in any abstract construction does rely on any causal link to some materiality in which the phenomenon or findings relate - at its worse you are left with an elaborate formulation of the practical impossibility)
Amongst ET Jaynes work was the speculation and proposal of a near infinite source of energy as indicated by the mathematics (the dual theorem):
E. T. Jaynes Note on thermal heating efficiency:
Kelvin showed the maximum efficiency with which heat can be converted into work; but there is a dual theorem about the maximum efficiency with which heat at one temperature can be converted into heat at another temperature. It has some surprising implications, in particular that the efficiency with which we heat our buildings could in principle be improved by a large factor. This long known, but still little appreciated, fact is of current pedagogical interest and practical importance. DOI: 10.1119/1.1508446
To ask if this is ‘real’ I am reminded of the different points of views of two theoretical physical scientists: PW Atkins as a practical chemist and ET Jaynes as a logical chemist expressed in their writing and mutual commentary on the topic of entropy. It very much represents an argument between a worldview tethered to practical reality (the ontological perspective) and a worldview shaped by an abstract model (the epistemic perspective) in which the abstract view presents novel, unobserved, phenomena and relationships.
PW Atkins began the conversation:-
Entropy in relation to complete knowledge (Atkins)
Careless thinkers out of touch with the fullness of modern physics have ignored the quantum limitation and have slipped from ‘lack of knowledge’ to subjectivity. Maxwell fuelled the flight into subjectivism by his discussion of diffusion, where he asserted that the disorder of thermal motion is a disorder only in relation to the mind that perceives them: for gone mind, then gone disorder. Maxwell’s casual remarks were compounded by some of Gibbs’s on coarse graining, and the scope for the successes of that technique to be ascribed to the intrinsicality of our ignorance. G. N. Lewis must also share the blame for opening the door to speculative doubt in his remark that the entropy increases when a known distribution goes over into one that is unknown, and the foundations of subjectivity set solid with Born’s ill-considered comment that irreversibility is a consequence of the explicit introduction of ignorance into the fundamental laws. As the authors remark, to accept that our knowledge is inevitably incomplete is one thing, but to claim that this incompleteness implies subjectivity is something quite different.
Once foundations have been laid by such minds, however casually, others will take the opportunity to build their shanty towns of argument. Thus in 1961, in what is for many a part of the modern era, and hence for which obscurantism is inexcusable, the procession moved further along the garden path with the remark that a change in the value of the entropy can occur when some relevant facet of the problem in hand has emerged, even if only in the mind of the observer. The Denbighs quite rightly adopt the tactics of dealing with ad horninern arguments about the entropy by showing that the same criticisms, when logically valid, are equally applicable to any other physical observable, such as the age of the Devonian rocks, which is also subject to revised estimation but is not thereby subjective.
Even closer to the present, in 1965, and even further along the same garden path, came the explicit claim that entropy is an anthropomorphic concept even at the phenomenological level, for it is a property, not of the physical system, but of the particular experiments ‘you or I choose’ to perform on it. The blatant nonsense of this nevertheless quite influential argument is exposed unintentionally by its author (and illuminated by the Denbighs), when he remarks that, for example, since steam is polar, its entropy depends on the electric field strength present, and it must always be understood implicitly that the field was not varied from one experiment to the next.
That style of criticism seems to expose the total ignorance of the scientific endeavour of the writers who reflect so imperfectly on entropy, and who provide a vehicle for the general public who cannot judge the propriety of their arguments. The only use of such arguments is for the exercise of the intellect that comes from rebutting them; otherwise they merely waste some people’s time and mislead the rest. I do not have space to comment at length on the middle portion of this elegantly brief little poisoned dart of a book, a portion that examines the red herrings that have been pulled across the landscape of statistical thermodynamics, and especially its role in coarse graining (which receives a full chapter) inasmuch as it deals with the question of subjectivity.
Comment on a Review by P.W. Atkins (Jaynes)
A recent essay review (Atkins, 1986), ostensibly of a book by K. G. Denbigh and J. S. Denbigh, seems more concerned with quoting from another work, and commenting on it in phrases not ordinarily seen in scientific journals. Yet this other work is not identified.
In the belief that the interested reader will wish to put the quoted remarks back into the context in which they were made and judge the issue for himself on the level of technical fact, we supply the missing reference (Jaynes, 1965). We note also that in the 22 years since that article appeared, the writer has published 19 other articles on the same general topic, part of a much larger literature in which the properties of entropy have been developed further, leading to new applications of the Gibbs formalism. Therefore we supply five key references which will afford entry into this currently active field.
The conceptual difficulties at issue here arise from failure to see the distinction between deductive physical prediction and inference. The former is available, at least in principle, when we know the relevant laws of physics and also the initial microstate of a system. Lacking this, we can still make the best inferences possible, taking into account the incompleteness of our information: as nearly all writers on statistical mechanics have recognized, predictions made with the second law lack the certainty of deductive proof, and are only highly plausible inferences.
One of these inferences, and perhaps the main functional use of entropy in thermodynamics, is to predict the work available in an isothermal process: W = TAS -AU. Some have argued that, since entropy has this “objective” physical meaning, it cannot represent a mere “subjective” measure of human information. However, we would observe that when we cause an interaction of some kind with a system, whether energy flows in one direction or the other depends on what microstate that system is in.
It is therefore a platitude, obviously valid in a much wider context than equilibrium thermodynamics, that the work we can extract from any system depends, necessarily, on how much information we have about its microstate. If entropy lacked this property of measuring human information, it could not serve its thermodynamic function.
In our view, far from attacking J. W. Gibbs, J. C. Maxwell and G. N. Lewis for their observations on the nature of entropy, we should applaud their insight in perceiving it so early. That today some have not yet comprehended their message only adds lustre to their accomplishments.
To some extent Jaynes was aware of the challenge and wrote elsewhere:-
The Second Law as Physical Fact and as Human Inference (Jaynes):
But Gibbs was only recognizing something that is true universally. In all of science. in or out of thermodynamics, what happens in the real world depends on physical law and is on the level of ontology. What we can predict depends on our state of knowledge, and is necessarily on the level of epistemology. He who confuses reality with his knowledge of reality generates not solutions, but paradoxes. However, there is still very little perception of this in the scientific community, and attempts to point it out can generate bitter controversy. (Jaynes)
And Jaynes also argues that ‘breaking away’ from materiality is also important in order to make full use of reasoning untethered to direct evidence and the ontological perspective (in pursuit of Godelian completeness):-
A backward look to the future (Jaynes)
When applied to problems of parameter estimation or hypothesis testing, probability theory as logic is generally called Bayesian inference, on historical grounds explained elsewhere, and it is accomplishing a major house-cleaning in the eld of statistics. The “orthodox” methods of inference as taught by statisticians since the 1930’s consist of about a dozen intuitive devices (confidence intervals, unbiased estimators, significance tests, etc.), without any connected theoretical basis. Each is usable in some small range of problems for which it was invented; but each produces contradictions and absurd conclusions when applied out of its proper range. Now all of these are basically methods for reasoning from incomplete information; that is, for information processing. Yet the orthodox practitioners never thought of probability in terms of information.
In other words, we contend that these are not new fields at all; only false starts. If one formulates all such problems by the standard Bayesian prescription, one has automatically all their useful results in improved form. The difficulties people seem to have in comprehending this are all examples of the same failure to conceptualize the relation between the abstract mathematics and the real world. As soon as we recognize that probabilities do not describe reality, only our information about reality, the gates are wide open to the optimal solution of problems of reasoning from that information. (Jaynes)