Herschel and the Physics of the Invisible Universe

Most of the universe is invisible to humankind. Moreover most of it is currently invisible to our modern equipment. As advanced and fine-tuned as it is at present, all of our detectors in the enterprise of physics could only ever see about 4% of the universe, theoretically. The rest, 96%, is essentially an invisible section of reality, made up of Dark Matter and Dark Energy. These percentage distributions is also merely based on what we know now about the structure of galaxies and their expansion, there is every reason to think that there is even more hidden physics, perhaps on forever.

We sometimes take it for granted that a lot of physics is invisible to our eye and everyday experience, particularly when considering our modern technological world depends on the existence of countless invisible wireless signals.

However the existence of invisible packets of energy, acting at a distance, would have been a completely occult notion at the time of Newton and Huygens with their respective theories of light being made up of quantifiable corpuscles and waves respectfully, expanding upon the first treatment of light as being a system of rays as described by Fermat into being a composite structure to describe such phenomena as reflection, refraction and of course the visible spectrum as a function of interaction of white light with a prism.

Newton's Corpuscular (or particle) theory of light, is based on the view of light being made up of equally elastic particles, held sway for much of the 18th century as it could explain refraction and reflection relatively well up until the phenomenon of interference could not be explained using particles alone.

The development of classical optics as described mathematically by Huygens and then used by Fresnel in the Huygens-Fresnel wave theory of light is necessary, among other things, to explain in the observation that parallel if a beam of light shines on a small round object, a bright spot will appear at the center of a circular shadow by means of constructive interference of the waves travelling around the object. Further developments of course in the experiments of double slit produced interference by Thomas Young further confirmed the wave theory of light as being necessary.



In any event, the notion of "action a at a distance" when considering forces acting on particles of light or matter was deeply troubling to Newton and he consciously developed his theories to eliminate what he considered to be non-existent forces. Newton's laws are a their core an empirical approximation of physics*(#1) and in a sense continue on the work laid out by Galileo before him. Newton, using the calculus he invented, ran mathematical studies on how things moved over time, and found a small set of rules which could be used to predict what would happen to, say, frictionless balls being pushed from a state of rest and how inertial masses create equal and opposite motion. The laws "work" in effect because they are effective at predicting the visible universe, which was assumed by scientists of the 17th century to be analogous to a clock with a perfectly visible and quantifiable clockwork.

Newton and other contemporaries such as Huygens did not examine physical laws which were in a sense "invisible" - Newton himself could not explain the origin of gravity in his theory and in a way his physics was a bookkeeping device to in a sense replace its explanation with its function and thus make predictions of the thing without having to explain the thing itself. and objects beyond simply labels of F12 and M1, F21 and M2, so that F12 = -F21 for example:

Huygens also did not explain the origin of the waves in his wave theory of light, that needed much more work in the development of electromagnetism and it was not until nearly the end of the 19th century that the invisible forces of electricity and magnetism acting in unison gave rise to the nature of the waves of light itself, as described by James Clerk Maxwell.

Nevertheless, Newton's theories were and still are very successful in applications, however it must be said that they do lack explanatory power of the nature of the forces themselves. They are still, in effect, "invisible" in this arrangement*(#2)

The beginning of modern science to look into the physics of the truly "invisible" really began with the famous 18th to 19th century astronomer William Herschel.

Sir William Herschel

bust by John Charles Lochée

1787

William Herschel was born on November 15th, 1738 in Hanover Germany. Herschel performed his extensive scientific work in Britain, which he migrated to when he was 19 to avoid military service during the infamous Seven-Years-War. Living in Britain, the young Herschel had been able to earn a living as as a skilled musician and composer for 15 years, during which his sister Caroline Herschel joined in him Britain and became a skilled musician, mathematician and astronomer in her own right. Both the Herschel siblings' work in astronomy was remarkably thorough, dedicated and careful. However, we shall see that he was not so careful to avoid the spark of true discovery that can often lie hidden just orthogonal to established knowledge.

Beginning his work in astronomy, William Herschel constructed his primary Newtonian telescope in the back garden of his home in London. He had constructed his telescope with the specific missions to study binary star systems to observe, over many years with his sister Caroline, the proper motion of stars and this singular focus led to a plethora of discoveries along the way between his initial years of astronomical study between 1779 and 1792. He soon discovered many more binary and multiple stars than expected, and compiled them with careful measurements of their relative positions in two catalogs presented regularly to the Royal Society in London.

Artist's rendition of a binary star system

In 1797, Herschel measured many of the systems again, and discovered changes in their relative positions that could not be attributed to the parallax caused by the Earth's orbit.

He waited until 1802 to announce the hypothesis that the two stars might be "binary sidereal systems" orbiting under mutual gravitational attraction, a baricenter essentially around empty space, a hypothesis he confirmed in 1803.

Such a discovery was very phenomenal at the time, as Herschel himself had been influenced by the writings of his fellow astronomer John Mitchel, who in 1783 proposed the concept of "Dark Stars", essentially invisible stars so massive that light cannot escape. Mitchel was also supportive of the theory of binary systems of stars existing around mutual gravitational centers and perhaps influenced a connection between the two.




In between this work, while looking for binary star systems, Herschel systematically reexamined objects discovered previously by Charles Messier and cataloged (but largely unclassified) in his famous catalog on stars. Messier was focused on searching for comets and so, understandably, did not spend much time on the objects he had first classified in a system of numbers we still use today for most objects seen in the Northern skies, M31 for example; the Andromeda Galaxy.

Herschel discovered that many objects, seen initially as stars, in the Messier catalog were in fact clusters of stars or nebulae* but one object he found, which had not been found by Messier before and which turned out to be the Planet Uranus. This was a significant discovery, as it was the first planet discovered since ancient times, all planets up to Saturn having been known since at least the Ancient Greeks and most probably earlier. It also helps put to sleep the notion that there is some hidden geometric mysticism to the planetary arrangements to many people: after all, what astrologer or psychic ever had anything to say before about the 7th planet!


This discovery also set Herschel up with the opportunity to do astronomy, and by extension, physics in general, as a full time profession. Herschel, somewhat obsequiously, called the new planet the "Georgian star" (Georgium sidus) after King George III, which predictably brought him favor with the King; the name of the Planet did not stick, particularly internationally. In any event, the King of England offered William a constant stipend of 200 pounds a month, less than he could of earned as a full time musician according to records but a job that Herschel took immediately! Caroline herself was eventually to get 50 pounds per month as William's assistant, making her the first woman in recorded history to receive a salary as an astronomer.

William and Caroline, now both in the employ of King George III, were now able to conduct astronomy on a scale that saw the largest telescopes of their time being constructed, such as Herschel's famous 20 foot, 18.7'' telescope

Herschel's 20 Foot Telescope

It could be argued this began the age of large scale telescope construction. Progress in astronomy on the surface of the earth really is a matter of simply building larger individual telescopes (or baselines of multiple telescopes) in better locations and this is a fact right up to the present day.

During his career, Herschel was as renowned as a maker of telescopes as he was as an astronomer, and he oversaw the construction of several hundred instruments, which were sold in England, Ireland and in the European continent. Using even larger telescopes, such as 40 foot telescope built in 1789, William was able to catalog much more deep sky objects and created the first surveys of the Milky way and discovered that many of the deep sky objects were so-called "spiral nebulae", really galaxies although the notion of a galaxy was unknown to astronomers of that time.

Caroline was also given a telescope, constructed by William, similar to the one with which he had discovered Uranus. With this telescope Caroline discovered 8 comets and over a dozen “nebulae” between 1783 and 1797.

These included: Spiral Galaxy NGC 253 Elliptical Galaxy M110, one of the satellite galaxies to Andromeda and discovered many comets in her time too, being one of the first women to be recognized for such work at the Royal Society in London.

During the rest of his life, William Herschel produced lists of thousands of nebulae and star clusters, and he was the first to distinguish between distant clusters and dusty nebulae.



As large as some of the telescopes he designed, William Herschel was also working with smaller telescopes, this time to examine the spectrum of stars and of the Sun itself. Herschel started at first to use color filters to separate the different bands of visible light from astronomical objects, a technique still used in amateur astronomy today to highlight certain features.

A Color Filter that Exposes the Near-Infrared and Near-Ultraviolet from Sunlight

In his observations of the Sun, in February 1800, Herschel described how the various colored filters through which he observed the Sun allowed different levels of heat to pass. The energy output of a 5700–5800K star, our sun, is greater in the visible wavelengths than in the infra-red, and most of the visible light should be passing equally through the atmosphere. How could these redder wavelengths then be hotter?

He performed a simple experiment to study the 'heating powers of colored rays': Like Newton over a century before he split the sunlight with a glass prism into its different constituent colours and this time made the next important step, using a thermometer as a detector he measured the temperature of each colour.

Herschel observed an increase as he moved the thermometer from the violet to the red part of the spectral 'rainbow'.

The result is due to the fact that the blue wavelengths in the classic Herschel experiment are more strongly scattered, hence more "spread out", than the redder wavelengths and thus more energy reaches the thermometers measuring the red end of the spectrum.

This experiment, given the limits of technology at the time, could not measure the discrete energy in each color individually but at the very least Herschel provided a kind of reference frame between different regions of the spectrum, with respect to their indices of refraction by the prism, and their temperatures.

Today, we can measure the discrete energies of wavelengths of light, independent of the effects of light scattering affecting the intensity. As a rule of thumb  thinking in modern terms, blue light is in fact more energetic than red light in terms of the excitation energy of electrons than it is in terms of its "temperature" as measured by Herschel.



Herschel also measured temperatures in the region just beyond the red color where no light was visible as a control. Herschel expected it to merely measure the ambient temperature of the room and, to his surprise, it recorded an even higher temperature there in apparent "shade". He deduced the presence of invisible 'calorific' rays, now called 'infrared' radiation.

An underlying understanding of the physics involved makes a big difference in how to interpret the results. The concept of heat had clearly been known since prehistory, as well as the fact that it travels through the air from a flame or heated object. What Herschel discovered was subtler than the existence of an invisible heat radiation.  Herschel found the first solid evidence that light and infrared are the same quantity that we know today to be electromagnetic radiation, a critical example of scientific reductionism.

Through a series of simple experiments, Herschel found the first piece in one of the great puzzles of physics that would take another century and the formulation of the quantum theory to solve, particularly when one considers that objects that emit light through heating must emit energy in discrete packets, quanta, in order for the visible world to make sense, as is the case for Max Planck's theory of radiation solving some of the absurdities from the so-called "ultraviolet catastrophe".

The Planck Black Body Radiation Curve

But perhaps the greatest lesson to be learned about the discovery of the infrared part of the spectrum of light is how it was discovered. Let us remember: it was not by complete accident. Herschel was thinking about the problem and searching for it the right way, based on an exploitation of the known principles of light, some of which he discovered (i.e. light filtered Red being hotter than Blue). However the discovery was found by a detector system that was not so fine-tuned and was exploring part of the answer in what would appear, on first glance, to be not in a local optimum location but then turned out to be the global optimum of the search for a discovery!

What does it really mean then when we can detect something that is a genuine discovery, using a detector system that is not set up properly? Does that mean that although we may set out to find something to prove (or should I say disprove!) a theory, even if we set up our equipment 100% correct (with that careful precision physicists are most famous for) we may not be guaranteed to find anything because what we are looking for is, in a sense, just a little nudge to the right or to the left?

In the history of science Herschel's discovery of IR radiation is in great company, with the discovery of the CMB as another similar example. I cannot say for all cases but it is sometimes the case and the unfocused or just downright wrong setup can give us real jewels of scientific knowledge.

Extra Points:

*#1
This is particularly evident with Newton's third law. Newton in his time never could explain the origins of forces that governed the laws he discovered, nor could Huygens . Without specifying the nature or origin of the forces on the two masses, Newton's 3rd law assumed they arise from the two masses themselves, that they must be equal in magnitude but opposite in direction so that no net force arises from purely internal forces. Hence physics is then reduced to a kind of balancing act between the 2 opposing masses.

Newton's Third Law is in reality, a meta-law (that is a higher level law invoking the existence of a more fundamental law) and we find, through Noether's Theorem, that it arises as a direct consequence of conservation of momentum, which essentially says that in an isolated system with no net force on it, momentum doesn't change. This means that if you change the momentum of one object, then another object's momentum must change in the opposite direction to preserve the total momentum. Forces cause changes in momentum, so every force must have an opposite reaction force.

But now we may ask:

Why is momentum conserved?

Conservation of momentum comes from an idea called Noether's Theorem, which states that conservation laws in general are a direct result of the symmetries of physical systems. In particular, conservation of momentum comes from translation symmetry. This means that the behavior of a system doesn't change if you select a new origin for your coordinates (put differently, the system's behavior depends only on the positions of its components relative to each other). This symmetry is found in all isolated systems with no net force on them because it is effectively a symmetry of space itself. Translation symmetry of a system is a consequence of homogeneity of space, which means that space is "the same everywhere" - the length of a rod doesn't change if you put it in a different place.

But now we may ask:

Why is space homogeneous?

In classical mechanics, this is one of the basic assumptions that allows us to do anything In this case it basically says that because the laws of physics as expressed in terms of Cartesian coordinates are the same no matter where you put the origin (the coordinate system and its origin is an arbitrary artificial invention) . Noether's theorem mathematically implies that momentum must be conserved in such a condition. 

In reality, according to General Relativity, space actually isn't homogeneous - it curves in the presence of massive objects. But usually it's close enough to homogeneous that classical mechanics works well (gravity is pretty spectacularly weak, after all), and so the assumption of homogeneity holds.

*#2 

In the contemporary world of Newton, the forces of magnetism and static electricity (through rubbing fur on amber and so forth) was known about, albeit nascent by the ancient Greeks and the Chinese too. This knowledge was pure empiricism and with very little theory behind it, and this could have been interpreted as an example of "action at a distance" in which some force overcomes the balance of Newton's third law with regards to the quantity of mass alone. Newton did not have a knowledge of charge or magnetic field strength, and nor would anyone until well into the 19th century.  

It seems that Newton did not pay much attention in incorporating the concept of magnetism or charge in his description of the physical world. He explicitly mentions electricity in Optics and he seems to have used magnets in experimental studies. That he didn't go very far with this is not particularly surprising: forces between permanent magnets do require knowledge of how calculus works in predicting the aspects of electric and magnetic fields and Newton's Laws generally follow afterward. The concept of opposing magnetic poles and electrical charges was not well quantified in Newton's time and so would have appeared somewhat occult to him.

Nevertheless, In the famous final paragraph of the Scholium Generale that Newton added to the second edition of the Principia, published in 1713, he wrote of “a certain most subtle spirit which pervades and lies hid in all gross bodies.” It was this active spirit that gave rise, he supposed, to the electrical attractions and repulsions that manifested themselves at sensible distances from most bodies after they had been rubbed, as well as to the cohesion of particles when contiguous. In addition, he surmised, it was the agency responsible for the emission, reflection, refraction, inflection and heating effects of light; and by its vibrations in “the solid filaments of the nerves,” it carried sensations to the brain, and commands of the will from the brain to the muscles in order to bring about bodily motion.