Craig Bauer is a professor of mathematics, but that’s just his day job. He’s best known as a specialist in ciphers and cryptography. In this guise, he’s one of the world’s leading historians on the subject. We spoke to him about his latest book, Unsolved: The History of Mystery of the World’s Greatest Ciphers.
Noah Charney: Cryptography seems, at first glance, to be largely the realm of mathematicians and logicians. But there are some other fields that have found success, such as linguists. What must a mind be good at, in order to solve puzzles?
Craig Bauer: Persistence and imagination are two of the most important traits for codebreakers. When a solver presents his or her result, the numerous failed attempts are rarely discussed, but they are almost always part of the real story of the solution. To be successful, you cannot be easily deterred by failure. And every time failure is encountered, a new approach must be tried, if there’s to be a chance at success. That’s where imagination or creativity comes into play. Math skills can come in handy, but they won’t do much good without persistence and imagination. Also, the psychological approach is not to be underestimated. One needn’t be a psychologist, but being able to get inside the mind of whoever wrote the message, and guess words or phrases he or she might have used, is a time-honored approach to making the first cracks in a cipher. This technique was used by the professionals who broke Nazi Enigma ciphers during World War II, as well as by amateurs. A great example of the latter is the husband and wife team of Donald and Bettye Harden. They took on the Zodiac killer’s first cipher and it was Bettye, who had no previous exposure to cryptology (!), who had the key insights. Recognizing the killer as someone who was egotistical and craved attention, she guessed the message would begin with “I.” She also thought it would contain the word “KILL,” and maybe even the phrase “I LIKE KILLING.” Indeed, it did!
Ciphers that appear in multimedia packages, like Hypnerotomachia Polyphilii or the Voynich Manuscript, seem to have levels of complication that are greater than, say, a numbers-only math puzzle. And yet they also seem more accessible for the general public to “play” along, for people who would never think that they could solve a math puzzle. Are they harder, easier, or both?
I don’t think that professionals have a great advantage over amateurs, when it comes to ciphers that are not mathematically sophisticated. Many of the unsolved ciphers I present were created by amateurs and, when a professional tries to solve them, he or she will tend to think as a professional. By contrast, an amateur may be more likely to mimic the thought process of the cipher’s creator and thereby solve it.
I’m an art historian specializing in iconography, and I like to teach interpretation of paintings in terms of a detective story. A painting is like a snapshot of a crime scene, which we detectives must interpret based on what we see, with a measure of a priori knowledge. How do you teach the approach to a cipher?
I’m a huge fan of context. When attacking a difficult cipher, I want to know everything I possibly can about who sent it, to whom it was sent, and everything going on at the time that might be relevant to the contents. Sometimes textbook problems give a message with no context, but in the real world, there is always a context. The message was intercepted or found somewhere. For older ciphers, there’s a chance we can simply find the solution, if we look in the right place. Solutions to several previously unsolved “Chaocipher” messages ended up in the National Cryptologic Museum’s collection. Various other archives certainly hold solutions to more mysteries. While I expect that some of the unsolved ciphers I present will be solved by traditional attacks, I hope that others will be simply be found!
I often think that humans have an in-built “treasure hunt instinct,” as I’ve sometimes called it. We are driven to want to learn secrets, solve puzzles and riddles, fill in blank pieces of what we know. Is that hard-wired into us?
I know exactly what you are describing. It can also be called the “sense of wonder” or the “need to know” or just plain old intellectual curiosity. Whether the treasure be the laws of the universe sought by physicists or the missing details of important people’s lives that biographers seek, or something else, I think many of us possess a desire to unravel mysteries. While I find all of these mysteries intriguing, I find myself spending the most time wondering about mysteries concealed in writing by the use of ciphers. This probably arises from my great love for books. To have writings that can’t be read is nearly unbearable! I think this drive to know that arises in so many different contexts is responsible for most of our progress as a species.
There is a nervousness sometimes that a cipher doesn’t actually have a solution, that it’s the scribblings of someone mad or bored, that it is a sort of trap to steal our attention in time. Have you come across such prank ciphers, and is there any way to tell, really, whether a cipher is just tough to crack or doesn’t actually have a solution?
A delicate ego can always rationalize away failed attempts to solve a cipher by deciding that there wasn’t any real message to be found there anyway! I’ve often seen what I think are legitimate ciphers dismissed as gibberish or pranks, and I think these labels are overused. But sometimes these labels are appropriate and, in rare instances, we can find strong evidence in support of them. For example, someone using a keyboard or typewriter to fake a cipher may err in his attempt to produce random text by unconsciously alternating his keystrokes like so: Right hand, left hand, right hand, left hand… Such a pattern would reveal the cipher as a hoax. Although of a different sort, I think there’s also great evidence for the unsolved Beale ciphers being a hoax. One of the ciphers is said to contain the location of literally tons of buried gold and silver, while another supposedly contains the names of thirty men who each had a share in the treasure, as well as the names and addresses of their heirs. Yet, this latter cipher only amounts to 618 letters. Dividing this by 30 we get just 20.6 letters, on average, per person, for a name, the names of this person’s heirs, and their addresses. This is not nearly enough! There are many other indicators that the Beale ciphers are a hoax, but this is the simplest, and I find it convincing. Hoaxers often make simple errors that reveal them.
How has machine learning helped the world of cryptography?
The development of modern computers was largely initiated, and accelerated, by the needs of the cryptanalysts. A great quote to this effect was given by Lieutenant General William E. Odom, Director of the National Security Agency from 1985 to 1988:
“For example, development of modern digital computational means—computers—occurred almost entirely as a result of the National Security Agency’s research and development efforts in the late 1950s. IBM and CDC essentially got their start in modern computers from National Security Agency funding, and without it, we might be two decades behind where we are in computers today.”
This strong motivation continues today in the push for quantum computers. But the demand is not just for hardware. The need is not just for faster machines, but also for smarter machines. Alan Turing played a key role in unravelling Enigma ciphers from WW II, and was also a pioneer in artificial intelligence. He thought machines would be truly thinking by the year 2000. We missed this mark, but efforts continue.
Is it in any way “cheating” to employ machine learning or computers to help solve old ciphers that were designed to be deciphered long before machines could assist us in doing so?
I don’t see it as cheating. If you develop new tools to solve a cipher, you’ve not only solved the cipher, you’ve also developed new tools. I’d say it’s doubly impressive!
How did you choose the ciphers for your new book, Unsolved?
The selection process turned out to be much harder than I expected. Originally, I planned a book of a little over 300 pages, but the final result was well over 600. I found it impossible to write a short book, when there were so many fascinating cases to consider. Actually, I feel like I showed considerably restraint by stopping at the length I did. I made no mention of the numbers stations of the cold war, which have modern manifestations online. These are accessible to everyone, but if anyone has broken such ciphers, they aren’t talking! I also left out unsolved military ciphers from WW II, and even some from as far back as WW I. The chapter on ancient ciphers could have been expanded to become a book all on its own. Leaving all of this out was very painful! I have some consolation in the fact that I can always write a sequel. But to address your question, I selected the material with an eye toward variety and accessibility, as well as intrinsic interest. However, some ciphers were left out not because they failed to meet one of these criteria, but rather because I became aware of them too late in the publishing process.
Is there one cipher that has been your nemesis and that you’d be most delighted to have a hand in solving?
As I was researching and writing, I quickly realized that I would never complete the book, if I kept stopping to try to solve the ciphers. In some instances, I made initial steps towards a solution, but then forced myself to stop. I included these steps in the book and I look forward to seeing if they lead readers to solutions. So, I don’t have a nemesis as such. However, I do want to learn more about the Paul Rubin case. In 1953, the 18-year-old Rubin was found dead in a ditch by the Philadelphia airport, with a cipher taped to his stomach, and enough cyanide in his system to kill ten men. From the position of his body, it appeared that Rubin had been murdered and then placed in the ditch. The case gets stranger, as it’s examined more deeply. Rubin, who was Jewish, had a picture of a Nazi aircraft in his wallet and also had a fountain pen gun. What on Earth was he up to? Part of what makes this case special to me is that, as far as I can tell, it hasn’t been looked at since the 1950s. In fact, I believe my book is the first place in which a clear image of the cipher has been published. I dug out as much information as I could, including getting the large FBI file on the case declassified by making a Freedom of Information Act request. Still, there’s a great deal that is not known. For example, I haven’t even been able to find a photograph of Rubin yet. I thought a lot about this young man as I was writing about him and reading the FBI file, and I have a picture of him in my mind, but I haven’t seen his real face.