Cryptologic Challenge: Navy-Army Cooperation
For several years in the early 1930s, the U.S. Navy was the only source of intercepted traffic from Japanese communications. After the Army obtained an intercept capability against diplomatic targets sometime in 1935, a "friendly rivalry" developed as both services attempted to intercept and read as much as possible "to gain credit" for the intelligence.31 In the late 1930s the Navy discovered that the same Japanese consulate which had twice yielded the Red Book in the early 1920s was also a likely source for "effective and reserve" ciphers and keys for all current diplomatic systems except the two machine systems A and B. Once again, by "borrowing" them the Navy was able to provide
the Army and the Asiatic Fleet with Japanese diplomatic ciphers and keys for manual systems before they came into use.
The decade of the 1930s also witnessed a resurgence of U.S. Army interest in cryptanalysis. In 1930, after the collapse of Yardley's New York "Black Chamber,"32 William F. Friedman was tasked to create an Army cryptologic capability in the office of the Chief Signal Officer. Starting with four civilian students whose names have become bywords in the U.S. cryptologic community -- Frank Rowlett, Solomon Kullback, Abraham Sinkov, and John Hurt -- Friedman began the slow and difficult training process which would ultimately lead to the compilation of War Department codes and ciphers and the solution of foreign military and diplomatic codes and ciphers.
It was to this embryonic work force that the Navy turned in 1931 for help against two cryptographic targets which at the time almost completely occupied OP-20-G's efforts -- Japanese diplomatic and naval communications. The introduction of the Blue Book, as the Japanese Navy Operations code was known, in February 1931 (replacing the Red Book) and an unexpected surge in cipher traffic on diplomatic circuits had created an immense work load for Navy cryptanalysts. This forced the Navy to realize that it could no longer handle both targets and to seek a division of effort with the Army, with which it would furnish intercepted traffic until the Army could develop its own collection capabilities.33
Not willing to give up all diplomatic communications, however, the Navy proposed that the Army analyze all counterpart Army radio communications and all diplomatic radio communications except for those of the four major naval powers, England, France, Italy, and Japan. This arrangement, the Navy claimed, would help it reduce an estimated two-year time lag in breaking the Japanese Blue Book.34
For a number of reasons, negotiations were not immediately fruitful. A primary cause of the lack of progress in the negotiations was that Army intercept sites, when established in the U.S. or even those existing in the Philippines, would not be able to hear low-power military radio transmissions. This unpleasant fact made the Navy's proposition partially irrelevant except in China. There Station A could and did irregularly intercept both Chinese and Japanese ground forces communications, which were provided to Army analysts. Talks continued without resolution until 1933, when a tentative position was developed from presentation to the Joint Army-Navy Board under a much broader heading, "Joint Effectiveness of Army and Navy Communications Systems." The joint proposal encompassed not only COMINT but communications and communications security matters as well. Possibly in return for its promise of cooperation on COMINT target distribution as outlined in 1931, the Army obtained concessions from the Navy in several vital areas including training intercept operators and in preparing a COMSEC Annex to Army War Plans. In addition, the Navy agreed to provide training for enlisted communicators and communications officers.35
In 1933 the official aim of both the Navy and the Army in the negotiations could be summed up in two words: "cooperation" and "uniformity." Uniform communications, uniform censorship rules in wartime, uniform authentication systems, and common recognition signals for aircraft, local defense forces, and defense districts were goals which motivated both sides. Since the Navy already had such tools in place within its framework of naval districts and the Army lacked such a structure, it clearly made sense for the Army to consider building on the Navy's experience.
Regarding COMINT matters, however, joint agreements were harder to resolve. The fragile system of cooperation on COMINT targets almost collapsed within days of its tentative approval when it was disclosed by the Army that, inexplicably, the State Department had completely rejected the proposal insofar as it pertained to the Army's collecting diplomatic communications. According to internal Navy correspondence, Army negotiators from the office of the Chief Signal Officer discussed the proposed division of effort with the chief of the Army's War Plans Division, who informed the State
Department. State's rejection of the plan was reported in a memorandum to DNC by OP-20-G on 10 April 1933. Despite State objections, however, some degree of cooperation between the Navy and Army seemed assured.36
With the Blue Book about to be solved, but undoubtedly aware that much work in defining mutual areas of interest remained to be done, the parties were not about to be deflected from their main military goal by a civilian State Department. A memorandum for the Joint Board was approved by Admiral Pratt on 24 April 1933. Attached to the memorandum was a list of twelve joint studies including the recommendation that "a joint study should be made in regard to radio intercept and radio intelligence problems ... (because of manpower shortages) division of the work should be agreed upon and ... exchanges of information, in outlying districts particularly, should begin without delay."37 On 13 July 1933, the Joint Board, with General Douglas MacArthur as senior member present, responded by recommending to the Secretary of War that nineteen separate joint committees be established to increase "the joint effectiveness of Army and Navy Communications Systems. ..." Among the committees which were to report back to the Joint Board for final action were two which pertained directly to radio intelligence: (1) Radio Intercept and Radio Intelligence Problems, chaired jointly by Major Spencer B. Akin, Signal Corps, and Commander Howard F. Kingman and (2) Communication and Radio Intelligence Development Board, chaired jointly by Major Akin and Commander F.D. Pryor (Commander Kingman was a member). The following day, 14 July 1933, the Secretary of War, George H. Dern, and Secretary of the Navy, Claude A. Swanson, approved the Joint Board's recommendation.38 Unfortunately, with their cryptanalytic work load once again in hand and despite the existence of the committees, Navy officials were unable to arrive at a satisfactory consensus with the Army on COMINT cooperation, and the subject appears to have languished until circumstances in 1940 once again demanded attention.
Status Quo in the Pacific
Until the end of 1938 the Japanese maintained a cryptographic status quo which enabled the U.S. Navy cryptanalysts to live in the best of worlds. After the Blue Book system was reconstructed by OP-20-G, it was a relatively simple matter for the better part of five years to follow the activities of the Japanese Navy. As one observer noted during the 1930s in the Philippines, "Japanese traffic was everywhere you looked." There were so many options available that a selection process was required to control collection.39 In Washington, OP-20-G's interest in Japanese diplomatic traffic until 1938-39 remained almost a purely technical one -- that is, solving ciphers and recovering keys simply for the cryptographic value. The one exception was the support provided overseas cryptanalysts, particularly at Station C, who were supplied Japanese cipher and key recoveries for their value in developing COMINT in support of the Asiatic Fleet. Station C's responsibilities included keeping CINCAF informed of developments in diplomatic as well as naval messages copied by Peking, Guam, and the Philippines.40
A New Attempt at Cooperation
During 1938-39, U.S. successes against both the naval and diplomatic targets began to unravel as the Japanese changed their long-standing cryptographic systems. These developments brought the two U.S. military departments back to the bargaining table in mid-1940. As usual, both sides agreed to go their own way on international commercial and counterpart communications. Regarding diplomatic communications, General Joseph O. Mauborgne, Chief Signal Officer, U.S. Army, proposed an elaborate study to determine which targets could be heard by the individual stations of each service. According to Mauborgne's proposal, responsibility would be assigned according to hearability, frequency, time of day, type of transmission, and, in the case of duplication, preponderance of copy without regard for the underlying value of any intelligence to the intercepting agency.
Convinced that the OP-20-G work load was already excessive, Safford originated several appeals to Rear Admiral Leigh Noyes, DNC, between July and September concerning the pitfalls of this approach. In October 1940, for example, he advised Noyes that the Navy did not want to do German, Mexican, and Italian traffic. He also said that the Signal Corps had little to do if it did not copy high-powered diplomatic transmitters since its stations could not hear the relatively low-powered military radios. He advised Noyes that the Navy should relinquish the entire diplomatic target rather than agree to the proposed Mauborgne scheme.
Before the study could be undertaken, the Army General Staff ordered the Signal Corps to copy the diplomatic circuits of Japan, Germany, Italy and Mexico.41 Although it meant wholesale duplication of collection, this directive left little room for the two departments to negotiate (no doubt to Safford's immense relief) and led eventually to the recommendation of August 1940 in which the U.S. Navy became responsible for deciphering and translating Japanese diplomatic and consular service messages on odd days of the month and the Army on even days (see Chart A). This narrow and highly simplified arrangement at least relieved Safford of the specter of two conflicting translations of the same message being delivered to the president. It did not, however, as will be seen, relieve the Navy's cryptanalytic and linguistic workload, particularly in 1941 as the crisis between Japan and the United States deepened and the number of diplomatic messages to and from Japan increased. The recommendation was nevertheless approved on 3 October 1940.42
Army and Navy Sites Authorized to Intercept Diplomatic Traffic, August 1940
|Site Location||Site Designator||Number of Collectors||Site Location||Site Designator||Number of Collectors|
|Fort Monmouth, NJ||1||19||Winter Harbor, ME||W||8|
|Presidio, CA||2||9||Amagansett, NY||G||4|
|Fort Sam Houston, TX||3||14||Cheltenham, MD||M||20|
|Corozal, CA||4||20||Jupiter, FL||J||4|
|Fort Shafter, HI||5||19||Bainbridge Island, WA||S||12|
|Fort Hunt, VA||7||24||Heeia, HI||H||8|
Despite a prevailing shortage of cryptanalytic manpower between 1924 and 1941, the U.S. Navy's efforts against Japanese naval codes and ciphers were marked by some brilliant successes. Much was due to the inspired work of people assigned to OP-20-G, such as Safford, Agnes Driscoll (neé Meyer), Dyer, Wright, and Holtwick. Some of the success, however, must be attributed to the ONI, which three times in this period "borrowed" Japanese naval and diplomatic manual codes and ciphers from the Japanese consulate in New York. The Army too deserves credit and praise for its work against high-level machine systems used in enciphering Japanese diplomatic messages.
From 1924 to 1940, U.S. cryptanalysts adopted a system of color designations for certain high-level Japanese cryptographic systems. The Japanese diplomatic machine ciphers were designated Red for the A machine and, in 1939, Purple for the B machine which replaced it at many embassies. In 1939, a naval attaché machine cipher was introduced. It was designated Coral by the U.S. and was in use until 1945.43 The Japanese Navy's main operational code was designated Red until 1930, Blue until 1938, and Black until 1940, when its designation was changed to JN-25, the Fleet General Purpose System.44
The Japanese Navy also employed several other cryptosystems to conduct it business which were not swept into the U.S. system of color designations. At OP-20-G, for example, one worker decrypted all messages in the Japanese navy-merchant vessel liaison code.45 U.S. cryptanalysts read the code in its entirety from the fall of 1939 to the tenth of August 1941.46 Six other Japanese naval systems were intercepted regularly. Two of these -- an auxiliary ship cipher and a minor general-purpose system -- were not worked. A third, an intelligence code, was considered of little importance after its contents were discovered, and it was ignored. The three remaining systems were worked intensively. They were the Japanese naval administrative system, a materiel system, and the fleet general-purpose system. The administrative and materiel systems had similar encipherment forms, and both encipherments were broken from time to time. When this occurred, two workers were assigned to recovery of the underlying codes. Success in the administrative system led to a limited capability to solve the general-purpose code. The materiel code was worked during the spring of 1940 in an unsuccessful attempt to learn details about the performance characteristics of the battleships Yamato and the Musashi, superbattleships built in violation of existing treaties, which were launched in 1941 and 1942, respectively. Regrettably, all recoveries on Japanese naval systems before Pearl Harbor yielded cryptanalytic technical information rather than current intelligence.47
In the grips of a rapidly expanding work load, the limited number of skilled U.S. cryptanalysts and linguists made it impossible to produce current intelligence except in the diplomatic field.48 The explosive growth of Japanese diplomatic and naval cipher traffic (1200 percent growth between 1930 and 1935)49 continued in both volume and numbers of systems throughout the 1930s. By the end of 1942, the Japanese Navy employed fourteen different minor systems which generated over 40,000 messages per
year in addition to messages obtained from the general-purpose system which, by November 1941, had reached 7,000 messages per month.50
Recovering the "Blue Book"
At the end of October 1938, however, without warning the Japanese Navy changed its operational code. Why the Japanese chose this moment to make the change is unknown. Perhaps they feared their old system had been penetrated, or perhaps this was the beginning of a cycle of changes. The change replaced the Blue Book, which had been used since 1931, with the Black Code.
The outgoing Blue Code was never used without a cipher to be stripped off before the code could be reconstructed. Navy cryptanalysts Safford, Dyer, and Driscoll solved the Blue Code in 1933, making possible the important successes against the Imperial Fleet exercises in 1934 and 1935. Their success had followed what was possibly the most difficult cryptanalytic task ever undertaken by the United States up to that time. In Safford's opinion, Driscoll's work in solving the system may have been even more brilliant than the Army's subsequent solution of the Purple machine because "there were no cribs or translations to help out."51 The introduction of IBM "tabulating machines" against the Blue Book was also a major advancement at the time.
Two additional codes augmented the Japanese Black Code beginning on 1 June 1939: the "Flag Officers Code," which saw very limited use and was never broken, and a five-digit enciphered general purpose code given the designator "JN-25." The Flag Officer's Code was one of Hawaii's principle tasks until mid-December 1941.
The JN-25 system required three books to operate: a code book, a book of random numbers called an additive book, and an instruction book. The original code book contained some 30,000 five-digit numbers which represented Kana particles, numbers, place-names, and myriad other meanings. A key characteristic of this system was that, when the digits in a group were added together, the total was always divisible by three. The book of random numbers consisted of 30 pages, each of which contained 100 numbers on a 10 x 10 matrix. These numbers were used as additives -- they were added to the code groups digit by digit without the carryover used in customary addition -- thus enciphering the code. The instruction book contained the rules for using the aperiodic cipher. The number of each page and the number of the line on the page where the selection of additives began served as "keys" which were included in each message at the beginning and end. This code subsequently became the most widely distributed and extensively used of all of Japan's naval cryptosystems.52
Using improved IBM card sorting equipment and newly developed analytic techniques and noting similarities to an earlier four-digit "S" system stolen from a consulate, Driscoll and her colleagues were soon stripping off daily keys and additives in the Able, or first cipher, and slowly reconstructing the code.53 After investing a year in attempting to understand its components, OP-20-G put aside all work against the current JN-25 cipher during the summer and fall of 1940 in favor of slow but steady progress toward actual reading of the underlying code. After keys were recovered on each new cipher, the traffic itself was filed for later study.
Though they were working on year-old traffic, the cryptanalysts recovered a segment of the Able code which led to discovery of pattern messages, such as medical reports, and stereotyped messages containing noon positions for convoys. On 1 October 1940, the Japanese introduced the fifth Able cipher (Able Five). It was quickly diagnosed by OP-20-G analysts. Once the new keying system was understood, Washington policymakers decided that all units, including Hawaii, should begin working on the current cipher in the hope that by January 1941 the first JN-25 message of the new year would be read on the same day it was sent. By December 1940, U.S. cryptanalysts had recovered the system of text additive, two systems of keys, and the actual code groups for the numbers 000 through 999. At this point the only factor which seemed to prevent complete exploitation of JN-25 was lack of manpower. Out of the total cryptanalyst population in Washington at this time (thirty-six in December 1940), only from two to five people could be spared to work on this still unreadable system.54
Turning Victory Into Defeat
On 1 December 1940, probably before Washington's work could be distributed to the field stations, Japan canceled the Able code which had been used since 1 June 1939. This action completely frustrated any hope of code exploitation by 1 January 1941. The new code was named JN-25 "Baker." It proved to have several unfamiliar features in key generation as well as new and larger code and additive books. For the next two month, however, until 31 January 1941, many messages were intercepted in which the Japanese employed Able code additives already recovered in Able Five. OP-20-G lost no time in exploiting this cryptographic blunder by placing the entire Corregidor effort and most of the Washington effort on the current cipher and code recoveries.55
With the progress made on recovery of the new code values, U.S. officials believed that the combined efforts of all units would again bring the system close to the point of reading current traffic by early summer 1941. Code recovery continued to progress well. Throughout the summer and fall of 1941, new discoveries about the nature of the code were routinely committed to a Registered Intelligence publication (RIP) and given wide if slow-moving distribution to the field units.
The actual reading of current Japanese messages before Pearl Harbor, however, was not to be. U.S. cryptanalysis of the ciphers had outstripped the U.S. capability for code
recoveries. That is, OP-20-G and Corregidor (as well and London and Singapore) had not recovered enough of the basic code, and JN-25 decrypts could not be produced in time to play a part in U.S. and policy or military decisions during this crucial period. Thousands of intercepted Japanese Navy messages in JN-25 were not exploited because, as a result of manpower shortages and higher priorities, the underlying code values remained unrecovered.57
These proved to be costly factors indeed, because analysts at Hawaii, Corregidor, and Washington never discovered the vital information contained in the untranslated messages. We now know that they contained important details concerning the existence, organization, objective, and even the whereabouts of the Pearl Harbor Strike Force, the Japanese Navy's First Air Fleet. Hidden in these messages was the full magnitude of the enterprise planned by the Japanese to begin on 7-8 December 1941. Had these messages been read on a current basis, it is possible -- even probably, given the analytic skills so evident in these centers -- that the early course of the war would have been significantly altered. Unfortunately, most of the U.S. Navy's cryptanalytic effort was devoted to another Japanese cryptographic problem: recovering the daily cipher, translating the texts, and reading the Japanese diplomatic messages.
31. History of Signal Security Agency, Series III.hh, CCH History Collection (classified).
32. Herbert O. Yardley, The American Black Chamber, originally published in book form 1 June 1931.
33. 29 October 1931 memorandum from DNC to CNO via DNI, "Allocation of RI Activities Between the Army and the Navy," Series VII.19, Box 4, Vol. 1, pre-1942, CCH History Collection (classified).
34. 29 October 1931 memorandum from OP-20-G to CNO via DNI, Series VII.19, Box 4, Vol. 1, CCH History Collection (classified).
35. 12 April 1933 memorandum for the Director of Naval Communications from J.W. McClaran, OP-20-G, Series VII.19, Box 4, Vol. 1, CCH History Collection (classified).
36. There is no record of this episode in the Department of State files.
37. RG 220, NA. Proceedings of the Joint Board, National Archives, Joint Board Memorandum, 24 April 1933.
38. A series of unclassified memoranda between CNO, the Joint Board, and the Secretary of War during the period March-July 1933, Joint Board #319, Serial 516. RG 220, NA.
39. Interview Prescott H. Currier, Captain USN (Ret), 14 November 1980, by Robert Farley and Henry Schorreck, OH 39-80 (classified).
40. SRH-149, SRH-305, RG 457, NA.
41. OP-20-G memorandum Serial 051220 dated 25 July 1940, "Coordination of Intercept and Decrypting Activities of the Army and Navy," Series VII.18, Box 4, 19, CCH History Collection (classified).
42. See 14 February 1946 memorandum for OP-20-4 from Captain L.F. Safford, "Responsibility for Decoding and Translating Japanese Intercepts," Series VII, Box 4, 19, CCH History Collection (classified).
43. "History of JNA 20" (CORAL), Vol. I. NSA Cryptologic Archival Holding Area (classified).
44. SRH-305, RG 457, NA, and SRH-149, RG 457, NA.
45. This code was stolen without the knowledge of the Japanese soon after its introduction in 1939. "History of OP-20-GYP," 2, Series IV.W.I.5.12, CCH History Collection (classified).
46. "History of OP-20-3-GYP," 2. Series IV.W.I.5.12, CCH History Collection (classified).
49. "Military Study Communication Intelligence Research Activities," United States Navy, 30 June 1937, SRH-151, RG 457, NA.
50. History of OP-20-GYP-1, Series IV.W.I.5.13, CCH History Collection (classified).
51. SRH-149, RG 457, NA; see also 7 December 1929 memorandum to DNI from DNC, "Radio Intelligence." At that time the main purpose of the Research Desk was deciphering Japanese codes, Series III.G.9, CCH History Collection; Japanese traffic volumes, Howe monograph, Early Background of the U.S. Cryptologic Community, Series VII.15, CCH History Collection; Army-Navy DOE, 1931, Navy proposed to divide diplomatic communications based on availability of intercept and Naval power, Series VII.19, Box 4, Vol. I pre-1942, CCH History Collection (classified); Japanese Blue Book introduced February 1931, SRH-305, RG 457, NA.
52. "History of OP-20-GYP-1," Series IV.W.I.5.13, CCH History Collection. See also Edwin T. Layton, And I Was There, (New York: William Morrow and Co., 1985), 77.
53. "History of OP-20-GYP-1," Series IV.W.I.5.13, CCH History Collection (classified).
55. Ibid.; see also Layton, And I Was There, 77-78, 249.
56. Ibid.; see also "History of OP-20-GY Series IV.W.I.5.10-12:13, CCH History Collection (classified).
57. "History of OP-20-GYP-1" (classified).
22 June 2001