With outputs of so many different kinds, it is clear that information requirements must necessarily be massive. Intelligence has long since ceased to be a game of spy versus spy and has become a field in which almost every imaginable source is plumbed for its contribution to the whole. Alongside the secret agents are ranged what former CIA director Richard Helms called in a 1967 speech the "machine spies." The raw information is massaged by analysts and value is added to it by careful comparison, review, and fitting together what are, in effect, jigsaw puzzles. Intelligence authorities are loath to discuss these "sources and methods," but the outlines of the process are readily apparent and an understanding of this process is important to according intelligence its correct place in American foreign policy.
Until the early nineteenth century, spies remained virtually the only source of intelligence, while intelligence organizations were most notable by their absence. Both organizations and technical means of intelligence collection began to enter the field about the time of the American Civil War. Scientific inventions drove these changes as the twentieth century opened, World War I demonstrated a continuing need for intelligence, and World War II provided a huge impetus for all kinds of intelligence collection methods. The coherent theory of intelligence discussed in this essay also began to come into focus with World War II. The sophistication of all techniques has improved constantly since then, and the Cold War served to spotlight evolving methods. Intelligence is now characterized by a synergistic dynamic in which information requirements drive technological improvements, while scientific talent solves collection problems, thus making new information important and creating fresh requirements.
Agents Spying has been characterized as "the second oldest profession," in the sense that its attempts to divine an adversary's intentions and capabilities are recorded throughout history. Agents remained important even in the era of the machine spy, for some kinds of information cannot be gathered by technical collection means. In the post–Cold War era, with an intelligence focus dominated by terrorism, drug trafficking, international criminal activity, and concerns regarding the proliferation of weapons of mass destruction, much of the key thinking and decision making of adversaries took place in a manner more accessible to the spy than to the machine. As cost became a factor as well, it was evident that in many cases spies were cheaper than machines, even with the extensive networks of the CIA and other agencies for the care and feeding of spies. At the beginning of the twenty-first century, spies were the province of the CIA's Directorate of Operations and the Defense Human Intelligence Service of the Defense Intelligence Agency. There were continuing conflicts, to some degree inherent in the nature of this kind of activity, between employment of spies to gather intelligence and their role in covert action, political action, or other fields. In addition, there were difficulties from a human rights perspective, because individuals recruited as agents frequently had checkered pasts. Only in the 1990s did the United States establish standards for personal character in individuals recruited as agents, and these were sometimes honored more in the breach than in the observance. Spies remained a necessary evil.
Individuals to be recruited were often identified by a third party and sometimes solicited by them as well. All aspects of the agent's relationship with intelligence were handled by a case officer, a CIA (or other agency) person who was not himself or herself the spy. The case officer typically reported to the CIA station in the country in which the espionage took place, though in exceptional cases a relationship might have run directly to headquarters. Material from agents was rewritten by "reports officers" before being circulated to analysts who used it to compile finished intelligence of the kinds noted already. Some policymakers, for example national security adviser Zbigniew Brzezinski during the Carter administration, insisted upon being shown the raw agent reports.
Attachés The diplomatic service assigns individuals to posts in foreign capitals as commercial, economic, and cultural attachés. These posts are often used by the CIA and other agencies to provide a cover occupation for intelligence officers overseas. More important, the military services openly send officers to foreign countries as attachés. These officers provide a channel of communication between the U.S. military and foreign services, provide occasional diplomatic assistance, and more or less openly gather intelligence. The practice of sending military attachés in general dates from the late eighteenth century and was adopted as a standard by Napoleonic France (1799–1815). In the United States, the army colonel Emory Upton conducted a two-year research visit to Europe and Asia in the 1870s and demonstrated the value of military officers gathering information abroad.
The U.S. Navy also conducted such roving visits beginning in 1870. Naval attachés (the Marine Corps shares in naval attaché assignments) have been permanently stationed in foreign nations since 1882, army attachés since 1889, and air force attachés (first as part of the Army Air Corps) since the 1930s. The Defense Intelligence Agency (DIA) administers the attaché program, using officers seconded from all the armed services. Reports go to DIA, where they are used in publications and circulated to other intelligence agencies. Attachés can provide considerable amounts of information, even from closed societies. For example, in 1954 an air attaché in Moscow furnished the initial intelligence on a Soviet heavy jet bomber. Similarly, U.S. naval attachés in Japan before World War II reported on Japanese technological improvements, including early data on oxygen-powered torpedoes, the highly capable Zero fighter, the large-caliber guns of the Yamato-class superbattleships, and other matters. Military attachés in Czechoslovakia (now the Czech Republic) during that same period sent home valuable material on Soviet military doctrine and organization.
Combat Intelligence Often ignored, combat intelligence is acquired by military forces in the course of their activities, and not only in wartime. A wide variety of intelligence can be encompassed by this category, which includes things learned by scout patrols sent out by ground forces, data from interrogation of prisoners (and defectors), lucky finds by forces in the field, and observations made during normal operations. Information from defectors proved a fruitful source during the Cold War, and would merit its own category except that interrogation of prisoners—the same function—fits more naturally into this category.
A few examples illustrate this category nicely. Union armies in the Civil War made a lucky find in September 1862 when a copy of General Robert E. Lee's orders to his Army of Northern Virginia was discovered wrapped around some cigars shortly before the Battle of Antietam. In World War II, German prisoners taken before the Battle of the Bulge (December 1944) revealed details of preparations for an attack that were not taken seriously, leading to surprise of U.S. troops along the front. Beginning in March 1978, in the course of normal operations, the submarine USS Batfish detected and followed a Soviet Yankee-class ballistic missile submarine for some seventy-seven days, thereby gaining key information about the practices and habits of these Soviet strategic forces. In 1983 information from the Russian intelligence officer and defector Oleg Gordievsky proved critical in interpreting a series of Soviet moves indicating they believed nuclear war might be at hand. The case of the war scare of 1983 also shows directly how this kind of intelligence can be reflected in analytical reports, since Gordievsky's information was incorporated in a special NIE done in early 1984.
Scientific Intelligence Scientific intelligence is gleaned from study of technical or scientific documents (often research papers) produced by the intelligence target, or from direct examination of equipment or machinery captured from an enemy in wartime or somehow acquired during peace. Scientific intelligence reached a takeoff point during World War II when the effort to gather information in this fashion first became systematic. Notable examples from that period include British successes in countering radio navigation systems used by German bombers in the Battle of Britain (1940), and in deceiving the Germans as to the accuracy of their rockets and ramjets (V-weapons) launched against Great Britain in 1944–1945. A U.S. example is the capture and analysis of the Japanese Zero fighter aircraft, results of which were incorporated into the design of a countering aircraft, the Grumman F6F Hellcat. A Cold War example is how the United States learned of Soviet aircraft design techniques from examining a MiG-25 supersonic aircraft whose pilot flew it to Japan to defect in 1977.
Electronic Intelligence Related to scientific intelligence in that it is also analyzed by scientists, electronic intelligence, too, received a great boost in World War II. It involves the reception and recording of electronic emissions, usually from radars, or of the telemetry transmissions from guided missiles, ships, or aircraft undergoing testing. Scientists are able to deduce from this information the radio frequency bands and other characteristics of radars, or a variety of information about systems being tested. Submarines, surface ships, aircraft, and space satellites have all been used to gather electronic intelligence. From the 1940s until at least the 1980s the United States maintained a vigorous program of flights by aircraft equipped to collect electronic intelligence along the periphery of the Soviet Union, China, and other communist countries. These flights, known in the trade as "ferret" operations, numbered over 20,000 and account for the vast majority of the planes shot down in the course of Cold War spy flights. In August 1964 the American destroyers involved in the Gulf of Tonkin incident in the Vietnam War were on an electronic intelligence mission. The same was true of the ship Liberty, attacked by the Israeli air force during the Six-Day War (June 1967), and the ship Pueblo, captured by North Korea in January 1968. Similarly, a U.S. Navy EP-3E reconnaissance plane off the southern Chinese coast became involved in an incident on 1 April 2001, when a jet fighter of the People's Republic of China subjected the American plane to such intense harassment that the two craft collided. The Chinese aircraft was lost, and the American plane had to make a forced landing in China, where the crew was held in custody for eleven days, until the United States made a formal statement that could be taken as an apology. In general, gathering electronic intelligence has been among the most dangerous kind of intelligence missions, and has provoked serious diplomatic repercussions as well. On the other hand, electronic intelligence can be quite useful. An example can be taken from 1978, when telemetry data from Russian missile tests indicated that a certain Russian ICBM was being tested with more reentry vehicles than it was credited with by diplomats negotiating the SALT II treaty. In this case the result was to negotiate new definitions for the treaty on how to count missiles for inclusion in the categories allowed by the treaty.
Communications Intelligence Communications intelligence is a large subject that includes secret writing, codes and ciphers, transmission and interception, and decryption. All these are aspects of the process of gaining access to, and then knowledge of the contents of, the private communications of a target, whether an individual or a state. This type of intelligence has probably existed as long as there have been spies, given the advantages of knowing what a spy was reporting. Government messages have long been sent in code, and breaking those codes offers knowledge of the inner thoughts of the opponent. In modern usage, messages are most often sent by radio, cable, teletype, E-mail, or other forms of electronic transmission. These communications are subject to interception, and this source of intelligence is considered among the best (and most sensitive, from a security standpoint). The general label of "communications intelligence" used here encompasses all aspects of contriving to intercept messages, analysis of the transmissions, decryption and decoding of the contents, and translation and making the results available to friendly intelligence analysts. In the United States each of the armed services has a component dedicated to communications intelligence, and all feed material to an umbrella civilian unit, the National Security Agency (NSA). With increasing technical sophistication the personnel requirements for collection of radio transmissions have decreased—the armed services alone employed roughly 120,000 people for this purpose in the 1960s, but the number went down to about half of that by 2000. The NSA employs approximately 20,000 people. With much of the burden of collection switched to satellites, ground stations have been abandoned in Pakistan (1969), Ethiopia (around 1975), Subic Bay in the Philippines (1985), and Berlin (about 1995). In 2000 there were still ground stations in Turkey, Japan, South Korea, and, by means of liaison relationship, in China. Diplomacy and foreign aid required to maintain communications intelligence ground stations have been a complicating factor in U.S. foreign policy.
Special collection operations are sometimes conducted by the CIA or other agencies for the benefit of the NSA. Best known of these are the tunnels built in Vienna (1949) and Berlin (1945–1955), and under the Russian embassy in Washington, D.C. (1985–?), for the purpose of placing listening devices and taps on telephone or teleprinter cables. Diplomatic fallout adverse to the United States occurred when these operations were revealed. In almost every case the target became aware of the special collection activity before its public revelation, and used that knowledge to feed false information to U.S. intelligence. Another special activity involved the use of submarines to secretly place taps on telephone cables underwater in the Russian Far East.
Communications intelligence works like a huge vacuum cleaner, and for all of its difficulties has proved a highly valuable intelligence source. Communications provided key information on the structure and activities of Soviet armed forces; important insights into negotiating aims of Soviet-American arms control talks from the 1970s to the 1990s; data on Chinese involvement in the Korean War; vital information on North Vietnamese commands and on the Ho Chi Minh Trail during the Vietnam War; material on Soviet maneuvers in various international crises; knowledge of the activities of drug cartels and others using telephone communications from the 1980s on; and much else. Communications intelligence played a huge role in World War II, a significant role in World War I, and evolved continuously from the moment when Morse code telegraphy began in the 1850s.
Photographic Intelligence Images taken with a camera have been an intelligence source since World War I. Observation from above ground level increases the scope of this intelligence, and aerial observation has been in use since a French officer used a balloon to watch the Austro-German enemy at the Battle of Fleurus in 1794. In the American Civil War balloons were used in this manner, and the first recorded instance of aerial photography dates from that time. The airplane was an important scout in World War I, with cameras soon added, and purpose-built cameras and aerial photography reached a stage of advanced development in World War II. During the Cold War the U.S. Air Force used specialized reconnaissance squadrons for overhead photography. In the mid-1950s the CIA joined the effort, first with high-altitude photo planes (the U-2 and SR-71) and then space satellites (CORONA, or KH-4, with follow-ons, currently up to KH-12). Sophisticated films, cameras, computer-driven digital readout techniques, and database-linked photo interpretation techniques have made overhead reconnaissance a premier intelligence source. The National Reconnaissance Office, formed in 1960 but whose existence was admitted only in 1992, controls the production and operation of satellite systems (aircraft reconnaissance systems known today remain under the auspices of the air force). Interpretation of imagery and circulation of intelligence reports based on photographic intelligence is the province of the National Imagery and Mapping Agency, formed in 1995 from components merged from the CIA and the Pentagon.
Some photography other than overhead reconnaissance is also important to intelligence. Submarines have taken pictures through their periscopes of coastal targets ranging from invasion beaches to adversaries' naval bases. Casual pictures taken by private citizens ("Aunt Minnie photographs") may show objects of intelligence interest. In addition, a quite critical contribution through photography is made by the miniature cameras that secret agents use to surreptitiously photograph documents, and the microdot and other photographic methods used to assist the communication of secrets from agent to handlers. These aspects of photography, and related research and development to create the cameras, are the responsibility of the CIA's directorates of administration, science and technology, and operations.
Domestic Collection Intelligence services acquire some information directly from Americans or from foreign citizens who are not agents or spies in any traditional sense. An individual traveling to a foreign country might be asked about impressions gained during the visit. Journalists, missionaries, and others residing abroad have frequently been a source for well-grounded local information. Academics expert in a certain field can provide information or perspectives of which agency personnel are unaware. Business-people may have valuable contacts and knowledge. At times, such as the period of the communist Union of Soviet Socialist Republics, this kind of collection from individuals was such a significant source that the CIA made efforts in advance to ask persons traveling to that country to be alert for information on specific matters. This technique remains useful in closed societies. At one time the CIA maintained the Domestic Contact Service to gather this information. The function still exists, although the work of that office has been subdivided elsewhere within the agency.
Open Source Information Evaluations of sources of intelligence consistently find that perhaps 80 to 90 percent of information necessary to intelligence can be obtained from sources that are completely public. The term "open sources" has become current for this category of material, which includes newspapers, magazines, technical journals, scientific papers, books, video programs, information that appears on the Internet, and similar items. Since the mid-1990s there has been a more explicit effort to improve the collection and use of open source material. For a short time the CIA contemplated a major directorate for open sources, but eventually settled on the Office for Open Source Collection, which today is located within the Directorate for Science and Technology. Open source information is highly desirable in that its collection involves no political or diplomatic obstacles, often includes material that already has been analyzed to some extent, and is relatively inexpensive compared to data gathered from technical collection sources. A significant drawback lies in the fact that intelligence officers and the policymakers who rely upon them often attribute greater credibility, almost automatically, to information that is "born secret," simply because it is secret.
Liaison with Friendly Intelligence Services Foreign intelligence services with which the United States is allied or maintains private relations are significant sources of information. The best known of these relationships is that with the British, forged during World War II and regularly renewed since. Similar relationships exist with Canada, Australia, New Zealand, Japan, France, Germany, Italy, Turkey, Greece, Israel, and other nations. Some relationships are multilateral and general in nature, similar to military alliances. Others pertain to specific subjects. Some relations exist even with former enemies, such as between the United States and Russia on organized crime. Sharing of intelligence information varies with the degree of trust—for example, the United States shares a wide range of information with Great Britain, but much less with France. The use of the CIA as an intermediary, such as between Israel and the Palestinian Authority from 1998 to 2001, can also have diplomatic aspects. The question of liaison relationships is also complicated by efforts of the partners to spy on one another. United States–Israeli intelligence cooperation soured after 1985, when naval intelligence analyst Jonathan J. Pollard was revealed as an Israeli agent. More recently, French–U.S. relations have been complicated by allegations that the nations engaged in industrial spying against one another, a situation that led to the recall of a CIA station chief in Paris in February 1995. Interests in preserving intelligence relationships may factor in diplomatic initiatives in many instances.
Counterintelligence The work of uncovering foreign spies in one's own country or intelligence services is termed counterintelligence or counterespionage. This work predates creation of the Office of Strategic Services in World War II, having been a major focus of U.S. naval intelligence even during the prewar years. From its inception the CIA had a counterintelligence staff within the Directorate of Operations; in recent years this has risen to the status of a center combining both CIA and FBI resources and headed by an FBI official. Counterintelligence can be a double-edged sword. The knowledge and access to secret information of counterintelligence units made them a favorite target for the Soviet secret service during the Cold War, as epitomized by the Russian recruitment of British counterintelligence chief Harold ("Kim") Philby, who, after years of suspicion, fled to Russia in early 1963.
The existence and duration of counterintelligence investigations can also have a negative impact on an intelligence organization. During the 1960s, when James J. Angleton headed the CIA's counterintelligence staff, inquiries aimed at suspected (but innocent) agency officers came close to paralyzing the CIA's foreign intelligence effort against the Soviet Union. Some observers argue that the reaction to the excesses of the 1960s was to dismantle much of the counterintelligence effort, leading to a permissive climate in which espionage flourished—thus "the year of the spy" in 1985, during which half a dozen major spies for Russia or China were uncovered, and the cases of Aldrich Ames (1994) and Robert Hanssen (2001), each of whom succeeded in spying for Russia for at least a decade without being caught.
Counterintelligence failures like these invariably lead to more stringent security regulations and regular monitoring and reinvestigation of veteran intelligence officers. In the United States there are demands for the broad use of polygraph (lie detector) tests, whose utility and accuracy are disputed. The net effect of heightened security, aside from catching spies, is that at some level the work of everyone engaged in intelligence becomes more difficult due to the compartmentalization, degree of precaution, and breadth of disclosure to security officials. In turn, this makes personnel retention a problem. In addition, stringent counterintelligence regimes inhibit the recruitment of spies in the adversary's camp because of the fear that the agents are enemy personnel deliberately attempting to mislead the side recruiting them. Consequently there is a trade-off between efficiency of intelligence work (with danger of espionage as a result) and tight security (with the consequence of lowered morale and performance from intelligence officers). The United States has not yet been able to solve this equation and has vacillated between extremely permissive and tight regimes.
Leaks of information owing to espionage damage the credibility of intelligence, with both policymakers and the public; relations with cooperating intelligence services (which fear information they share may end up in the hands of enemies); and the cohesion of the intelligence community. Counterintelligence is necessary but poses a continuing dilemma.