though even there, as we shall see, it has limited usefulness and great dangers. In exploratory development and research the precise identification of objectives and scheduling are less important than trying to cover all good bets, selecting first-rate scientists and productive laboratories, promoting competition, and preserving flexibility to follow up vigorously on break-throughs. SOME COMMON PITFALLS TOO LITTLE “DUPLICATION” One of the most important and obvious corollaries of the uncertain character of research and development is the desirability of some duplication. (Here, as is common in discussions of research and development, "duplication" means diversification or pursuit along several routes.) If, as is typically the case, there are two, three, or a dozen possible paths to some research and development objective, each with supporters and detractors, it will frequently pay to try two or more, not just one, which may fail or take an inordinately long time. In any rationally conducted program there should be duplication in this sense, and there will inevitably occur what appears with the aid of hindsight to be waste the effort devoted to the unsuccessful (or less successful) paths." But how much diversification and where? No one knows enough to give precise answers. Some original and suggestive theoretical analysis. indicates that in many circumstances there are great gains from pursuing two, three, or four paths, but rapidly diminishing returns from further duplication. Not only does the use of several routes buy time; it also, up to a point, actually saves money! The optimal amount of duplication in any particular situation is sensitive to the following factors: 8 1. There should be more duplication, the greater the expected payoff from the research. In the Manhattan District Project, six completely distinct and independent methods of separating fissile material were under development concurrently; if the expected payoff had been less, fewer of these expensive "duplicating" projects could have been justified. Incidentally, the method that succeeded in producing the material for the first bomb was regarded at first as among the least promising. There is a strong case for some duplication in the development of critical weapon systems, despite their great cost, because of the disastrous consequences if the one horse that we back runs last. For an excellent description of the uncertainties in R and D and the importance of "duplication," see Ely Devons, "The Aircraft Industry," in Duncan Burn (ed.), The Structure of British Industry, Vol. 2, Cambridge University Press, Cambridge, England, 1958, pp. 54-69. For an ingenious attempt to quantify the role of these factors, see Richard R. Nelson, "The Economics of Parallel R and D Efforts: A Sequential-Decision Analysis,” The RAND Corporation, RM-2482, November 12, 1959. 2. There should be more duplication, the greater the uncertainties. If, in developing a missile, the airframe is straightforward engineering, while there appear to be great uncertainties in developing the guidance, the important thing is to try several approaches to the guidance problem - perhaps completely independent approaches using competing con tractors. 3. There should be more duplication, the cheaper it is to duplicate. More weapon systems should be developed than are ultimately procured (it may have made sense to develop two intermediate-range ballistic missiles - Thor by the Air Force and Jupiter by the Army-yet not to buy quantities of both for operational use). There should be more duplication in the development of difficult or critical components than in the development of whole weapon systems. And there should be. most duplication in exploratory development and research, where the cost of trying another path or testing it is usually a tiny fraction of prototype fabrication costs. As a result of factors (2) and (3), we want a great deal of "temporary" duplication. Later, as we get more information and reach more expensive stages of development, we want to decrease the number of routes to be followed. 4. There is a stronger case for duplication if the alternatives are qualitatively different, and if the factors that will determine their success or failure are independent. TOO LITTLE COMPETITION This is not quite the same thing as too little duplication, although the two are closely related in practice. Duplication is possible without competition - for example, if the multiple paths are explored by the same organization or individual. And competition is possible without duplication. (The Army and Navy would compete for budget even if they never developed similar kinds of hardware.) Competition provides a spur that is frequently lacking in the research and development world of cost-plus contracts and scheduled progress. Some dedicated individual researchers do not require it, but there is nothing so stimulating to most people and organizations engaged in research and development as the fear that another company or laboratory will beat it to the objective." Even competition among the military Services has proved highly beneficial. Robert Schlaifer concluded as follows from his study of the development of aircraft engines: The American system of administering development did, however, have one feature greatly superior to both the British and the German systems. This was 'It helps significantly if some real monetary prize is awarded the winner-like a production contract. the sponsorship of development by two separate agencies, the Army and the Navy. The first and most obvious result of a separate organization for naval aviation was, of course, that development specially directed at the needs of naval aviation was not slighted in the United States as it was abroad. In addition, however, the existence of two independent agencies meant that the mistakes or omissions of one were corrected in a surprisingly large number of instances by the actions of the other. Whatever may be the merits of the case for unification of the military services in other respects, there can be no doubt that the sponsorship and direction of development by two separate agencies brought results worth very much more than the cost.10 Unfortunately much of the competition between companies engaged in research and development is now concentrated at the stage of preliminary design. It has become a common practice for the military Services to invite industry to submit proposals (in the form of preliminary designs on paper) for the development of new weapon systems, and then to select one (or at most two) of these proposals for further support. As a result, there is intense competition among the companies involved in producing the most glamorous and promising drawings. But too often this kind of competition is a competition in optimism and exaggeration. The company that exaggerates more gets the contract. Competition is effective in promoting progress only if the consumers are well enough informed to make rational selections; and there is abundant evidence that no one-administrator or scientist is good at picking winners at the preliminary design stage. On the other hand, judging the performance of actual hardware - even test models is relatively easy and reliable. Exaggerated claims can be debunked on the test stand. Few things would have a more salutary effect on the efficiency of research and development than shifting some of the competition that now takes place at paper stages forward to early hardware stages. There are problems in doing this at reasonable cost, but longer-run savings justify a major effort to solve them. - PREMATURE, OPTIMISTIC, AND TOO DETAILED REQUIREMENTS FOR ADVANCED SYSTEMS Wishful thinking stimulated by the claims of contractors and enthusiasts, neglect of the uncertainties, and a natural desire on the part of responsible military officials to have the research and development establishment produce on order what they consider necessary for defense, have sometimes resulted in a tendency for the Services to specify their requirements for advanced weapon systems prematurely, optimistically, and in too great detail. Typically this kind of overspecification is 10 Robert Schlaifer, Development of Aircraft Engines, in R. Schlaifer and S. D. Heron, Development of Aircraft Engines and Fuels (Boston, Division of Research, Harvard Business School, 1950), p. 1I. attempted following the submission of preliminary designs — or even earlier. Studies have been made of the reliability of early estimates of (1) cost, (2) development time, and (3) physical and performance characteristics of new, advanced weapon systems. The results are striking. In the case of estimates made at the preliminary design stage, costs are typically underestimated by factors of 2, 5, 10, or in some cases substantially more. Development time is typically underestimated by I to 5 years. And the final system usually differs markedly from the predicted system both in physical and performance characteristics.11 Usually, although not quite always, the estimates are optimistic — a bias one could expect from the premium on exaggeration by contractors in design competitions. But more serious than the bias (which could be discounted if it were systematic) is the wide range and scatter of the estimates. Choices at this stage must recognize these uncertainties. Precise technical and performance specifications at this stage involve self-delusion, Premature specification makes it inevitable that many changes in specifications will have to be made as the system is developed. Detail also multiplies the number of changes in specifications that will be required, and obstructs the objective of getting a working model quickly into test. Excessive optimism in drawing up performance specifications can make the development so difficult that it must fail, or take much longer and cost much more than planned, or require a downgrading of the requirements.12 It is not unusual for weapon system requirements to be so optimistic that several inventions or advances in the state of the art are needed on schedule if the development is to succeed. Prior to the successful test of the H-bomb, the development of the Air Force's first ICBM had been thwarted by requirements for payload and accuracy that proved infeasible. Only when these requirements were substantially relaxed after the successful H-bomb test could development proceed to the point where vehicles could be tested. The need for changes in specifications would not matter so much if "For example, more than half the aircraft developed since World War II ended up using different engines from those envisaged at the preliminary design stage. Aircraft expected to be supersonic turn out to be subsonic. Nuclear weapons expected to have a "yield" of x in fact have yields of 3x-or .ix, and so on. See A. W. Marshall and W. H. Meckling, "Predictability of the Costs, Time, and Success of Development," The RAND Corporation, P-1821, December 11, 1959. 13 We will suggest later that instead of trying to specify all performance characteristics well in advance (x miles per hour, y C.E.P., z range, etc.), which places an intolerable strain on the art of prediction, the military Services would be well advised to indicate limiting values of such characteristics and acceptable trade-offs between them (in short, their "preference surfaces”). 68138 0-61 -5 they could be made quickly and easily as the need became apparent. But it is a fact of life in large bureaucracies that decisions are difficult to unmake and remake and that the process is time consuming. The military research and development establishments are not an exception to this general rule. As a result much time is lost in obtaining authorization for changes in specifications through all the echelons of authority. Even more time is lost in attempting to find a way around obsolete or excessively optimistic specifications to obviate the necessity of appealing through the echelons of the hierarchy for new decisions. An important reason for early, fairly detailed specification of weapon systems is the need to match the various components under development - for example, in the case of a missile, the airframe, power plant, warhead, and guidance. This need is real, and must be achieved at the appropriate stage. Where a system is being put together from previously developed and tested components (as was the Air Force's Thor), matching and the detailed specifications required by matching may be imposed without too much risk at the preliminary design stage. But where a new system is really advanced, where the components have yet to be developed and tested for feasibility and performance, premature concern physical matching can delay development by years. The urgent thing is usually to get the critical components developed to the point where they can be tested (having “duplicate” efforts on the most critical ones to gain quality, time, or both). When it is known that they work is early enough to worry about matching configurations in detail. PREMATURE COMMITMENT OF LARGE FUNDS The illusion that the future can be foreseen with something approaching certainty, plus pressure from the contractor, plus a desire by the military Services to save time and to guarantee the availability of a budget to complete the development, produce a tendency to commit large funds prematurely to highly uncertain weapon system developments. Because of the great uncertainties that affect technological development, and the equally great uncertainties regarding the military usefulness of particular developments, it is desirable to retain the flexibility to terminate developments at short notice and low cost. One type of premature commitment is the commitment to production tooling at an early stage of the development. There are exceptional cases where some commitment may be justified at an early stage, but most of the tooling ordered before the testing of prototypes has to be scrapped because of changes in specifications. This appears to be the sort of gamble where you gain little if you win (six months of lead time at most except in periods of heavy pressure on the capacity of tool producers); lose a |