linguistics as well as in the recommended reduction of support aimed at relatively short-term ... and should be supported are (1) basic developmental research in computer methods for handling ...... 365 min or 13 percent. Typing. 645 min or 24 ...
LANGUAGE AND MACHINES COMPUTERS IN TRANSLATION AND LINGUISTICS
A Report by the Automatic Language Processing Advisory Committee Division of Behavioral Sciences National Academy of Sciences National Research Council
Publication
National Academy of Sciences
1416
National Research Council Washington, D. C.
1966
Printing and Publishing Office National Academy of Sciences 2101 Constitution Avenue Washington, D.C. 20418 Price: $4.00 Library of Congress Card Catalog Number: 66-61843
August 20, 1965
Dear Dr. Seitz: In April of 1964 you formed an Automatic Language Processing Advisory Committee at the request of Dr. Leland Haworth, Director of the National Science Foundation, to advise the Department of Defense, the Central Intelligence Agency, and the National Science Foundation on research and development in the general field of mechanical translation of foreign languages. We quickly found that you were correct in stating that there are many strongly held but often conflicting opinions about the promise of machine translation and about what the most fruitful steps are that should be taken now. In order to reach reasonable conclusions and to offer sensible advice we have found it necessary to learn from experts in a wide variety of fields (their names are listed in Appendix 21). We have informed ourselves concerning the needs for translation, considered the evaluation of translations, and compared the capabilities of machines and human beings in translation and in other language processing functions. We found that what we heard led us all to the same conclusions, and the report which we are submitting herewith states our common views and recommendations. We believe that these can form the basis for useful changes in the support of research aimed at an increased understanding of a vitally important phenomenon—language, and development aimed at improved human translation, with an appropriate use of machine aids. We are sorry that other obligations made it necessary for Charles F. Hockett, one of the original members of the Committee, to resign before the writing of our report. He nonetheless made valuable contributions to our work, which we wish to acknowledge. Sincerely yours, J. R. Pierce, Chairman Automatic Language Processing Advisory Committee Dr. Frederick Seitz, President National Academy of Sciences 2101 Constitution Avenue Washington, D.C. 20418
July 27, 1966
Dear Dr . Seitz: In connection with the report of the Automatic Language Processing Advisory Committee, National Research Council, which was reviewed by the Committee on Science and Public Policy on March 13, John R. Pierce, the chairman, was asked to prepare a brief statement of the support needs for computational linguistics, as distinct from automatic language translation. This request was prompted by a fear that the committee report, read in isolation, might result in termination of research support for computational linguistics as well as in the recommended reduction of support aimed at relatively short-term goals in translation. Dr. Pierce’s recommendation states in part as follows: The computer has opened up to linguists a host of challenges, partial insights, and potentialities. We believe these can be aptly compared with the challenges, problems, and insights of particle physics. Certainly, language is second to no phenomenon in importance. And the tools of computational linguistics are considerably less costly than the multibillion-volt accelerators of particle physics. The new linguistics presents an attractive as well as an extremely important challenge. There is every reason to believe that facing up to this challenge will ultimately lead to important contributions in many fields. A deeper knowledge of language could help: 1. To teach foreign languages more effectively. 2. To teach about the nature of language more effectively. 3. To use natural language more effectively in instruction and communication. 4. To enable us to engineer artificial languages for special purposes (e.g. pilot-to-control-tower languages). 5. To enable us to make meaningful psychological experiments in language use and in human communication and thought. Unless we know what language is, we don't know what we must explain. 6. To use machines as aids in translation and in information retrieval... However, the state of linguistics is such that excellent research that has value in itself is essential if linguistics is ultimately to make such contributions. Such research must make use of computers. The data we must examine in order to find out about language is overwhelming both in quantity and in complexity. Computers give promise of helping us control the problems relating to the tremendous volume of data, and to a lesser extent the problems of data complexity . But we do not yet have good, easily used, commonly known methods for having computers deal with language data.
Therefore, among the important kinds of research that need to be done and should be supported are (1) basic developmental research in computer methods for handling language, as tools to help the linguistic scientist discover and state his generalizations, and as tools to help check proposed generalizations against data; and (2) developmental research in methods to allow linguistic scientists to use computers to state in detail the complex kinds of theories (for example, grammars and theories of meaning) they produce, so that the theories can be checked in detail. The most reasonable government source of support for research in computational linguistics is the National Science Foundation. How much support is needed? Some of the work must be done on a rather large scale, since small-scale experiments and work with miniature models of language have proved seriously deceptive in the past, and one can come to grips with real problems only above a certain scale of grammar size, dictionary size, and available corpus. We estimate that work on a reasonably large scale can be supported in one institution for $600 or $700 thousand a year. We believe that work on this scale would be justified at four or five centers. Thus, an annual expenditure of $2.5 to $3 million seems reasonable for research. This figure is not intended to include support of work aimed at immediate practical applications of one sort or another.
This recommendation, which I understand has the endorsement of Dr. Pierce's committee, was also sent out for comment to the membership of the Committee on Science and Public Policy. While the Committee on Science and Public Policy has not considered the recommended program in computational linguistics in competition with other National Science Foundation programs, we do believe that Dr. Pierce's statement should be brought to the attention of the National Science Foundation as information necessary to put the report of the Advisory Committee in proper perspective. Sincerely yours, Harvey Brooks, Chairman Committee on Science and Public Policy Dr. Frederick Seitz, President National Academy of Sciences 2101 Constitution Avenue Washington, D.C. 20418
In computational linguistics and automatic language translation, we are witnessing dramatic applications of computers to the advance of science and knowledge. In this report, the Automatic Language Processing Advisory Committee of the National Research Council describes the state of development of these applications. It has thus performed an invaluable service for the entire scientific community. Frederick Seitz, President National Academy of Sciences
AUTOMATIC LANGUAGE PROCESSING ADVISORY COMMITTEE John R. Pierce, Bell Telephone Laboratories, Chairman John B. Carroll, Harvard University Eric P. Hamp, University of Chicago* David G. Hays, The RAND Corporation Charles F. Hockett, Cornell University † Anthony G. Oettinger, Harvard University Alan Perlis, Carnegie Institute of Technology
STAFF
A. Hood Roberts, Executive Secretary Mrs. Sandra Ferony, Secretary
* Appointed February 1965 † Resigned December 1964
Preface
The Department of Defense, the National Science Foundation, and the Central Intelligence Agency have supported projects in the automatic processing of foreign languages for about a decade; these have been primarily projects in mechanical translation. In order to provide for a coordinated federal program of research and development in this area, these three agencies established the Joint Automatic Language Processing Group (JALPG). Early in its existence JALPG recognized its need for an advisory committee that could provide directed technical assistance as well as contribute independent observations in computational linguistics, mechanical translation, and other related fields. In October 1963 the Director of the National Science Foundation, Leland J.Haworth, requested on behalf of the three agencies that the National Academy of Sciences establish such a committee. This was done, and in April 1964, with funds made available by the three agencies, the Automatic Language Processing Advisory Committee of the National Academy of Sciences—National Research Council, under the chairmanship of John R. Pierce, held its first meeting. The Committee determined that support for research in automatic language processing could be justified on one of two bases: (1) research in an intellectually challenging field that is broadly relevant to the mission of the supporting agency and (2) research and development with a clear promise of effecting early cost reductions, or substantially improving performance, or meeting an operational need. It is clear to the Committee that the motivation for support of much of the work in automatic language processing has been the practical aim represented in (2) above. In the light of that objective, the Committee studied the whole translation problem. This report presents the findings and recommendations of the Committee.
Contents Human Translation
1
Types of Translator Employment
2
English as the Language of Science
4
Time Required for Scientists to Learn Russian
5
Translation in the United States Government Number of Government Translators Amount Spent for Translation
6
7
9
Is There a Shortage of Translators or Translation? Regarding a Possible Excess of Translation The Crucial Problems of Translation
11
13
16
The Present State of Machine Translation
19
Machine-Aided Translation at Mannheim and Luxembourg
25
Automatic Language Processing and Computational Linguistics 29 Avenues to Improvement of Translation Recommendations
32
34
APPENDIXES 1.
Experiments in Sight Translation and Full Translation
2.
Defense Language Institute Course in Scientific Russian
3.
The Joint Publications Research Service
4.
Public Law 480 Translations
41
39
35 37
5.
Machine Translations at the Foreign Technology Division, U.S. Air Force Systems Command 43
6.
Journals Translated with Support by the National Science Foundation 45
7.
Civil Service Commission Data on Federal Translators
8.
Demand for and Availability of Translators
9.
Cost Estimates of Various Types of Translation
10.
An Experiment in Evaluating the Quality of Translations 67
11.
Types of Errors Common in Machine Translation
76
12.
Machine-Aided Translation at the Federal Armed Forces Translation Agency, Mannheim, Germany
79
50
54 57
13.
Machine-Aided Translation at the European Coal and Steel Community, Luxembourg 87
14.
Translation Versus Postediting of Machine Translation
15.
Evaluation by Science Editors and Joint Publications Research Service and Foreign Technology Division Translations 102
16.
Government Support of Machine-Translation Research
17.
Computerized Publishing
18.
Relation Between Programming Languages and Linguistics 118
19.
Machine Translation and Linguistics
20.
Persons Who Appeared Before the Committee
113
121 124
91
107
Human Translation In order to have an appreciation either of the underlying nature and difficulties of translation or of the present resources and problems of translation, it is necessary to know something about human translation and human translators. Thus, early in the course of its study the Committee heard from a number of experts in translation. These experts seem to agree that the three requisites in a translator, in order of importance, are (1) good knowledge of the target language, (2) comprehension of the subject matter, and, (3) adequate knowledge of the source language. Therefore, while good translations into English are made by some translators whose native tongue is not English, in general, translators whose native tongue is English are preferable. Furthermore, while good translations are made by some translators who have a general appreciation of scientific knowledge, the best technical translations are generally made by experts in the technical field covered. It also seems clear that a restricted competence in the source language is adequate when the translator is expert in the subject matter. It was emphasized by several persons who made presentations to the Committee that translators need good dictionaries and reference hooks. This need is especially important when a long work is split up for translation, for in such cases adequate dictionaries or glossaries are essential if technical terms are to be translated consistently. Translators use a variety of aids, including dictating machines and typewriters, but they do not always produce a final copy suitable for reproduction. The final copy, with figures and equations inserted, is usually produced by the central service. Despite the substantial services performed by the Joint Publications Research Service (JPRS) or by similar agencies, the greater part of the cost of translation usually goes to the translator. One experiment that has come to the attention of the Committee indicates that a rapidly dictated translation is almost as good as a “full translation” and takes only about one fourth the time (see Appendix I ). 1
Types of Translator Employment The two main types of translator employment are in-house and contract. Each type has particular advantages and disadvantages for the translator and for the individual or organization requiring the translation.
IN-HOUSE The advantages to the in-house translator are that he is employed full time and enjoys all the benefits (leave and retirement, for example) that are offered to other full-time employees in the organization. In addition, he has available to him better reference facilities than his free-lance counterparts. The advantages to the employer of an in-house translator are chiefly the following: 1. The translator can give spot or oral translations when needed. 2. There is greater possibility for mutually beneficial collaboration between the translator and the requester. 3. The translator can provide fast service when needed. 4. The security of classified information is easily maintained. The disadvantages to the employer of the in-house translator are: 1. The arrangement (counting overhead and fringe benefits) is generally more expensive than using free-lance translators. 2. Problems in scheduling may arise from time to time, with the translator having either too much or too little to do. 3. Since it is impossible for the in-house translator to be an expert in all fields, it is difficult to get consistently good technical translations done in-house.
2
CONTRACT The advantages of a free- lance contract arrangement for the translator are: 1. If he can handle a relatively wide range of subject matter in some of the more uncommon and therefore higher-paying languages, he may earn considerably more than he would as an in-house translator. 2. He has considerably more freedom in deciding when and how much he will work. The advantages of the contract arrangement to the buyer of translations are: 1. He can obtain technically competent translations in many fields of subject matter. 2. He never pays for time not spent in translating. 3. He has a much lower overhead. The disadvantages of the contract arrangement to the buyer are: 1. The translator is not on the premises for immediate consultation. 2. Security of classified documents is more difficult to maintain.
3
English as the Language of Science It is easy to overestimate the need for translation if one simply looks at the rapidly increasing volume of scientific literature being published throughout the world. The United States is in a particularly fortunate position because English is the predominant language of science. A survey [R. T. Beyer, "Hurdling the Language Barrier," Phys. Today 18 (1), 46 (1965)] of 3,000 abstracts listed in Physics Abstracts and 350 physics abstracts listed in Referativny Zhurnal gave the following results: Language of Paper Abstracted ______
Physics Abstracts
Referativny Zhurnal
English Russian French German Other
76 14 4 4 2
63 24 3 2 8
percent percent percent percent percent
Although the ratio of English-language articles to non-English articles varies with the subject field, it is generally true that the English-speaking scientist has less need to read in a foreign language or to have translations made than does a scientist of any other native tongue.
4
percent percent percent percent percent
Time Required for Scientists to Learn Russian The Committee believes that in some cases it might be simpler and more economical for heavy users of Russian translations to learn to read the documents in the original language. An article by J. G. Tolpin, titled, "Surveying Russian Technical Publications: A Brief Course" [Science 146, 1143 (1964)], indicates that in eight to sixteen 2-hr class periods scientists can learn to identify articles of interest in Russian publications. Sometimes they can extract what they need from equations, tables, graphs, and figures. In many other cases, a partial oral translation of the material of interest is all that is needed. These are illustrations of the generally acknowledged fact that the technically competent reader needs only a little knowledge of a foreign language in order to make use of foreign journals in his field.* Indeed, several well-known studies† indicate that in 200 hr or less a scientist can acquire an adequate reading knowledge of Russian for material in his field. An increasing fraction of American scientists and engineers have such a knowledge. The capability for teaching government personnel to read Russian scientific text already exists, but so far this service has remained largely unused. The Defense Language Institute, West Coast Branch (formerly the Army Language School), has developed two courses of instruction and special texts for this purpose. One course runs 6 weeks, the other 10. The Committee has been informed that the Defense Language Institute would welcome the enrollment of students. Information concerning the 10-week course is presented in Appendix 2. * A corollary that should be given more emphasis is that even the best translation is of no use to a man who cannot fully understand the subject matter and place it in the context of other work here and abroad. †R.D.Burke, Some Unique Problems in the Development of Qualified Translators of Scientific Russian, P-1698, The RAND Corp. (May 12, 1959). W.N.Locke, J. Chem. Educ. 27, 426 (1950). M. Phillips, The Foreign Language Barrier in Science and Technology, Aslib, London, England (1962), p. 15.
5
Translation in the United States Government
It should be emphasized that there is no single official government translation system. Indeed there is considerable variety in the methods used by the various government agencies for filling their translation needs. The methods used include contract only, in-house translation, the services of the Joint Publications Research Service (Appendix 3), and a combination of these methods. Certain agencies are using PL 480 counterpart funds to augment their domestically obtained translations (Appendix 4). Others, principally the U.S. Air Force, utilize the postedited machine output of the Foreign Technology Division, Wright-Patterson Air Force Base (Appendix 5). In addition, the National Science Foundation, while not a primary producer of translations, is supporting the cover-to-cover translation of 30 journals (Appendix 6, Table 1).
6
Number of Government Translators The exact number of government in-house translators is impossible to determine, although it is a simple matter to determine the number of persons in the Civil Service classification, "Translator." It sometimes happens that the translator who decides to better his economic situation must first contrive to secure a more prestigious occupational title. Thus the way is open for advancement, even though the bulk of his duties might remain the same, The picture is further obscured by the fact that bilingual persons in other job categories are often called upon to produce rough or oral translations for their colleagues or superiors. This situation is not, of course, peculiar to agencies of the U.S. Government. Keeping in mind the indefiniteness of the number of persons actually classified under "Translator," we give the figures obtained from the Civil Service Commission for October 1962: Translators and clerk-translators employed in the United States Translators and clerk-translators employed worldwide
262 453
(For the number of translators in each division and grade, in each agency, and for the CSC salary schedule for 1964, and CSC qualification standards, see Appendix 7.) From the data supplied by the CSC, we have figured the average yearly salary of the federal translator (clerk-translator not included) employed in the United States to be approximately $6,850. When one compares this figure with the median annual salary of government scientists ($9,000. American Science Manpower, 1962, A Report of the National Register of Scientific and Technical Personnel, NSF 64-16, National Science Foundation, Washington, D.C., 1964), it is apparent that technically trained bilingual persons would derive more advantages from working as scientists and technologists in their subject specialties than from serving as technical translators in their respective fields. Despite the fact that the average pay for government translators 7
is not as high as the average for government scientists, there seems to be a very low rate of turnover among government translators. Indeed, the facts are that the supply exceeds the demand. Although there is not now on hand at the U.S. Employment Service (Washington, D.C.) a single request for a full-time translator, there are approximately 500 translators on its rolls who desire work (part time or full time). (For the availability of translators and their languages, see Appendix 8.)
8
Amount Spent for Translation
Considering the various methods used to secure translations, it is not surprising that federal agencies have paid many different prices for translation—prices ranging from $9 to $66 per 1,000 words. (It is not altogether unheard of for a translation purchaser to pay a translator who does exceptionally good work for more words than he actually translates.) At its first meeting, the Committee decided that it would be useful to have a fairly reliable estimate of the amount of money the government was spending for translation. Although the figures collected by the Committee constitute only an estimate—and a rough estimate, at that—we feel that it is the best estimate of the government’s translation expenditures made up to this time. Amounts spent by government agencies for translations done by: $ Millions JPRS Commercial Agencies Select Committee) PL 480 NSF Domestic In-House FTD MT
Fiscal Year 1964 Fiscal Year 1964 (Est. by H. R.
1.3 3.6
Fiscal Year 1965 Fiscal Year 1965 Fiscal Year 1963 1 March - 2 October 1964
1.5 1.1 5.3 0.27 13.07
Total
It is clear from the above figures that translation in the government is a very small field of activity when compared with most undertakings in which the government supports research and development. Bernard Bierman, a New York translation agency owner and a director of the American Translators Association has estimated that the commercial translation agencies in the United States do about $7.5 million worth of business each year. When this figure is added to the $ 13 million spent by the government, the sum is
9
about $20 million. To this should be added perhaps $2 million for the amount spent for nongovernment in-house translators. Thus the estimate of the amount of money spent on translation would be raised to approximately $22 million.
10
Is There a Shortage of Translators or Translation? In the past, it has been said that there is an unfulfilled need for translation or a shortage of translators. With respect to translators of other languages into English, the Committee finds that this is not so. This conclusion is based on the following data: 1. The supply of translators greatly exceeds the demand. The rolls of the U.S. Employment Service, the availability of translators to work at rates as low as $6 per 1,000 words (or lower), and conversations with translators confirm the Committee's conclusion. 2. The Joint Publications Research Service has the capacity to double its output immediately (with a very small increase in office staff) if called upon. The JPRS has 4,000 translators under contract, and in the average month it utilizes the services of only some 300 of them. To choose one important language as an example, the JPRS could with no difficulty handle up to two and a half times the present demand for Chinese translation. 3. The National Science Foundation's Publication Support Program will carefully consider, through a proper professional society, the support of the translation of any foreign journal that such a society nominates. Thirty journals were being translated cover to cover in Fiscal Year 1964 (see Appendix 6, Table 1). One translation has a circulation of only 200 copies. This comes close to providing individual service. In 12 years of NSF support, 19 translated journals have become self-supporting (see Appendix 6, Table 2). The Committee rejects any argument, based on the fact that the demand for the PL 480 translations is five times greater than the program can satisfy, that there is a shortage of translation. Such an argument is rejected on the grounds that the demand for almost any free commodity is insatiable. Forty-five (mostly government) information facilities, in response to a questionnaire issued by the Select Committee on Government Research (House of Representatives, 88th Congress), 11
indicated that the work of their facilities had been limited by a lack of translators. These 45 facilities were again asked by the Automatic Language Processing Advisory Committee whether their facility had been limited by a lack of translators, and if so whether this lack was attributable to a lack of authorized positions for translators or to a lack of qualified translators. The Committee received 25 replies. Some said that their facilities had no translation function. One said that it had not been limited by a lack of translators and that this situation was attributable to a lack of authorized positions. Six indicated that they were not limited by a lack of translators. Of the nine facilities that answered clearly in the affirmative that they had been limited by a lack of translators, seven indicated that this was attributable to a lack of authorized positions. Of the two remaining, only one, the nongovernment research center, said its lack was attributable to a lack of qualified translators. The others simply replied by saying that they did not have sufficient requests for services to justify permanent positions. The results of the survey confirm the Committee's belief that there is no shortage of translators, although there may be a shortage of authorized positions for translators. This, then, is a fiscal problem for the agencies and the Civil Service Commission, and not a problem for research and development offices supporting research in mechanical translation. The Committee concludes that all the Soviet literature for which there is any obvious demand is being translated [see A.G. Oettinger's "An Essay in Information Retrieval or the Birth of a Myth," Information and Control 8 (1), 64 (1965) concerning a claim of duplicated research], and, although it is less easy to evaluate the needs or coverage of open or closed material for intelligence, the Committee regards it as decisive that it has not encountered a single intelligence organization that is demanding more money for human translation. The Committee has heard statements that the use of translation is analyst-limited; that is, even if more material were translated, analysts would not be available to utilize it. Thus, it is ironic that several agencies propose to spend more money for "machine translation." The Committee is puzzled by a rationale for spending substantial sums of money on the mechanization of a small and already economically depressed industry with a full-time and parttime labor force of less than 5,000.
12
Regarding a Possible Excess of Translation While the Committee is not concerned with any lack of translation, it does have some concern about a possible excess of translation. Translation of material for which there is no definite prospective reader is not only wasteful, but it clogs the channels of translation and information flow. Routine translation should be confined to journals or books with reasonably assured paid circulation and additional translations should be made only in response to specific requests. In support of this position we quote from a letter received by the Committee from a research organization of the Department of Defense: We have found that the available translation services generally do not cover our technical areas to the depth that we require for our studies. As a result, we are continually putting in requests for translations of additional journal articles and such things as Soviet patents. Our problem has been the inability to obtain quick reaction to these special requests and it is this factor that has hampered rather than limited our work. If we had one recommendation to make to a survey such as yours, it would be that a better balance should be established between what is routinely translated and the special translation requests of users. We have found that many articles are being translated in our area that do not warrant the effort and it appears to us that some of the routine translations could be abandoned in order to make more translation services available for quick reaction to special requests.
It is possible that the cover-to-cover translations contain, in addition to much valuable information, many uninspired research reports that the U.S. scientist could have been mercifully spared. An interesting study, conducted in 1962, investigated the value of the articles contained in the Soviet journals translated in the National Library of Medicine/Public Health Service translation program [Report of Study of NLM/ PHS Russian Translation Program (Contract PH -86-62-9), Institute for Advancement of Medical Communication (Jan. 15, 1962)]. The method of evaluation used was 13
parallel editorial refereeing of the Soviet articles by counterpart American journals. Copies of the translated articles were sent to the editors in chief of counterpart American journals for distribution to their referees. The preliminary results were as follows. Of the total of 36 articles taken from two issues of the Sechenov Physiological Journal of the USSR, 31 percent were judged acceptable for publication in the American Journal of Physiology or the Journal of Applied Physiology. Of the total of 41 articles taken from two issues of Biophysics (USSR), 23 percent were judged acceptable for publication in the Biophysical Journal. In addition the referees indicated that another eight articles should be acceptable to the appropriate American journal. Of the 25 papers taken from two issues of Problems of Oncology, 76 percent were considered acceptable to Cancer. The referees indicated that another two articles would have been acceptable at one time but "would not now be considered new enough to merit publication." Further evidence of a possible excess of translation is to be found in The Need for Soviet Translations Among American Chemists, a report to the American Chemical Society by Herner and Company (June 4, 1962): On the other hand, the biggest argument that the respondents had with the translations presently available to them was not with their quality but with time lags in their issuance. The translation process—particularly when cover-to-cover translations are involved—is a relatively slow one. In view of the finding of the medical editors, one might well wonder whether a relatively high proportion of mediocre or inferior papers are not delaying the appearance of a small proportion of superior and significant papers. Perhaps even more revealing than the specifically stated reasons for nonuse of Soviet translations are the answers to the question in the questionnaire in regard to preferred media for receiving Soviet scientific information. Three methods outranked all others. These were: Englishlanguage abstracts of Russian publications, regular English-language reviews of Soviet developments in specific fields, and translations of individual articles as needed. These three methods are of course not mutually exclusive but complementary. Interestingly, the number of respondents who preferred to get their Soviet information in the form of cover-to-cover translations was only half the number who preferred to get their translations as needed. . . . The only things that might be done to round out the Soviet coverage that is presently available in chemistry is, first, to make sure that Soviet papers that are worthwhile in the opinion of the abstractors or editors are given detailed abstracting because they are likely not to ho readily available in English; second to provide means of obtaining cheap copies of cited Soviet papers, possibly through the Chemical Abstracts Service; and a third
14
To develop a mechanism for making selected translations available on request again possibly through the Chemical Abstracts Service. All three areas of improvement would probably require subsidization by the Government.. However, it would probably mean a far smaller expenditure than would be required to support an expanded program of cover-to-cover translations. It would also probably produce a far greater return.
It is the Committee's belief that the total technical literature does not merit translation, and it is futile to try to guess what someone may at some time want translated. The emphasis should be on speed, quality, and economy in supplying such translations as are requested. A service such as the Joint Publications Research Service, which charges the user for a translation, is less conducive to translation without use than is a service such as the U.S. Air Force Systems Command's Foreign Technology Division, which supplies translations free within certain areas.
15
The Crucial Problems of Translation There is no emergency in the field of translation. The problem is not to meet some nonexistent need through nonexistent machine translation. There are, however, several crucial problems of translation. These are quality, speed, and cost.
QUALITY The Committee believes strongly that the quality of translation must be adequate to the needs of the requester. The production of a flawless and polished translation for a user-limited readership is wasteful of both time and money. On the other hand, production of an inferior translation when one of archival quality is called for is even more wasteful of resources. It seems clear to the Committee that, in many cases, translations of adequate quality are not being provided. Despite the fact that adequate quality is essential, the government has no reliable way to measure the quality of translation. In view of this, one member of the Committee has set up an experiment in the evaluation of quality. This work is described briefly in Appendix 10. A reliable way to measure quality would be of great importance in determining proper cost of translation. The correlation between cost and quality is far from precise. Concerning this correlation, we quote from the presentation made to the Committee on September 30, 1964, by Dr. Kurt Gingold, President of the American Translators Association: There is no absolute correlation between cost and quality. There are some excellent translators who charge moderate rates, while some incompetents manage—at least temporarily—to charge much higher prices. Such correlation as exists is probably better at the low than at the high end; in other words, a cheap translation is almost always defective in some way, while an expensive translation is not always of superior quality. By and large, however, one gets what one pays for.
16
SPEED Reasonable speed and promptness are essential in translation. The Committee is convinced that in this regard there is considerable room for improvement. Of 2,258 scientists responding to a questionnaire concerning translated Soviet journals, 1,407 commented on lag time of publication; 24.5 percent of the comments were to the effect that lag time should be reduced (American Use of Translated Soviet Scientific Journals, a user study prepared by the Syracuse University Research Institute for the National Science Foundation and available from the Clearinghouse for Federal Scientific and Technical Information, Report No. TT-65-64026). The lag time (from receipt) for the average document processed by the AN/GSQ-16 (XW-2) Automatic Language Translator of the USAF Foreign Technology Division (FTD) is 109 days (44 days for high-priority items). Also at FTD, the average processing time for documents translated by outside contractors was usually 65 days plus 1.3 days for each 1,000 words of Russian translated. The most rapid translation service offered on a customary basis at regular prices that has come to the attention of the Committee is that the Joint Publications Research Service (JPRS), which guarantees 50 pages in 15 days, 100 pages in 30 days. The lag time (from receipt) in publication of the translated journals supported by NSF ranges from 15 to 26 weeks. On the average, half of this lag is accounted for by time spent in translation and editing (Appendix 6, Table 3). Thus, we see that many of the delays in "translation" do not lie in the process of translation itself, but rather in time spent in editing and production, and sometimes in avoidable delays. In the FTD machine-aided translation, the delays are in production and postediting, together with the delays caused by queues in the many operations that must be done in tandem in this particular form of machine-aided translation. It should be mentioned that for high-priority items extra fast translation service can be had by splitting long texts into segments, or by paying an additional fee that may range from 25 to 50 percent of the base rate or even higher, depending on the particular circumstances.
COST Cost is important because in many cases it is the only measure the government can sensibly use in deciding how its translation is to 17
be done. As we have seen, it varies considerably―from $9 to $66 per 1.000 words. Machines are probably inappropriate for some forms of translations, such as very high quality diplomatic translation and literary translation. But translations of scientific material can be done with or without machine aids. As to quality and speed, at extra cost, better quality and higher speed can be attained if long texts are split into segments. Thus, cost for a particular result is the criterion that the government should apply in deciding on means of translation. (See Appendix 9 for estimates of the costs of various types of translation.)
18
The Present State of Machine Translation
"Machine Translation" presumably means going by algorithm from machine- readable source text* to useful target text, without recourse to human translation or editing. In this context, there has been no machine translation of general scientific text, and none is in immediate prospect. The contention that there has been no machine translation of general scientific text is supported by the fact that when, after 8 years of work, the Georgetown University MT project tried to produce useful output in 1962, they had to resort to postediting. The postedited translation took slightly longer to do and was more expensive than conventional human translation. The "mechanical translation" facility of the USAF Foreign Technology Division (FTD) postedits the machine output when it produces translations. Dr. Gilbert King of Itek Corporation told the Committee that Itek plans to establish a "machine translation" service, but that it will provide postedited translations. Dr. J.C.R. Licklider of IBM and Dr. Paul Garvin of Bunker-Ramo said they would not advise their companies to establish such a service. Unedited machine output from scientific text is decipherable for the most part, but it is sometimes misleading and sometimes wrong (as is postedited output to a lesser extent), and it makes slow and painful reading † (See Appendix 10.) A recent study by the American Institutes for Research [D.B. Orr and V.H.Small, "A Reading Comprehension Test," Prelim. Rept., Contr. No. AF30(602-3459), June 30, 1965] had as its principal objective comparison of the accuracy and speed with which the * Machine-readable text is simply text that can be used as an input to a computer. It includes punched cards, punched paper tape, and magnetic tape, and is ordinarily prepared from printed text by a keyboard operator. † Excellent machine output of simple or selected text has been attained in several experiments; this is of no practical and limited theoretical significance. 19
same Russian documents can be read when they have been translated into English by the FTD machine translation (MT) system (one set postedited, the other set just as it came OUT OF the computer) and when they had been translated into English by a human translator in the conventional manner. In physics, tests showed that the reader of raw MT output was 10 percent less accurate, 21 percent slower, and had a comprehension level 29 percent lower than when he used human translation. When he used postedited output, he was 3 percent less accurate, 11 percent slower, and had a comprehension level 13 percent lower than when he used human translation. In the earth sciences, when he used raw MT output, he was 16 percent less accurate, 21 percent slower, and had a 25 percent lower comprehension level than when he used human translations. When he used postedited output, he was 5 percent less accurate, 11 percent slower, and had a comprehension level 23 percent lower than when he read human translations. Subjectively, a lot of the trouble seems to lie in unnatural constructions and unnatural word order, though strange translations of individual words or multiple translations of one word, with the choice left to the reader, are bothersome. (For a classification of the types of errors common in machine translation see Appendix 11.) The paragraphs below are typical of the recent (since November 1964) output of four different MT systems. Each sample gives the first and last (except for translation No. 4) paragraphs and a paragraph from the middle of a Russian article on space biology. Bunker-Ramo Corporation No. 1 Biological experiments, conducted on various/different cosmic aircraft, astrophysical researches of the cosmic space and flights of Soviet and American astronauts with the sufficient/rather persuasiveness showed/ indicated/pointed, that momentary/transitory/short orbital flights of lower/below than radiation belts/regions/flanges of earth/land/soil in the absence of the raised/increased/hightened sun/sunny/solar activity with respect to radiation are/appear/arrive/ report safe/not dangerous/secure. Received/obtained by astronauts of the dosage of the radiation at the expense of the primary cosmic emission/radiation and emissions/radiations of the external/outer radiation belt/region/flange are so/such a small, that can not render/show/give the harmful influence/action/effect on/in/at/to the organism of man. Mammals (dog, mouse/mice, rat, guinea pigs), fly/flies of the drosophilae, vegetable/vegetational objects/items/objectives. Seeds of higher/superior/supreme plants/vegetables (wheat, peas, onion/bow, the pine tree, beans, radish, carrot etc), microspore of the tradescantia/spiderwort, the
20
culture of the alga/seeweed chlorella on/in/at/to tissue, cellular, subcellular, and molecular levels (Gyurdzhian, 1962A. . Antipov et al., 1962) were used in these experiments. In experiments on/in/at/to mammals the special/particular/peculiar attention/consideration/was given to the research/analysis/investigation of the state/condition/position of the system of the blood/hemogenesis formation, the determination/definition/ decision of intermediate products of the exchange of nucleic acids (desoxycytidine and di)epolo$itel* substances), the study/investigation of the state/ condition/position of the natural immunity, the determination/definition/ decision of the maintenance/content of serotonin in the blood. Moreover, the control for/during/per/beyond the condition/state pigmentation of hair for/ at/by/from black mice (the line/strain CSUB57 BL) was conducted. Physiological shifts/improvements were studied also/as well on/in/at/to seeds of higher/superior/supreme plants, vegetables microorganisms, cells of various different tissues/cloth in the culture etc. Thus, the consideration/investigation certain/some from/of principal/ basic radiobiological problems shows/indicates/points/displays, that in the given region/area still/yet/more/back/some more very many/very much unsolved questions. This is clear/plain, since cosmic radiobiology is very the young section/division of young science—the cosmic biology. However there is/there are/is/eat basis to hope, that by common/general/total efforts of scientific various/different professions of different/various countries of the world/peace radiobiological researches in the cosmic space will be successfully continued/carried on and were expanded/broadened. Computer Concepts, Inc. No. 2 The biological experiments that were carried out on different cosmic flying apparatus, ASTROFIZICESKIE the research of cosmic PROSTRANSTVA and the flights of Soviet and American KOSMONAVTOV with sufficient UBEDITEL6NOST6H showed, that the short-time orbital flights below of the radiational belts of earth in the absence that was raised by the SOLNECNO1 one of activity in a radiational attitude are BEZOPASNYMI. Dose of radiation on at the expense of primary cosmic radiation and the radiation of an exterior radiational belt the obtained by KOSMONAVTAMI are so little, that aren't able to render a harmful influence to the organism of a man. Mammals (dogs, meeth, rats, sea SVINKI) were utilized in these experiments. The flies of drosophila, vegetable objects, semena of higher plants (wheat, GOROX, LUK, a pine tree, BOBY, REDIS, a carrot and others), MIKROSPORY of TRADESKANQII the culture of an alga chlorella in different nourishing mediums, the numerous biological and QITOLOGICESKIE ones objects on the TKANEVOM, cellular, subcellular and molecular levels (Ghrdjian, l962 and Antipov from Soavt 1962) and in experiences to mammals particular attention was being allotted to the research of the condition of the system of KROVOTVORENI4, to the definition of the intermediate products of the exchange of nucleic acids DEZOKSIQITIDINA and DIWEPOLOJITEL6NYX substances, to the study of the condition of natural IMMUNITETA, to
21
the definition of the content of SEROTONINA in KROL. Besides, control after the condition of PIGMENTAQII of VOLOS at CERNYX meeth (the line of C (57) of Y) was being carried out. Physiological SDVIGI were being studied also on SEMENAX of higher plants, microorganisms, the cells of different tissues in culture and T. of D. Thus, the examination of some from fundamental RADIOBIOLOGICESKIX problems shows, that in this a field still very much NEREWENNYX questions. This is clear, since cosmic RADIOBIOLOGI4 is very young RAZDELOM young science efforts of the scientific different specialties of the different countries of the world successful PRODOLJENY will be expanded there are. FTD, USAF No. 3 Biological experiments, conducted on different space aircraft/vehicles, astrophysical space research and flights of Soviet and American astronauts with/from sufficient convincingness showed that short-term orbital flights lower than radiation belts of earth in the absence of heightened solar activity in radiation ratio are safe. Obtained by astronauts of dose of radiation at the expense of primary cosmic radiation and radiation of external radiation belt are so small that cannot render harmful influence on organism of person. In these ESKPERIMENTAKH were used mammals (dog, mice, rat, guinea pig), fly of Drosophilae, vegetable objects, seeds of highest plants (wheat, pea, onion/bow, pine, beans, radish, carrot and others), microspore of tradescantia, culture of alga chlorella on different nutrient media, numerous biological and TSITOLOGICHCHESKIE objects on tissue, cellular, subcellular and molecular levels (Gyurozhian 1962A, Anti-Pov with/from Soavt, 1962). In experiments on mammals special attention was allotted investigation of state of system of sanguification, determination of intermediate products of exchange of nucleic acids (deoxycytidine and Dischepositive substances), study of state of natural immunity, determination of contents gray-fineness in blood. Furthermore, was conducted counterol for/after state of pigmentation of hairs for black mice (line bl). Physiologic shifts were studied also on seeds of highest plants, microorganisms, cages of different fabrics in culture etc. Thus, consideration of certain from basic radiobiological problems shows that in given region still very many unsolved questions. This and intelligibly, since space radiobiology is very young division of young science—space biology. However is base to trust that jointly scientists of different specialties of various countries of world/peace radiobiological investigations in outer space will be successfully continued and expanded.
22
EURATOM, Ispra, Italy No.4 (Essentially the Georgetown MT system) Biological experiments, which were conducted on different cosmic LETATEL6NYX APPARATI, the astrophysical investigations of cosmic space and the flights of Soviet and also American KOSMONAVTOV with the sufficient convincingness showed, that the short-term orbital flights of below radiation belts of ground upon the absence of the increased solar activity in radiation in relation are safe. Obtained by KOSMONAVTAMI of dose of radiation at the expense of initial cosmic radiation and the radiations of external radiation belt are so small, that cannot have harmful action on the organism of man. In these experiments there were used mammals (dogs, mice, KRYSY, the maritime piglets), MUXI DROZOFILY, vegetable objects. The seeds of higher plants (wheat, the pea, LUK, pine, beans, REDIS, MORKOV6 etc.) MIKROSPORY TRADESKANQII, the culture of alga of chlorella on the different feed environments, numerous biological and QITOLOGICESKIE objects on TKANEVOM, cellular, SUBKLETOCNOM and molecular levels (Ghrdjian, 1962 and Antipov with Soavt 1962). In experiments on mammals special attention was devoted to the investigation of state of system of KROVOTVORENI4, the determination of intermediate products the exchange of nucleic aids (DEZOKSIQITIDINA and DIWEPOLOJITEL6NYX substances), the study of the state of natural IMMUNITETA. The determination of content of SEROTONINA in blood. Besides this, there was conducted the check for the state or PIGMENTAQII the hair at black mice (the line C(57) Y)the Physiological) shifts were studied also on the seeds of higher plants, microorganisms, the cells of the different tissues in culture and T D.
The reader will find it instructive to compare the samples above with the results obtained on simple, or selected, text 10 years earlier (the Georgetown IBM Experiment, January 7, 1954) in that the earlier samples are more readable than the later ones. The quality of crude oil is determined by calory content.. The quality of saltpeter is determined by chemical methods. TNT is produced from coal. They obtain dynamite from nitroglycerine. Ammonite is obtained from saltpeter. Gasoline is prepared by chemical methods from crude oil. They prepare ammonite. Gasoline is produced by chemical methods from crude oil. The price of crude oil is determined by the market.. Calory content determines the quality of crude oil. TNT is prepared from coal.
The development of the electronic digital computer quickly suggested that machine translation might be possible. The idea captured the imagination of scholars and administrators. The practical 23
goal was simple: to go from machine-readable foreign technical text to useful English text, accurate, readable, and ultimately indistinguishable from text written by an American scientist. Early machine translations of simple or selected text, such as those given above, were as deceptively encouraging as "machine translations" of general scientific text have been uniformly discouraging. However, work toward machine translation has produced much valuable linguistic knowledge and insight that we would not otherwise have attained. No one can guarantee, of course, that we will not suddenly or at least quickly attain machine translation, but we feel that this is very unlikely. Victor H. Yngve of the MIT Research Laboratory of Electronics, in answer to a request from Committee Chairman John R. Pierce, expressed his views as follows: I concur with your view of machine translation, that at present it serves no useful purpose without postediting, and that with postediting the over-all process is slow and probably uneconomical. As to the possibility of fully automatic translation, I am convinced that we will some day reach the point where this will be feasible and economical. However, there is considerable basic knowledge required that we simply don't have at the moment, and it is anybody's guess how soon this knowledge can be obtained. However, I am dedicated to trying to obtain some of this knowledge. The question as to whether fully automatic translation will ever be economical must wait until we see whether it is possible at all. I feel that if it is possible, then it will be economical in the future because of the rapid advances in computer technology.
In his paper, "Implications of Mechanical Translation Research" [Proc. Am. Philosophical Soc. 108, 275 (1964)], Dr. Yngve notes: Work in mechanical translation has come up against a semantic barrier. . . We have come face to face with the realization that we will only have adequate mechanical translation when the machine can "understand" what it is translating and this will be a very difficult task indeed. . . "understand" is just what I mean . . . some of us are pressing forward undaunted.
The Committee indeed believes that it is wise to press forward undaunted, in the name of science, but that the motive for doing so cannot sensibly be any foreseeable improvement in practical translation. Perhaps our attitude might be different if there were some pressing need for machine translation, but we find none.
24
Machine-Aided Translation at Mannheim and Luxembourg As it becomes increasingly evident that fully automatic high-quality machine translation was not going to be realized for a long time, interest began to be shown in machine-aided translation. The Committee has knowledge of two important machine-aided translation systems in operation: the Federal Armed Forces Translation Agency, Mannheim, Germany, and the Terminological Bureau of the European Coal and Steel Community, Luxembourg. At these centers the approach is conservative; a machine is used to produce specialized glossaries helpful in the translation of particular documents. (Although the translation system in operation at the USAF Foreign Technology Division, Wright-Patterson Air Force Base, is being called, with increasing frequency, "machine-aided translation," it is actually a system of human-aided machine translation, relying, as it must, on posteditors to make up for the deficiencies of the machine output.)
MACHINE-AIDED TRANSLATION AT THE FEDERAL ARMED FORCES TRANSLATION AGENCY, MANNHEIM, GERMANY The Federal Armed Forces Translation Agency conducted an experiment designed to determine to what extent and in what areas machine output could aid the human translator. Two translators were given identical English texts to be translated into German. Neither translator was a specialist in the technical field treated in the text. Translator A had the conventional dictionaries and other reference works found in technical libraries and access to experienced experts. Translator B was given only a text-based or text-related glossary (TRG) that listed all and only the technical terms in the original text in the sequence in which they occurred plus their German equivalent or equivalents. To minimize any differences in the translators' abilities, a second text was 25
translated in which translator A used the TRG and translator B worked in the conventional way. The procedure above was repeated with two different translators and two different technical texts. Results of the test indicated that a translator working with conventional aids requires between 5086 percent (average, 66 percent) more time than a translator working with a text-related glossary. In addition to increased speed, another advantage of the TRG type of translation was that using this method the translators made one third fewer errors. We quote below from a translation of a paper titled "Production of Text-Related Technical Glossaries by Digital Computer, A Procedure to Provide an Automatic Translation Aid," by F.Krollmann, H. J. Schuck, and U. Winkler (the German original appeared in the January 1965 issue of Beiträge zur Sprachkunde und Informationsverarbeitung): These two experiments have shown that the speed (and thus the cost) of the translator's work as well as the quality of his product (and thus the output of the editor) can be considerably improved if it is possible to relieve the translator of the unproductive and tiresome search for the correct technical term that frequently cannot possibly be included yet in any of the conventional dictionaries. These figures would suggest that, ideally, the error quota in translations of technical-scientific texts can be reduced by approximately 40 percent—a figure which experience indicates can be improved by at least another 10-15 percent since better understanding of the text frequently results in improved linguistic rendition (unambiguity of style)—and that translator productivity can be increased by over 50 percent.
The system works in the following way. The translator reads through the text to be translated and underlines the English words for which he desires to know the German equivalent. The text is then given to a keypunch operator who punches the cards for the underlined words and at the same time performs morphological reduction of the English words (in most cases this simply involves omitting the inflectional suffixes). The information on the cards is then put into the computer, which can produce three or four textrelated glossaries in about 10 min. The TRG system became operational in 1965 and in early 1966 was connected by a data-link with a Telefunken TR-4 computer in Trier. At present the Federal Air Force Translation Agency has a cooperative agreement for exchange of terminologies with the U.S. Defense Language Institute/West Coast Branch, the British Admiralty, the European Coal and Steel Community, and others. An analysis of a test run and some sample output is to be found in Appendix 12. This technique was developed by the Federal 26
Ministry of Defense of West Germany which very kindly made available for the Committee use of the material in Appendix 12.
MACHINE-AIDED TRANSLATION AT THE EUROPEAN COAL AND STEEL COMMUNITY, L U X E M BOURG The Terminological Bureau of the European Coal and Steel Community (CECA) was established in 1950 to provide assistance to the Translation Bureau, which had the task of performing translations into and out of the four official languages of CECA—French, Dutch, Italian, and German. The Head of the Terminological Bureau, Mr. J. A. Bachrach, estimates that a minimum of 25 percent of the translator's time is spent on terminological questions and that, in difficult documents, up to 75 percent of the translator's time is spent on these problems. In collaboration with Mrs. Lydia Hirschberg of the Free University of Brussels and her group, various approaches to this problem were considered. Soon a system was devised by which the translator's time-consuming job of finding the answers to questions of terminology was made easier. The system utilized at CECA is one of automatic dictionary look-up with context included. The operation is similar to that used at Mannheim, but the output is somewhat different. It is similar in that the translator indicates, by underlining, the words with which he desires help. The entire sentence is then keypunched and fed into a computer. The computer goes through a search routine and prints out the sentence or sentences that most nearly match (in lexical items) the sentences in question. The translator then receives the desired items printed out with their context and in the order in which they occur in the source. The translation of the sentence is not done by the computer, but by a human translator. However, since the data produced by each query are added to the data base, the more the system is in use, the greater is the probability of finding sentences that have the desired term in the proper context. A sample of typical CECA FrenchEnglish output in shown in Appendix 13. The information that has been built up by CECA not only is of value in answering the queries of translators but also enables CECA to publish specialized glossaries in a very short time. Appendix 13, a copy of one extract from a five-language glossary prepared for the Congress on Steel Utilization is attached. The Committee finds it difficult to assess the difficulty and cost of postediting. An initial reaction is apt to be like that of R. T. Beyer [Phys. Today 18 (1), 50 (1965): 27
I must confess that the results were most unhappy. I found that I spent at least as much time in editing as if I had carried out the entire translation from the start. Even at that, I doubt if the edited translation reads as smoothly as one which I would have started from scratch. I drew the conelusion that the machine today translates from a foreign language to a form of broken English somewhat comparable to pidgin English. But it then remains for the reader to learn this patois in order to understand what the Russian actually wrote. Learning Russian would not be much more difficult. Someday, perhaps, the machines will make it, but I as a translator do not yet believe that I must throw my monkey wrench into the machinery in order to prevent my technological unemployment.
The Committee had some postediting done as an experiment (see Appendix 14). Postediting took as long as translation, yet people said they were willing to do it for less per word! FTD figures indicate that in-house postediting is done faster than in-house translation. Studies of the FTD operation indicate that keyboard transcription of the Cyrillic text is a very minor part of the total cost. Thus, automatic character recognition could cut the cost of the operation only a little. On the other hand, a large fraction of the cost is in putting the final translation together, with figures and equations, and reproducing it. If we compare the cost of human in-house translation ($40 per 1,000 Russian words) with the cost of machine-aided translation within FTD ($36 per 1,000 Russian words), machine-aided translation appears to be somewhat less expensive. But FTD machineaided translation is costlier than contract translation ($33 per 1,000) and far costlier than Joint Publications Research Service (JPRS) translation ($16 per 1,000 English words). Appendix 15 gives data on a comparison by experts of the quality of some recent JPRS translations and FTD machine-aided translations. The text of the JPRS translations was judged to be better than that of the FTD translations. The quality of the reproduction of text and figures was judged to be poor in both cases, with JPRS superior to FTD. We wonder why the Air Force pays more for translations made by FTD than superior and prompter JPRS translations would cost.
28
Automatic Language Processing and Computational Linguistics Over the past 10 years the government has spent, through various agencies, some $20 million on machine translation and closely related subjects (see Appendix 16). This is more than the government cost of translation for 1 year. Other moneys have been allocated to information retrieval, library automation, and programmed instruction. Although techniques of machine construction and programming for time-shared operation have been developed with partial support from the government, the computer industry has spent its own resources in machine development, and expenditures in connection with automatic language processing have played a distinctly minor role in advances in computer hardware. Industry has also been responsible for the development of important techniques of computer justification and hyphenation of newsprint and related matters of composition (see Appendix 17), perhaps because the market was easy to determine. As opposed to its small effect on computer hardware, work toward machine translation, together with the computational linguistic work that has grown out of it, has contributed significantly to computer software (programming techniques and systems). These contributions are discussed in considerable detail in Appendix 18. By far the most important outcome of work toward machine translation has been its effect on linguistics, which is described in more detail in Appendix 19. The advent of computational linguistics promises to work a revolution in the study of natural languages. A decade ago, most linguists believed that syntax had to do with word order, inflection, function words (e.g., prepositions and conjunctions), and intonation or punctuation. They also believed that most sentences uttered by native speakers in ordinary contexts were syntactically unambiguous. Today, they know that these two beliefs are mutually inconsistent. Their knowledge is the immediate result of computer parsing of 29
ordinary sentences, using reasonable grammars as hitherto conceived and programs that expose all ambiguities under a fixed grammar. Today there are linguistic theoreticians who take no interest in empirical studies or in computation. There are also empirical linguists who are not excited by the theoretical advances of the decade—or by computers. But more linguists than ever before are attempting to bring subtler theories into confrontation with richer bodies of data, and virtually all of them, in every country, are eager for computational support. The life's work of a generation ago (a concordance, a glossary, a superficial grammar) is the first small step of today, accomplished in a few weeks (next year, in a few days), the first of 10,000 steps toward an understanding of natural language as the vehicle of human communication. The revolution in linguistics has not been solely a result of attempts at machine translation and parsing, but it is unlikely that the revolution would have been extensive or significant without these attempts. We see that the computer has opened up to linguists a host of challenges, partial insights, and potentialities. We believe these can be aptly compared with the challenges, problems, and insights of particle physics. Certainly, language is second to no phenomenon in importance. And the tools of computational linguistics are considerably less costly than the multibillion-volt accelerators of particle physics. The new linguistics presents an attractive as well as an extremely important challenge. There is every reason to believe that facing up to this challenge will ultimately lead to important contributions in many fields. A deeper knowledge of language could help 1. to teach foreign languages more effectively; 2. to teach about the nature of language more effectively; 3. to use natural language more effectively in instruction and communication; 4. to enable us to engineer artificial languages for special purposes (e.g., pilot-to-control tower languages); 5. to enable us to make meaningful psychological experiments in language use and in human communication and thought (unless we know what language is we do not know what we must explain); and 6. to use machines as aids in translation and in information retrieval. However, the state of linguistics is such that excellent research, which has value in itself, is essential if linguistics is ultimately to make such contributions. 30
Such research must make use of computers. The data we must examine in order to find out about language is overwhelming both in quantity and in complexity. Computers give promise of helping us control the problems relating to the tremendous volume of data, and to a lesser extent the problems of data complexity. But, we do not yet have good, easily used, commonly known methods for having computers deal with language data. Therefore, among the important kinds of research that need to be done and should be supported are (1) basic developmental research in computer methods for handling language, as tools for the linguistic scientist to use as a help to discover and state his generalizations, and as tools to help check proposed generalizations against data; and (2) developmental research in methods to allow linguistic scientists to use computers to state in detail the complex kinds of theories (for example, grammars and theories of meaning) they produce, so that the theories can be checked in detail.
31
Avenues to Improvement of Translation
We have already noted that, while we have machine-aided translation of general scientific text, we do not have useful machine translation. Further, there is no immediate or predictable prospect of useful machine translation. We have noted that the important contributions of machine translation have been primarily to linguistics and secondarily to computer programming. We have noted that while translation itself is vital, needs for translation are being met by a small though capable activity. We find, however, that there are attractive opportunities for improvement in translation, and we urge work aimed at such improvement. We have noted the importance of quality in translations. We have noted that cost varies markedly with asserted quality. It is important, therefore, to achieve some objective evaluation of accuracy and quality. Work toward practical useful tests, such as that described in Appendix 10, is of the greatest importance. Machine aids may be an important adjunct to human or machineaided translation. USAF Foreign Technology Division (FTD) figures show that production costs (assembly and reproduction of the final translations) are very high. It appears that delays in translated journals are attributable to production rather than to translation. Adoption of mechanized means of editing and production might be desirable (see Appendix 17). Here the main cost of research and development can best be borne by other, larger fields than translation. Machine-aided translation may be an important avenue toward better, quicker, and cheaper translation. What machine-aided translation needs most is good engineering. What will help the human being most—special glossaries, dictionary look-up of some or all words in the text, or a rough translation such as that produced by FTD? How can the delays due to queues at many tandem steps be avoided? How can production costs be cut? 32
Automatic character recognition is often mentioned as important. to machine-aided translation. FTD figures indicate that automatic character recognition could slightly decrease the cost of the operation. Automatic character recognition work is being supported heavily in connection with several kinds of activity (information retrieval, post office, for example) where the financial savings through successful character recognition would be much greater than in machine-aided translation. Hence, character recognition should be adopted when and if it will save money, but research and development need not be supported in connection with machine translation. Finally, how much should be spent on research and development toward improving translation? It would be unreasonable to spend extravagantly on a relatively small business that is doing the job satisfactorily. The Committee cannot judge what the total annual expenditure for research and development toward improving translation should be. However, it should be spent hardheadedly toward important, realistic, and relatively short-range goals.
33
Recommendations
The Committee recommends expenditures in two distinct areas. The first is computational linguistics as a part of linguisticsstudies of parsing, sentence generation, structure, semantics, statistics, and quantitative linguistic matters, including experiments in translation, with machine aids or without. Linguistics should be supported as science, and should not be judged by any immediate or foreseeable contribution to practical translation. It is important that proposals be evaluated by people who are competent to judge modern linguistic work, and who evaluate proposals on the basis of their scientific worth. The second area is improvement of translation. Work should be supported on such matters as 1. practical methods for evaluation of translations; 2. means for speeding up the human translation process; 3. evaluation of quality and cost of various sources of translations; 4. investigation of the utilization of translations, to guard against production of translations that are never read; 5. study of delays in the over-all translation process, and means for eliminating them, both in journals and in individual items; 6. evaluation of the relative speed and cost of various sorts of machine-aided translation; 7. adaptation of existing mechanized editing and production processes in translation; 8. the over-all translation process; and 9. production of adequate reference works for the translator, including the adaptation of glossaries that now exist primarily for automatic dictionary look-up in machine translation. All such studies should be aimed at increasing the speed and decreasing the cost of translations and at specifying degrees of acceptable quality. 34
Appendix 1 Experiments in Sight Translation and Full Translation In 1963, an experiment in sight translation was conducted by Dr.H. Wallace Sinaiko of the Institute for Defense Analyses ("Teleconferencing, Preliminary Experiments," Research Paper P-108, IDA, Nov. 1963). Sight translation is a procedure in which written material being received via teleprinter is read and a translation is dictated to a typist simultaneously. In this experiment, professional conference interpreters translated the complete text of the minutes of the 921st meeting of the U.N. Security Council into English and French. This experiment showed that the accuracy of the sight translation was uniformly high and that when the interpreters were working in an unaccustomed direction, i.e., English into French or French into English, both the time required for the sight translation and the number of errors were increased somewhat, although not seriously. Another experiment (full translation) used highly experienced Department of State translators in two-man translating - review teams. The partners in each team divided the incoming batches of material between themselves, each translating a part and then reviewing the part translated by his colleague. The quality of the translations was very high, but scarcely higher than the sight translation. COMPARISON OF SIGHT AND FULL-TRANSLATION METHODS
Original U.N. Security Council Meeting, consecutive interpretation Sight translation Full translation
Time, hr
Rate, words per min
2.0 9.7 37.6
102.0 21.0 5.4
Although the sight translation was four times faster than the full translation and of comparable quality, it would be dangerous to conclude from this that present translation output could be quadrupled 35
by use of the sight-translation method. Since the material translated in this experiment was, presumably, all straight text, it lent itself nicely to this type of translation. It is doubtful that such a system could operate with the same efficiency on scientific texts containing photographs, charts, tables, formulas, and other graphics. Nevertheless, the Committee feels that certain features of this system might be applicable to certain circumstances. One agency in Washington that uses the dictation method states that on texts that are suitable (few graphics to be inserted) the daily output per translator is doubled—from 2,400 to about 5,000 words. These experiments stress an important difference between human and machine approximation in translation. Once the deeper meaning of the content of a text is grasped, the human translator immediately leaps to relatively grammatical output. The time taken by him in successive approximation probably involves choices among optional transformations, seeking the best base from which final stylistic polishing may be made in order to recapture the flavor of the original. On the other hand, the machine does its approximating by moving through successive choices among ungrammatical versions. Therefore, it would seem that there are good reasons why cheap, hasty, and truncated jobs might be better done by humans than by machines.
36
Appendix 2 Defense Language Institute Scientific Russian Course
The following information, provided by the Defense Language Institute, West Coast Branch, concerns the 10-week DLIWC Scientific Russian Course. The purpose of the course is to train students to read and translate Russian technical and scientific texts in their fields of interest with the help of dictionaries and to speak and understand conversational Russian to a limited degree. The length of the course is 10 weeks; 5 days per week; 6 hr per day. For teaching purposes the classes are divided into sections of usually not more than eight students. The teaching materials used during the course consist of four textbook volumes specially developed for this course and dealing with essential Russian grammar, speech patterns, and exercises in the translation of scientific texts. A special reference volume is also provided. Recent Soviet publications on scientific topics in the students' particular fields of interest are introduced in the form of supplementary training materials. The teaching materials for the Scientific Russian Course were developed so as to ensure maximum effectiveness. After an initial period, during which the essentials of the Russian language are taught, the students switch over to teaching materials entirely corresponding to their aims and specialities. The course is, therefore, flexible and can accommodate specialists in various fields of scientific knowledge. In conformity with the objectives outlined above, the main emphasis in the implementation of the course is laid on reading and on translating from Russian into English. The course involves the study of essential structural patterns of the Russian language that are indispensable for the understanding of scientific texts. Since Russian is a highly inflected language, special stress is laid on the recognition of morphological change
37
in words and its importance in grasping the exact meaning of sentences. This is especially important in texts involving. mathematical formulas and definitions where any distortion of meaning might easily lead to entirely erroneous conclusions. While speaking and aural-comprehension abilities are not specially emphasized in the course, the students are taught to speak and understand conversational Russian, though only to a limited degree. Work in this particular field involves the use of tape recorders. At the end of the course the graduates have a vocabulary of approximately 750 words used in everyday exchanges. With respect to scientific terminology, the course features the study of so-called "cognates"—internationally used terms derived from the same root. The aim here is to teach the students to recognize such words without the help of dictionaries and thus to facilitate and speed up their work. After completing the course, the graduates are able to read, understand, and translate very complex texts in their fields of interest.. The first scientific Russian course was implemented at this Institute in 1961. In the past 4 years, this 10-week course was attended by specialists in space mechanics, applied mathematics, electrical engineering, chemistry, physics, and aeronautics. In view of the important scientific and technological achievements that have been taking place in the Soviet Union in the last few decades, it is hardly necessary to stress the utility of a course that makes it possible for the specialists to learn in a comparatively short time enough Russian to read contemporary Soviet scientific literature in their fields of interest, and thus to keep abreast of developments in that country.
38
Appendix 3 The Joint Publications Research Service
The Joint Publications Research Service (JPRS), a component of the Clearinghouse for Federal Scientific and Technical Information, U.S. Department of Commerce, was established in 1957 by a group of federal agencies that needed English translations of books, newspapers, periodical articles, and other materials being published in a variety of languages. Using a small staff of professional linguists, a search was made to locate the thousands of specialists—chemists, physicists, political scientists, economists—who, although already working in their special fields, possessed knowledge of a foreign language and were willing to translate materials in their fields on a part-time, contract basis at home. New York was chosen for the first office because of its large population, which, it was felt, would yield the greatest number of linguists of any single area in the United States. Success in finding competent translators was immediate, and another office was opened in Washington, D.C., in August 1957. Three years later, with a still-growing load, a third JPRS office was opened in San Francisco. Although begun as a cooperative venture in 1957, the JPRS was absorbed by the Office of Technical Services in 1958, when it assumed responsibility for collecting translations and making them available to the public. The growth of the JPRS can be seen by comparing the 38,000 published pages produced from March 1957 through June 1958 with the 273,449 pages published in Fiscal Year 1964. The first year's production was about 70 percent scientific and technical material, whereas production for Fiscal Year 1964 was about half that, or 35 percent. A considerable number of translations published by the Atomic Energy Commission (AEC) are translated by the JPRS but sent to the AEC for publication as a part of its series; the same holds for translations done for the Army Biological Laboratory, Redstone 39
Arsenal, the National Institutes of Health, the Federal Aviation Agency, and other agencies. Materials of broad current interest spotted by analysts, scientists, and others in government are sent to the JPRS for translation and for publication. Over the years, under this program, JPRS has developed serial titles under which a great deal of similar information has been placed. For example, Translations on International Communist Developments contains materials from any foreign newspaper or periodical that sheds light on the developments, policies, debates, or other activities of the Communist parties of all countries. Copies of these and of all other translations are then distributed not only to the initiating component, but to all participating organizations. The series are then available on subscription to anyone outside the government who is interested. In science and technology, the JPRS series on Foreign Developments in Machine Translation and Information Processing, 173 issues of which have been published, has proven valuable to researchers in the field. For example, a recent Office of Technical Services special bibliography on machine translation lists 250 citations of reports and translations on the subject; 118 of these were JPRS reports. JPRS charges the government agencies for which it works the same price for all translations regardless of subject matter or language. This price is currently $16 per 1,000 words of English. This figure has been arrived at by a study of the total costs involved and includes overhead. Of the $16 per 1,000 words paid by the requesting agency, the translators are getting, on the average, $8 to $11 for simple newspaper-type material (the low) and $20 for Chinese (the high). Editing costs about $1.50 per 1,000 words, the typing about $1.50, and the overhead about $2.00. The translation comes back from the contractors on tapes, in rough draft, and in completed typewritten form. The amount paid the translator is dependent (in addition to the language of the original) upon how much extra work the JPRS has to do on the translation after the contractor has submitted it. The policy of the JPRS regarding lag-time is as follows: 50 pages of translation will be done and returned to the requester in 15 days; 100 pages will be done in 30 days. The JPRS currently has about 4,000 translators under contract, with a potential of an additional 1,500 available almost immediately. On the average, JPRS utilizes the services of about 300 of its translators in any given month. Thus, it appears that JPRS is producing translations reasonably quickly and quite economically, and, furthermore, that it has the capability of immediately expanding its operations. 40
Appendix 4 Public Law 480 Translations
The National Science Foundation is responsible for conducting a science-information program financed exclusively with excess foreign currencies that have accrued to the credit of the U.S. Government from the sale of U.S. surplus agricultural commodities in a number of foreign countries. Title I of the Agricultural Trade Development and Assistance Act of 1954 (Public Law 480), as amended, authorized the President to enter into agreements with friendly nations for the sale abroad of U.S. surplus commodities for foreign currencies. These currencies are inconvertible and may not be used outside the country involved. Under the law cited above, U.S. Government agencies are authorized to use foreign currencies "to collect, collate, translate, abstract, and disseminate scientific and technological information and to conduct research and support scientific activities overseas including programs and projects of scientific cooperation between the United States and other countries." In January 1959, the President assigned to the Foundation the responsibility for initiating a unified coordinated program for meeting the requirements of the agencies of the Executive Branch for translation and other science-information activities authorized under Public Law 480. The Foundation entered into contracts with Israel and Poland in 1959 and with Yugoslavia in 1960. Each contract provides for translation and publication of scientific literature and patents, translation and preparation of abstracts (in cooperation with U.S. abstracting and indexing services), publication of critical review papers, compilation of bibliographies, and the preparation of guides to their scientific institutions and information systems. At the present time, the Foundation coordinates and administers this program for the Departments of the Interior, Agriculture, Commerce, and Health, Education and Welfare, the Atomic Energy Commission, the National Aeronautics and Space Administration, and the Smithsonian Institution. 41
The Foundation does not select the material to be translated. The selection is done by research scientists in the participating federal agencies. In Poland, Polish scientific information is translated; in Yugoslavia, Yugoslavian material; and in Israel mainly, although not exclusively, Russian scientific literature. Russian books and monographs must have been published at least 1 year before they are translated by the overseas contractor; Yugoslavian and Polish journals only are translated on a current basis. The translation programs overseas are supplemental to, and not competitive with, the "domestic" translation program. In these programs no dollar expenditures are involved. The combined efforts of the programs in Israel, Poland, and Yugoslavia represent the translation and republication of about 250,000 pages of foreign scientific literature (95 volumes of scientific journals, 374 books, 1,004 selected articles, 18,495 abstracts, 13,000 patents).* This covers the period from Fiscal Year 1959 through fiscal 1965.
*The statement above was taken from "A Summary of U.S. Translation Activities" (in Seminar on Technical and Scientific Translation, Apr. 1517, 1965, Indian National Scientific Documentation Cent re, New Delhi) by Ernest R. Sohns of the Office of Science Information Service, National Science Foundation. The Committee appreciates Dr. Sohns' cooperation in providing this report.
42
Appendix 5 Machine Translation at the Foreign Technology Division, U.S. Air Force Systems Command
In December 1962, the USAF Scientific Advisory Board Ad Hoc Committee on Mechanical Translation of Languages recommended the implementation of "a limited initial operational capability for mechanical translation of at least 100,000 words of Russian per day using the IBM Mark II translation equipment and Phase II translation system." This system became fully operational in February 1964 at the U.S. Air Force Systems Command's Foreign Technology Division (FTD) at Wright-Patterson Air Force Base, Ohio. Operations at FTD have recently been the subject of a study by Arthur D. Little, Inc., and it is from this study that the following data have been taken: 1. The cost of machine translation (excluding overhead and equipment amortization) is about $36 per 1,000 Russian words. 2. FTD's in-house human-translation cost, excluding overhead, is about $40 per 1,000 Russian words. 3. FTD's contract translation cost is about $33 per 1,000 Russian words, including contractor's overhead. 4. Postediting (31 percent) and recomposition (40 percent) are the main cost components in the machine-translation process, accounting for over 70 percent of the total cost; input processing accounts for only 11 percent. 5. The average total machine-translation processing time is 109 days. The average for high-priority documents is 44 days. 6. During the period June-September 1964, the average output per working day was 103,146 Russian words translated into English. The average output per hour was 7,569 words. The average working day for the computer, therefore, amounts to 13 hours. 7. Input costs to the machine-translation system amount to $4.10 per 1,000 Russian words. From the A. D. Little data and from the results of a comparison with the work done by the Joint Publications Research Service (see 43
Appendix 3), one sees that the FTD postedited machine translations are slow, expensive, of poor graphic arts quality, and not very good translations. The FTD machine-translation facility currently has a staff of 43 persons, including the posteditors. Their final product is 100,000 words of poor translation per day. Since JPRS could do the same amount of translation faster and for less than half the price, the Committee is at a loss to understand why the FTD does not rely on the services of the JPRS.
44
Appendix 6 Journals Translated with National Science Foundation Support
ABBREVIATIONS USED IN THIS APPENDIX AGI
American Geological Institute
AGS
American Geographical Society
AGU
American Geophysical Union
AIBS
American Institute of Biological Sciences
AIChE
American Institute of Chemical Engineers
AIP
American Institute of Physics
AMS
American Mathematical Society
ASME
American Society of Mechanical Engineers
CB
Consultants Bureau Enterprises, Inc.
ESA
Entomological Society of America
GChS
The Geochemical Society
IEEE
Institute of Electrical and Electronics Engineers
IJSM
International Journal of the Science of Metals
ISA
Instrument Society of America
OSA
Optical Society of America
SIAM
Society for Industrial and Applied Mathematics
SSSA
Soil Science Society of America
ST
Scripta Technica, Inc.
45
TABLE 2.
Translation Journals That Achieved Self-Sufficiency
IN JANUARY 1964 Soviet Phys. - JETP
AIP
Soviet Phys. - Dokl.
AIP
Soviet Phys. - Usp.
AIP
Soviet Phys. - Solid State
AIP
Soviet Phys. - Acoust.
AIP
Soviet Phys. - Cryst.
AIP
Soviet Phys. - AJ
AIP
Soviet Phys. - Tech. Phys.
AIP
Appl. Math. Mech.
ASME/Pergamon
Phys. Metals Metallog.
IJSM/Pergamon
Dokl. - Biol. Sci. Sect.
CB
Dokl. - Botan. Sci. Sect.
CB
Dokl. - Biochem. Sect.
CB
Plant Physiol.
CB
Microbiology
CB
IN JANUARY 1965 Metal Sci. Heat Treat. Metals
IJSM/CB
Metallurg.
IJSM/CB
Refractories
IJSM/CB
Friction and Wear in Machinery
ASME
48
Appendix 7 Civil Service Commission Data on Federal Translators
TRANSLATORS AND CLERK TRANSLATORS Total Translators in Each Division and Grade UNITED STATES Translators
Clerk Translators
Number 6 22 14 26 15 40 10 52 23 7 2 217
Grade 4 5 6 7 8 9 10 11 12 13 14
Number 16 24 3 1
Grade 4 5 6 7
1
9
__ 45
Total U.S. Translators and Clerk Translators: 262
WORLDWIDE Clerk Translators___________
Translators Number 6 36 17 40 29 71 16 54 26
Grade 4 5 6 7 8 9 10 11 12
50
Number 17 54 22 3 1
Grade 4 5 6 7 9
WORLDWIDE (Cont'd) Number 7 3 25 330
Grade 13 14 Not Graded*
Number 26 123
Grade Not Graded*
Total Worldwide Translators and Clerk Translators: 453 * Employed by an agency that does not use the grading system.
Classification of Translators and Clerk Translators According to Representative Agency Translators U.S.A. 17 26 3 32 11 32 13 9 4 5 18 36 1 1 1
Clerk Translators*
Worldwide 17 26 5 112 13 37 14 9 4 5 18 36
U.S.A. 5 2 17 2 6 1
Worldwide 5 2 1 51 6 34 6 4
1
1
1
1
9
9
1 1 1 1
9 2
17 2 1
1 6
2 1 7
2 1
1
Agency______________________ Library of Congress Dept. of State Treasury Dept. Dept. of the Army Dept. of the Navy Dept. of the Air Force Dept. of Justice Post Office Dept. Dept. of the Interior Dept. of Agriculture Dept. of Commerce Dept. of Health, Education and Welfare Canal Zone Government Federal Aviation Agency Federal Communications Commission General Services Administration Housing and Home Finance Agency U.S. Information Agency National Aeronautics and Space Administration National Labor Relations Board Panama Canal Company Railroad Retirement Board Veterans' Administration
*A clerk translator primarily does clerical work and is required to have some familiarity with the language involved in his work. The bulk of clerk translators are located on the Mexican border, in Puerto Rico, and on Indian reservations.
51
Civil Service Salary Schedule, 1964 Grade
Minimum
Maximum
Mean
4 5 6 7 8 9 10 11 12 13 14
$ 4,480 5,000 5,505 6,050 6,630 7,220 7,900 8,650 10,250 12,075 14,170
$ 5,830 6,485 7,170 7,850 8,610 9,425 10,330 11,305 13,445 15,855 18,580
$ 5,155 5,743 6,338 6,950 7,620 8,323 9,115 9,978 11,848 13,965 16,375
CGS QUALIFICATION STANDARDS, TRANSLATOR SERIES (EFFECTIVE DECEMBER 1959)* Translator GS-5/11 Category I positions require sufficient knowledge of the languages involved to render adequate translations of simple, uncomplicated, nontechnical material such as birth, marriage, and death certificates, proofs of residence, and correspondence dealing with relatively simple inquiries for information about benefits, services, etc. Positions in this category are found only at GS-5 and GS-7. Category II positions require that the translator have a native ability† in the language into which the translation is made, and a comprehensive knowledge$ of the language from which the translation is made. Translations cover a broad variety of subjects such as science, economics, legal, and diplomatic work, as well as any other type of technical or specialized subject-matter material that may require translation. The level of difficulty of positions in this category is determined not by degree of language proficiency alone but also by the knowledge and comprehension of the subject matter involved. Positions in this category are found at all levels between GS-5 and GS-12. * Quoted from GS-031. † Native ability in a language is the ability to speak or write a language so fluently that the expression of thought is structurally, grammatically, and idiomatically correct and reflects a range of vocabulary in the language commonly characteristic of a person who has received his education through the high-school level in a country of the language. $ Comprehensive knowledge of a language means the ability to read the language easiIy. It represents an ability acquired usually acquired through academic study and is a lesser ability than "native ability" as defined here. 52
LANGUAGE AND EXPERIENCE REQUIREMENTS ― CATEGORY I POSITIONS Written Tests are Required for All Positions Grade GS-5. Candidates must be able to translate from one foreign language into English or from English into one foreign language. Candidates must be able to translate from two Grade GS-7. foreign languages into English, or from English and one foreign language into one other foreign language. In addition, candidates for grade GS-7 must have 1 year's specialized experience in preparing written translations of nontechnical material of routine or repetitive nature in the appropriate languages. LANGUAGE AND EXPERIENCE REQUIREMENTS ― CATEGORY II POSITIONS Written Tests are Required for All Positions Positions in this category require the ability to translate from at least two foreign languages into English or from English into a foreign language and from the same foreign language into English. In addition to basic language ability, candidates must have the following number of years of specialized experience: Grade
Total, yr
GS-5 GS-7 GS-9 GS-11
0 1 2 3
This work experience must demonstrate the ability to prepare written translations in the appropriate languages, involving technical material in one or more specialized subject-matter fields such as architecture, automotive mechanics, physics, biology, legal or judicial procedures, foreign affairs, statistics, etc. This translation work must be of such a nature that the finished products appear to have been written by a native subject-matter specialist or technician in terms of sense, tone, style, and terminology. The degree of finish will depend upon the level of difficulty involved. For all levels above GS-7, 1 year of this specialized experience must be equivalent in scope and difficulty to that of the next lower level in this series.
53
Appendix 8 Demand for and Availability of Translators
A. GEOGRAPHICAL DEMAND According to the U.S. Department of Labor, Bureau of Employment Security, the geographical demand for translators during calendar year 1964 was centered in Washington, D.C. (see below). The only other demand recorded on the bureau's interarea recruitment records was as follows: Month, 1964
No. of Openings
January February March April May June July August September October November December
4 5 2 2 3 3 2 2 2 2 2 3
Locations ____________________ Minn., Mo., Ark., Hawaii N.J., Pa., Mo., Ark., Hawaii Mo., Ark. Mo., Ark. N.J., Ohio, Mo. N.J., Ohio, Mo. Minn., Mo. N.J., Mo. N.J., Mo. N.J., Mo. N.J., Mo. N.J., Ill., Mo.
Although New Jersey and Missouri each appear more frequently than do the other states, the Bureau feels that this repetitive requirement reflects difficulty in securing qualified persons rather than a turnover of translator personnel.
B. GOVERNMENT AGENCIES IN THE WASHINGTON AREA THAT ANNOUNCED VACANCIES IN FISCAL YEAR 1964 (Data supplied by the U.S. Employment Service, District of Columbia Professional Placement Center) 54
Agency
Language(s)
Central Intelligence Agency Department of State
Information not available Arabic, Persian, Turkish, Slavic French All Hindi Information not available
U.S. Information Agency U.S. Joint Publications Research Service Voice of America National Security Agency
C. GOVERNMENT VACANCIES BY TYPE OF EMPLOYMENT (Data supplied by United States Employment Service, District of Columbia Professional Placement Center) I. Full-Time Translators Note: The U.S. Employment Service defines full-time employment in the following categories: (a) Permanent full-time—A position that lasts more than 30 days and has a 5-day, 40-hr week. (b) Temporary full-time—A position that lasts 4 to 30 days and has a 5-day, 40-hr week. (c) Short-time full-time—A position that lasts less than 4 days and has an 8-hr day. The only agency that requested permanent full-time translators was the National Security Agency. No translators were requested under categories (b) and (c). II. Part-Time Translators Note: The U.S. Employment Service defines part-time employment in the following categories: (a) Permanent part-time—A position that lasts more than 30 days and has less than an 8-hr day. (b) Temporary part-time—A position that lasts 4 to 30 days and has less than an 8-hr day. (c) Short-time part-time—A position that lasts less than 4 days and has less than an 8-hr day. Permanent part-time translators (a) were requested by the U.S. Joint Publications Research Service. Temporary part-time translators (b) were requested by The U.S. Department of State Foreign Service Institute. No short-time part-time translators (c) were requested. It is interesting to note that the agency requesting category (b) translators did not request category (c) translators.
D. NUMBER OF AVAILABLE TRANSLATORS IN THE WASHINGTON AREA The U.S. Employment Service, District of Columbia Professional Placement Center, has 523 translators registered. (The number 55
of available translators (826) exceeds the number of translators registered (523) because many translators indicated their ability to work in more than two languages). A sample of the number of translators available for work in some of the more exotic languages is shown below. Language_____________
No. of Available Translators
African Languages Akau Amharic Efik Fante Hausa Ibo Mandingo Swahili Twi Yoruba
2 4 1 2 2 3 1 6 1 3
Chinese Languages Mandarin Cantonese Shanghai Fukien
21 3 3 1
Indian Languages Bengali Gujarati Hindi Malayalam Tamil Telugu Urdu
6 4 11 4 5 5 4
Philippine Languages Bikol Chabokano Ermitano Tagalog Wraywaray
1 1 1 5 1
The Committee would like to express its appreciation to Miss E. Catherine Phelps, Manager of the U.S. Employment Service, District of Columbia Professional Placement Center, for her cooperation in providing these data for the Committee's use.
56
Appendix 9 Cost Estimates of Various Types of Translation Before attempting to determine the costs of various types of translation, it might be instructive to see what the costs would be for an operation that made no use of translations, that is, a system that utilized subject specialists who were also skilled in a second language. Let us assume that we have an agency that employs 100 analysts and let us further assume the following: 1. that 50 of the analysts are competent in Russian in their subject field, 2. that each analyst earns $12,000 per year, 3. that each analyst reads 1,000 words of Russian per day in his work, 4. that each analyst works 220 days per year, and 5. that, therefore, the agency consumes a total of 11,000,000 Russian words a year. Since the major effort in past work on machine translation (MT) has been to develop a program to translate Russian into English, let us now restrict our discussion to the 50 analysts who are proficient in Russian. Salaries for these 50 would amount to $600,000 per year. Other costs such as Social Security, annual and sick leave, and retirement could be calculated at approximately 33 1/3 percent of their gross salaries. Thus the cost for these analysts would be approximately $800,000 per year. Obviously, no duplication checks would be necessary to determine whether a translation of any given work was already in existence. The Committee has no figures on the cost of maintaining facilil i e s necessary for the making of checks to prevent the duplication of translation. If these costs could be determined and if they proved to be substantial, it might be the case that it would be more economical not to make duplication checks of documents less than some specific number of pages in length. In any event, the duplication 57
checks would be superfluous for an agency employing persons proficient in a foreign language.
MAJOR COSTS OF ITEMS OF AN AGENCY UTILIZING 50 ANALYSTS PROFICIENT IN RUSSIAN 50 Analysts at $12,000 per annum Direct cost overhead at 33 1/3 percent of the above Duplication checks Total
$600,000 200,000 0 $800,000
Figured at 220 working days per analyst the total volume of words of Russian read would amount to 11,000,000 or about $75 for each 1,000 words read. Time lag after receipt of document Total Cost of Translation
none 0
MONOLINGUALS If the 50 analysts could not read Russian and had to rely on translation, a number of possibilities exist for providing them with English translation. The agency could 1. employ in-house translators in the conventional method, 2. employ translation using the dictation (or sight) method of translation, 3. employ contract translators, 4. utilize the services of JPRS, 5. provide the analysts with unedited "raw" (MT) output, 6. provide the analysts with postedited MT, or 7. use a system of machine-aided translation. Throughout the subsequent discussion, the Committee has relied heavily on the cost figures developed by Arthur D. Little, Inc., and contained in An Evaluation of Machine-Aided Translation Activities at FTD [Contract AF 33(657)-13616, May 1, 1965]. References to this study are indicated below by (ADL) followed by the appropriate page number.
IN-HOUSE TRANSLATORS At the Foreign Technology Division, the in-house translators work at a rate of about 240 Russian words per hour (ADL, p. 29), yielding a daily output of approximately 2,000 words. Thus one translator can produce enough to keep two analysts in translations. 58
Since ADL estimates (ADL, p. 21) that the cost for in-house translation is $22.97 per 1,000 Russian words, the cost for 11,000,000 Russian words would be $252,670. We assume that direct costs were included in this figure ($5.60 per hr) for translator time. Other costs that must be included in this type of operation are those of space, equipment, recomposition, and proofreading and review.
MAJOR COSTS FOR IN-HOUSE HUMAN TRANSLATION 25 Translators' salaries and direct cost overhead Recomposition ($14.15 per 1,000 words, ADL, p. 21) Proofreading and review ($2.97 per 1,000 words, ADL, p. 21) Duplication checks
$252,670 155,650 32,670 ?
Total
$432,990
IN-HOUSE TRANSLATION EMPLOYING DICTATION The Committee's study described in Appendix 14 revealed that the average typing speed of the translator was only 18 words a minute and that typing took approximately 25 percent of the total time needed to produce the translation. It would seem then to be advantageous to use the translator for translating and to use trained typists to do the typing. One agency (see Appendix 1, page 35) found that on suitable texts (those with few graphics to be inserted), the daily output of the translator was doubled. A typist trained in the use of dictating equipment can type about 8,000 words of English per day. To convert this to the number of Russian words one must employ a factor of 1.35 English words per Russian word. Thus the 8,000 English words would represent 6,000 words of original Russian text. If the over-all output of the translator were to be increased by as little as 25 percent, his output would amount to 2,500 words per day. At this rate of output, only 20 translators would be needed instead of 25, and about eight typists would be needed to keep up with the output of the translators. Although some savings are realized from this type of system, owing to the fact that typists are paid at about half the rate of translators, such savings are offset to some extent by the additional space and equipment required. It seems likely, however, that the use of this system would result in a more attractive product, the ropy having been prepared by well-trained typists. Furthermore, an estimated increase of only 25 percent, upon which we have 59
based our computations, may be unduly conservative. If this is so— and the Committee would like to see studies made to determine more accurately the actual advantages of various systems—the dictation method would be even more attractive.
CONTRACT TRANSLATION Since contract translation costs vary widely, we will once more base our computations on data in the Arthur D. Little, Inc., report. The ADL team found that the cost per 1,000 Russian words was $24.57 for the translation process, $5.40 for insertion of graphics, and $2.97 for proofreading and review, or a total of $32.94 (ADL, p.21). The Committee has been told by a reliable and knowledgeable individual connected with the translation at FTD that the proofreading and review procedure was unnecessary since the translations produced by the contractor were of excellent quality. Trusting this individual's judgment, but at the same time being aware that the ADL report is a careful study of what practices were in force (regardless of their necessity or degree of efficiency) at FTD, the Committee conjectured that $1.50 per 1,000 Russian words, rather than $2.97, might be a reasonable cost for the proofreading and review procedure; therefore, our computation differs from the ADL study. It is a fact that contractors have a lower overhead than in-house translators, and it is hoped that the significance of this item will not be overlooked by the reader. An annual production of 11,000,000 Russian words by contract would cost the using agency $270,270 59,400 16,500
for translation for graphics for proofreading and review
$346,170
Total
Since the average document to be translated is about 8,000 (Russian) words in length (ADL, p. A-8), our hypothetical agency would have to handle and control only six or seven documents a day, and few or no additional personnel would be needed for this task. Thus the $346,170 estimated above would approximate the total cost.
THE JOINT PUBLICATIONS RESEARCH SERVICE (JPRS) The JPRS (Appendix 3) utilizes subject matter specialists who work at home on a part-time, contract basis. Thus, JPRS is able to 60
handle a large quantity of translations in many languages in many fields at low rates. Because it does handle a large quantity of translations, JPRS is able to charge the same price for all translations regardless of subject matter or language. The current price is $16 per 1,000 words of English. Applying the factor of 1.35 English words for each Russian word, one can see that 11,000,000 Russian words are the equivalent of 14,850,000 English words and that, therefore, the JPRS charge for such translation would amount to $237,600. Once again, as with any contract translation, the number of additional personnel would be minimal, and the cost above would be close to the true cost.
UNEDITED MACHINE TRANSLATION (MT) The development of an MT program capable of producing translations of such a quality that they would be useful to the reader without requiring the intervention of a translator anywhere in the process has long been the goal of researchers in MT. As far as the Committee can determine, two attempts have been made to give analysts "raw" or unedited machine output. Neither proved to be satisfactory. The FTD experience is stated with admirable succinctness: "This [acceptance of postedited MT] marks a considerable change in attitude toward MT's which, in their earlier unedited form, were generally regarded as unsatisfactory" (ADL, p. F-5). We have worked out a simple equation that shows how many dollars may be saved by using the unedited machine output. Let CH = cost of human translation (dollars/1000 words), CM = cost of MT (dollars/1000 words), W
= loaded salary of user of the translation (dollars/hr),
TH = reading time for human translation (hr/1000 words), TM = reading time for MT (hr/1000 words), N
= number of people who read the translation,
S
= saving by MT (dollars/1000 words).
Then S =
CH - CM - WN (TM - TH)
Presumably th e saving would be greatest if the reader merely read machine print-out, referring to th e untranslated original for
61
figures and equations. Here the cost of machine output could best be compared, not with the cost of JPRS translations, but with the cost of dictated and uncorrected human translations, either voice on tape, or a typewritten transcription of the tape. As we have pointed out in Appendix 1, such translation can be carried out several times as fast as "full translation." Unfortunately, we do not know what the costs are for translations that are dictated but not typed. It would seem likely, however, that savings would be substantial, since there would be no costs (a) for typist-transcriptionists or (b) for recomposition. Whether the savings involved would be offset by increased difficulty of use by the analyst is not known. Although the analyst would not be presented with a written translation, he would at least be assured of having all the words translated, unlike the raw MT output. Most translations are apparently read by more than one reader. According to one agency, the preparation of 175 copies of a translation for distribution is standard for documents that appeared originally in the open literature and this distribution accounts for about 90 percent of the documents translated. For the remaining 10 percent (the classified documents) only one copy is prepared, but the requester has the privilege of making as many copies as he deems fit. Even more astonishing is the estimate of the Arthur D. Little, Inc., team that "about 615 members of the Air Force R & D community (40,000 members) would be expected to have a common interest in the average translated document" (ADL, p. F-9). It was shown by John B. Carroll, in the study that he did for the Committee (see Appendix 10), that the average reader tested took twice as long to read raw MT as he did to read a human translation. The ADL team found that the average reading rate of those tested was 200 words per minute for well-written English (ADL, p. D-6) or 0.08 hr per 1,000 words. From these two studies we determined the reading rate for raw MT to be 100 words per minute or 0.16 hr per 1,000 words. Raw MT should be compared, as has been mentioned, with an equally inelegant product. But the Committee has no idea of the cost of a comparable product or the time required to read (or listen to) it, and these factors are crucial in the calculation of savings according to our equation. Prudence demands that we compare raw MT with a product about which we have more certain knowledge concerning cost and reading rates even though such translations are of higher quality. For the purposes of comparison, we have chosen the JPRS for the simple reasons that (1) it is relatively inexpensive and (2) the costs are known and stable. Applying our equation, we have 62
CH = $21.60 (the JPRS cost per 1,000 Russian words, the conversion factor of 1.35 being applied to $16.00, the cost per 1,000 English words), CM = $7.63 [input typing $4.09, machine costs $3.21, output typing $0.33 (ADL, p. 20)], W
= $10.00 [$12,000 salary per annum ÷ 220 working days = $60.00, $60.00 + (60/3) (direct costs) = $80.00 loaded salary per day, $80.00 ÷ 8 = $10.00 (loaded salary per hour)], TH = 0.08, TM = 0.16.
Utilizing the figures above, but varying N (the number of readers), we arrive at the savings made by the use of raw output. If the number of readers is 1: S = $21.60 - 7.63 - [(10 × 1) (0.16 - 0.08)], S = $21.60 - 7.63 - 0.80, S = $13.17. If the number of readers is 10: S = $5.97. If the number of readers is 15: S = $1.97. If the number of readers is 17: S = $0.37. If the number of readers is 18: S = -$0.43. If the number of readers is 20: S = -$2.03. If the number of readers is 80: S = -$40.13. If the number of readers is 175: S = -$127.03. If the number of readers is 615: S = -$478.13.
Obviously, the break-even point occurs between 17 and 18 readers. But we have seen that, in one agency at least, about 90 percent of the translations are distributed to 175 readers, whereas only 10 percent are prepared for a single reader. By simple computation it can be determined that whereas the use of JPRS for all translation would result in a loss of $14,487, the use of MT for all translation would result in a loss of $1,257,597. It might be argued that MT is still economical when used to provide translations that 63
are user-limited; but, since relatively few translations seem to be destined for use by less than 18 readers, the volume would probably be too small to warrant the maintenance of an elaborate computer facility with its attendant personnel. To the Committee, machine output (such as that shown on pages 20-23) seems very unattractive. We believe that the only valid argument for its use would be a compelling economic argument. If it can be shown that the use of unedited machine output, taking proper account of increased reading time on the part of the readers, would result in worthwhile savings over efficient human translation of the most nearly comparable kind, then there is a cogent reason for using unedited MT. But, unless such a worthwhile saving can be convincingly demonstrated, we regard the use of unedited machine output as regressive and unkind to readers. In considering the cost of producing unedited machine output we must use the real current cost. It is nice to think that savings may be made someday by using automatic character recognition, but actual savings should be demonstrated conclusively before machine output is inflicted on users in any operational manner.
POSTEDITED MACHINE TRANSLATION (MT) To provide 11,000,000 words of postedited Russian-to-English MT per year would cost $397,980 [$36.18 per 1,000 Russian words (ADL, p. B-7)]. This estimate should be regarded as a very low one, since the ADL team did not include overhead costs (ADL, p. 3). ADL figures (ADL, p. E-5) that for 100,000 words per day, 44 individuals would be required; for input typing, 14; for machine operation, 1.6; for output typing, 1.4; and for postediting, 28. Since we are assuming a 50,000-word-per-day consumption, we will halve this estimate, giving a total of 22 personnel. The point the Committee would like to make in this connection is that since 22 personnel would be required, 14 of whom (the posteditors) have to be proficient in Russian, one might as well hire a few more translators and have the translations done by humans. Another, perhaps better, alternative would be to take part of the money spent on MT and use it either (1) to raise salaries in order to hire bilingual analysts—thus avoiding translation altogether—or, (2) to use the money to teach the analysts Russian.
MACHINE-AIDED TRANSLATION (M-AT) We will call M-AT any system of human translation that utilizes the computer to assist the translator and that was designed originally 64
for such a purpose. A system such as that at the FTD might properly be called human-aided machine translation, since the postediting process was added after it became apparent that raw output was unsatisfactory and since humans are employed essentially to make up for the deficiencies of the computer output. Specific costs for the two types of M-AT systems in operation (see Appendixes 12 and 13) are not known to the Committee, but from the given figures that show the proportion of translator time saved, it is possible to make some rough estimates. Both the Federal Armed Forces Translation Agency and the European Coal and Steel Community indicate that a saving of about 50 percent of the translator's time could be expected by the use of a machineaided system. Since translators' salaries constitute the largest item in the budget for a human-translation facility, such savings would probably be substantial. Input typing costs would not be as great as those at FTD, where the entire document to be translated is keypunched, since only the individual words or sentences with which the translator desires help are keypunched. Furthermore, the programming involved is relatively simple and small, and inexpensive computers are adequate. The relatively modest increases in staff, equipment, and money necessary for the production of translator aids are likely to be offset by the increase in quality of the product. It is possible, therefore, that the savings of an M-AT system might approach 50 percent of the cost of translator salaries in a conventional human-translation system. If this estimate is sound, then the cost for an M-AT system to produce 11,000,000 words of Russian-toEnglish translation would be $314,655 ($126,335 for salaries, $155,650 for recomposition, $32,670 for proofreading and review).
SUMMARY Throughout our discussion of costs, we have been conscious of the fact that we were not in possession of all the necessary data. We present the following estimates with diffidence and would welcome any studies that would more precisely determine actual translation costs and quality, whether they affirm or deny the validity of our estimate.
65
ESTIMATES OF COSTS AND QUALITY FOR VARIOUS TYPES OF TRANSLATION Type
Quality
In-house (conventional translation) Good In-house (dictation) Good Contract Fair to good JPRS Fair Raw MT Unsatisfactory Postedited MT Fair M-AT Excellent Analysts proficient in Russian -
Cost for 11,000,000 Russian Words $ 440,000 440,000350,000 240,000 80,000 + 400,000 310,000 0
CONCLUSION Since no one can be proficient in all languages, there will always be a need for translation. Yet, publication is not evenly distributed among the some 4,000 languages of the world, and this is especially so in the areas of science and technology. Russian-to-English translation constitutes a large part of the total translation done in the United States, and there are no signs that this situation is likely to change radically in the foreseeable future. This being the case, the present policy of using monolingual analysts and providing them with translations year after year seems lacking in foresight, particularly since the time required for a scientist to learn a foreign language well enough to read an article in his own field of specialization is not very long, and since the facilities are available to train him. In our hypothetical agency, the costs of providing fair and good translations were from 30 to 55 percent greater than the estimated costs of a facility using analysts proficient in Russian. To allow heavy users of Soviet literature to continue to rely on translations seems unwise.
66
Appendix 10 An Experiment in Evaluating the Quality of Translations
This experiment* was designed to lay the foundations for a standard procedure for measuring the quality of scientific translations, whether human or mechanical. There have been other experiments on this problem [e.g., G. A. Miller and J. G. BeebeCenter, Mechan. Transl. , 3, 73 (1958); S. M. Pfafflin, Mechan. Transl. 8, 2 (1965)], but their methods for evaluating translations have been too laborious, too subject to arbitrariness in standards, or too lacking in reliability and/or validity to become generally accepted. The measurement procedure developed here gives promise of being amenable to refinement to the point where it will meet the requirements of relative simplicity and feasibility, fixed standards of evaluation, and high validity and reliability. A detailed report of this experiment will be submitted for publication elsewhere; the present brief report will serve to indicate the general nature of the measurement procedure and some of the chief results.
THE MEASUREMENT PROCEDURE It was reasoned that the two major characteristics of a translation are (a) its intelligibility, and (b) its fidelity to the sense of the original text. Conceptually, these characteristics are independent ; that is, a translation could be highly intelligible and yet lacking in fidelity or accuracy. Conversely, a translation could be highly accurate and yet lacking in intelligibility; this would be likely to occur, however, only in cases where the original had low intelligibility. Essentially, the method for evaluating translations employed in this experiment involved obtaining subjective ratings for these two characteristics—intelligibility and fidelity—of sentences selected * Conducted by John B. Carroll with funds provided by the Automatic Language Processing Advisory Committee.
67
randomly from a translation and interspersed in random order among other sentences from the same translation and also among sentences selected at random from other translations of varying quality. When a translation sentence was being rated for intelligibility, it was rated without reference to the original. "Fidelity" was measured indirectly: the rater was asked to gather whatever meaning he could from the translation sentence and then evaluate the original sentence for its "informativeness" in relation to what he had understood from the translation sentence. Thus, a rating of the original sentence as "highly informative" relative to the translation sentence would imply that the latter was lacking in fidelity. All ratings were made by persons who were specially selected and trained for this purpose. There were two sets of raters. The first set of raters (called here "monolinguals" for convenience) consisted of 18 native speakers of English who had no knowledge of the language of the original (Russian, in this case). They were all Harvard undergraduates with high tested verbal intelligence and with good backgrounds in science. In rating "informativeness" these raters were provided with carefully prepared English translations of the original sentences, so that in effect they were comparing two sentences in English—one the sentence from the translation being evaluated, and the other the carefully prepared translation of the original. The second set of raters ("bilinguals") consisted of 18 native speakers of English who had a high degree of competence in the comprehension of scientific Russian. Their ratings of the intelligibility of the translation sentences may well have been influenced by their knowledge of the vocabulary and syntax of Russian; at any rate, no attempt was made to prevent them from using such knowledge. To rate "informativeness," they made a direct comparison between the translation sentence (in English) and the original version. All ratings were made on nine-point scales that had been established by the writer prior to the experiment by an adaptation of a psychometric technique known as the method of equal-appearing intervals. Thus, points on these scales could be assumed to be equally spaced in terms of subjectively observed differences. In the case of the intelligibility scale, each of the nine points on the scale had a verbal description (see Table 4). The same was true of the "informativeness" scale except that verbal descriptions were omitted for a few of the points (sec Table 5). In this way each degree on the scales could be characterized in a meaningful way. For example, point 9 on the intelligibility scale was described 68
TABLE 4. Scale of Intelligibility 9—Perfectly clear and intelligible. Reads like ordinary text; has no stylistic infelicities. 8—Perfectly or almost clear and intelligible, but contains minor grammatical or stylistic infelicities, and/or midly unusual word usage that could, nevertheless, be easily "corrected." 7—Generally clear and intelligible, but style and word choice and/or syntactical arrangement are somewhat poorer than in category 8. 6—The general idea is almost immediately intelligible, but full comprehension is distinctly interfered with by poor style, poor word choice, alternative expressions, untranslated words, and incorrect grammatical arrangements. Postediting could leave this in nearly acceptable form. 5—The general idea is intelligible only after considerable study, but after this study one is fairly confident that he understands. Poor word choice, grotesque syntactic arrangement, untranslated words, and similar phenomena are present, but constitute mainly "noise" through which the main idea is still perceptible. 4—Masquerades as an intelligible sentence, but actually it is more unintelligible than intelligible. Nevertheless, the idea can still be vaguely apprehended. Word choice, syntactic arrangement, and/or alternative expressions are generally bizarre, and there may be critical words untranslated. 3—Generally unintelligible; it tends to read like nonsense but, with a considerable amount of reflection and study, one can at least hypothesize the idea intended by the sentence. 2—Almost hopelessly unintelligible even after reflection and study. Nevertheless, it does not seem completely nonsensical. 1—Hopelessly unintelligible. It appears that no amount of study and reflection would reveal the thought of the sentence.
as follows: "Perfectly clear and intelligible. Reads like ordinary text; has no stylistic infelicities." Point 5 (the midpoint of the scale): "The general idea is intelligible only after considerable study, but after this study one is fairly confident that he understands. Poor word choice, grotesque syntactic arrangement, untranslated words, and similar phenomena are present, but constitute mainly 'noise' through which the main idea is still perceptible."
PREPARATION OF TEST MATERIALS AND COLLECTION OF DATA The measurement procedure was tested by applying it to six varied English translations--three human and three mechanical — 69
TABLE 5. Scale of Informativeness
(This pertains to how informative the original version is perceived to be after the translation has been seen and studied. If the translation already conveys a great deal of information, it may be that the original can be said to be low in informativeness relative to the translation being evaluated. But if the translation conveys only a certain amount of information, it may be that the original conveys a great deal more, in which case the original is high in informativeness relative to the translation being evaluated.) 9—Extremely informative. Makes "all the difference in the world" in comprehending the meaning intended. (A rating of 9 should always be assigned when the original completely changes or reverses the meaning conveyed by the translation.) 8—Very informative. Contributes a great deal to the clarification of the meaning intended. By correcting sentence structure, words, and phrases, it makes a great change in the reader's impression of the meaning intended, although not so much as to change or reverse the meaning completely. 7—(Between 6 and 8.) 6—Clearly informative. Adds considerable information about the sentence structure and individual words, putting the reader "on the right track" as to the meaning intended. 5—(Between 4 and 6.) 4—In contrast to 3, adds a certain amount of information about the sentence structure and syntactical relationships; it may also correct minor misapprehensions about the general meaning of the sentence or the meaning of individual words. 3—By correcting one or two possibly critical meanings, chiefly on the word level, it gives a slightly different "twist" to the meaning conveyed by the translation. It adds no new information about sentence structure, however. 2—No really new meaning is added by the original, either at the word level or the grammatical level, but the reader is somewhat more confident that he apprehends the meaning intended. 1—Not informative at all; no new meaning is added, nor is the reader's confidence in his understanding increased or enhanced. 0—The original contains, if anything, less information than the translation. The translator has added certain meanings, apparently to make the passage more understandable.
of a Russian work entitled Mashina i Mysl' (Machine and Thought), by Z. Rovenskii, A. Uemov, and E. Uemova (Moscow, 1960). These translations were of five passages varying considerably in type of content. (All the passages selected for this experiment, with the original Russian versions, have now been published by the Office of Technical Services, U.S. Department of Commerce, Technical 70
Translation TT 65-60307.) The materials associated with one of these passages were used for pilot studies and rater practice sessions; the experiment proper used the remaining four passages. In preparing materials for the rating task, 36 sentences were selected at random from each of the four passages under study. Since six different translations were being evaluated, six different sets of materials were prepared (in two forms, one for the monolinguals and one for the bilinguals) in such a way that each set contained a different translation of a given sentence. In this way no rater evaluated more than one translation of a given sentence. Each set of materials was given to three monolinguals and to three bilinguals; thus, there were 18 monolinguals and 18 bilinguals. Each rater had 144 sentences to evaluate first for intelligibility and then for the informativeness of the original (or the standard translation of it) after the translation had been seen. The raters required three 90-min sessions to complete this task, dealing with 48 sentences in each session. The raters were not informed as to the source of the translations they were rating, although they were told that some had been made by machine. Before undertaking this task, the raters attended a 1-hr session in which they were given instruction in the rating procedures and required to work through a 30-sentence practice set. During the rendering of ratings for intelligibility, the raters held stopwatches on themselves to record the number of seconds it took them to read and rate each sentence.
RESULTS The results of the experiment can be considered under two headings: (a) the average scores of the various translations, and (b) the variation in the scores as a function of differences in sentences, passages, and raters. Table 6 gives the over-all mean ratings and time scores for the six translations, arranged in order of general excellence according to our data. Consider first the mean ratings for intelligibility by the monolinguals. Translation 1, a published human translation that had presumably been carefully done, received the highest mean rating, 8.30, on the scale established in Table 4. But 8.30 is still appreciably different from the maximum possible mean rating of 9.00, and it is evident that not even this "careful" human translation was as good as one might have expected. Furthermore, the mean rating of Translation 1 is not significantly different from that of Translation 4 (8.21), a "quick" human translation made by rapid dictation 71
procedures. The mean ratings of Translations 1 and 4 do, however, differ significantly from the mean rating (7.36) of Translation 2, another "quick" human translation. It may be concluded that the measurement procedure studied here is sensitive enough to differentiate among human translations. A similar remark may be made about the sensitivity of this procedure to differences in the intelligibility of machine translations. Translations 7 and 5 were shown to be significantly more intelligible, on the average, than Translation 9. Of most current interest, however, are the results having to do with the comparison of the human and the machine translations. Machine translations 7, 5, and 9 received mean ratings, respectively, of 5.72, 5.50, and 4.73. A scale value of 5 refers to a translation in which "the general idea is intelligible only after considerable study, but after this study one is fairly confident that he understands ..." All these machine translations are significantly less intelligible, on the average, than any of the three human translations. As machine translations improve, it should be possible to scale them by the present rating procedure to determine how nearly they approach human translations in intelligibility. The monolinguals' mean ratings on "informativeness" (reflecting the lack of fidelity of the translations) show an almost perfect inverse relationship to the mean ratings on intelligibility, and they differentiate the various translations in the same way and to the same extent. This result means that in practice, when ratings are averaged over sentences, passages, and raters, "intelligibility" and "fidelity" are very highly correlated. The detailed results of this study show that only in the case of a few particular sentences do the mean ratings of intelligibility and informativeness convey different information. Furthermore, the mean reading times per sentence show almost precisely the same pattern of results as the ratings. In fact, the mean reading times are linearly related to the mean ratings, a result that supports the conclusion that the points on the rating scales are evenly spaced. The results from the ratings by bilinguals contribute nothing more to the differentiation of the translations than is obtainable with the monolinguals' ratings. Bilinguals' intelligibility ratings of the translations are slightly (and significantly) higher, on the average, than those of the monolinguals, and correspondingly, their informativeness ratings are slightly lower. Yet, they took significantly longer to read and rate the sentences. Apparently their knowledge of Russian caused them to work harder on trying to understand the translations. One is inclined to give more credence to the results 72
from the monolinguals because monolinguals are more representative of potential users of translations and are not influenced by knowledge of the source language. It is also to be noted that the data from the monolinguals differentiate the translations to a somewhat greater extent than do the data from the bilinguals. The results concerning the differences in ratings due to differences in sentences, passages, and raters can now be considered. (The detailed tables of these results are omitted here to save space.) The more important results may be summarized as follows: 1. The results do not differ significantly from passage to passage; that is, on the average the various passages from a given translation receive highly similar ratings. For intelligibility ratings, however, there is a small but significant interaction between translation and passage, indicating that translations are to some extent differentially effective for different types of content. (This interaction effect is present both for human and for machine translations.) 2. There is a marked variation among the sentences. In fact, as may be seen from Figure 1, there is some overlap between sentences from human translations and from mechanical translations; or, in other words, there are some sentences translated by machine that have higher ratings than some other sentences translated by human translators, even though, on the average, the humantranslated sentences are better than the machine-translated ones. These results imply that in order to obtain reliable mean ratings for translations, a fairly large sample of sentences must be rated. 3. Variation among raters is relatively small, but it is large enough to suggest that ratings should always be obtained from several raters—say at least three or four.
CONCLUSION This experiment has established the fact that highly reliable assessments can be made of the quality of human and machine translations. In the case of the six particular translations investigated in the study, all the human translations were clearly superior to the machine translations; further, some human translations were significantly superior to other human translations, and some machine translations were significantly superior to other machine translations. On the whole, the machine translations were found to fall about at the midpoint of a scale ranging from the best possible to the poorest possible translation. 73
What is still needed, however, is a system whereby any translation can be easily and reliably assessed. The present experiment has determined the necessary parameters of such a system.
FIGURE 1. Frequency distribution of monolinguals' mean intelligibility ratings of the 144 sentences in each of six translations. Translations 1, 4, and 2 are human translations; Translations 7, 5, and 9 are machine translations.
74
Appendix 11 Types of Errors Common in Machine Translation
Two studies have recently been made of the types of errors made in mechanical translation. The first study was very kindly made available to the Committee by the IBM Thomas J. Watson Research Center, Yorktown Heights, New York. By counting and classifying the corrections made by posteditors, this study determined the types and frequency of errors found in the output of four machine translations (Russian to English).
GENERAL CLASSIFICATION AND PERCENTAGE OF ERRORS OF ARTICLE I Total number of words:
Approximately 1,200
Translite rated words Multiple meanings and ambiguities W o r d o r d e r r e a r r a n ge d Misce llane ous inse rtions and corre ctions Total
No. — 96 23 45 164
% — 8.0 2.0 3.6 13.6
GENERAL CLASSIFICATION AND PERCENTAGE OF ERRORS OF ARTICLE II Total number of words:
Approximately 1,200
Tr an slite r ate d w or d s Multiple meanings and ambiguities Word order rearranged Misce llane ous inse rtions and corre ctions Total
No. 6 132 17 77 232
76
% 0.5 11.0 1.4 6.4 19.3
GENERAL CLASSIFICATION AND PERCENTAGE OF ERRORS OF ARTICLE III Total number of words: Transliterated words Multiple meanings and ambiguities Word order rearranged Miscellaneous insertions and corrections Total
Approximately 1,700 % 1 9 2 7 19
No. 17 143 36 122 318
GENERAL CLASSIFICATION AND PERCENTAGE OF ERRORS OF ARTICLE IV Total number of words (including individual digits and symbols in all formulas): Transliterated words Multiple meanings and ambiguities Word order rearranged Miscellaneous insertions and corrections Total
Approximately 1,600 % — 5.8 0.9 29.0 35.7
No. 1 87 14 436 538
The second study was made by Arthur D. Little, Inc., and was done in a manner similar to the IBM study. That is, machine translation output was postedited and the errors classified and counted. From the study, the A. D. Little group was able to tell the percentage of total corrections made in each category. The original consisted of approximately 200 pages of scientific Russian. One set of approximately 100 pages was edited by two different editors. The second set contained "approximately 100 pages from seven MT articles edited by at least four different editors."*
* An Evaluation of Machine-Aided Translation Activities at F.T.D., Contract AF 33(657)-13616, Case 66556, May 1, 1965, p. G-10.
77
PERCENTAGE OF TOTAL CORRECTIONS COUNTED* Error
%
Word omission A. Articles B. Others
18.76 15.98 34.74
Wrong words A. Prepositions B. Verb tense, voice, suffix C. Others
3.78 5.56 16.24 25.58
Russian left in
4.48
Choice A. Choice of two B. Choice of two, both wrong
8.17 3.57 11.74
Unnecessary word
3.09
Symbol
4.5
Phrase not interpreted
3.14
Word order
12.73
Total Number of Corrections:
*Ibid., p. G-17.
78
7,573
Appendix 12 Machine-Aided Translation at the Federal Armed Forces Translation Agency, Mannheim, Germany
SEMIAUTOMATIC TRANSLATION AID SYSTEM (STAGE 1) Translated from German by the Federal Armed Forces Translation Agency, Annex to Report MüV - Az.: 55-05 (30) dated, February 18, 1965.
Report on Sixth Test Run On TR4 Computer Facility I. GENERAL During the week of February 8 to 12, 1965, a second improved model test run was conducted using the TR4 computer facility of the Telefunken Company, Konstanz. The test run was designed to test as an integral system all routines and subroutines developed so far. The test, which represents the culmination of the development work done in Stage I of the semiautomatic translation aid system, can be regarded as quite successful: it confirmed the soundness of the approach. Practical application of the procedure (Stage I) now depends on when the Federal Armed Forces Computer Center is operational so that the entire body of linguistic information now stored on punched cards can be transferred to magnetic tape. Optimization of the program will be effected on the basis of practical experience.
II. DESCRIPTION OF TEST RUN The testing material consisted of three English-language texts (socalled partial interrogation batches). The texts bore different job numbers and were assigned to different translators who underscored in the text those terms with which the machine was to be presented. Double or triple underscorings of compounds meant that in addition to the translation of the compound itself the 79
translation of one or more of its elements was desired in order to utilize optimally the information stored in the machine dictionary. Where appropriate, the underscored expressions were reduced to the reference form (nominative singular, infinitive, etc.). The terms were then punched on cards and read into the computer in the sequence of their occurrence in the text. Read-in of the three partial interrogation batches was in the sequence of ascending job numbers. The dictionary used in this text did not contain the entire A-to-Z stock of vocabulary but was a microglossary specially compiled for the purposes of this test. This fact already points to the model character of the test. The output units were printed out by an OFF-LINE high-speed printer. This second model test run differed from the first model test run [cf. Report ÜDBw - MÜV - Az.: 55-05 (30) dtd 14 Oct. 1964] in that it presupposed large quantities of data. While in the first test sorting had been circumvented, the second test included a sorting (SORT-2) program using four magnetic tapes. Since the sorting procedure has already been discussed in Report ÜDBw - MÜV Az.: 55-05 (30) of 10 Dec. 1964, it need not be described here.
III. FORMAT OF OUTPUT LISTS What has been said about the format of the output lists in Reports ÜDBw - MÜV - Az.: 55-05 (30) of 14 Oct. and 10 Dec. 1964 is true also for the output lists produced in the present test with the exceptions that in the present test the lists have a title line and each partial interrogation batch begins on a new page. Print-out of more than one partial batch is in the sequence of the alphabetical order of the abbreviated names of the translators.
IV. INTERPRETATION OF SOME "MISSING" NOTATIONS 1. The missing notations, some of which were introduced intentionally for reasons of illustration, are attributable to the following causes: a. Interrogation of compounds with variable context-related elements Examples: freak midget craft (GRE 8969 034) midget-type submarine (GRE 8969 043) cyclic control system (HER 8970 029) low-power gain recovery (MUL 8968 038) In some cases interrogation without the variable elements was successful. b. Interrogation of words and word compounds which occur as "quasi-technical terms" in certain contexts and which because of their elusive character are not contained in the dictionary 80
Examples:
ASW package (GRE 8969 025) oscillatory mode (HER 8970 005) hydraulically boosted (HER 8970 037) distributed fashion (MUL 8968 030) c. Spelling variants Examples: antisubmarine air barrier (GRE 8969 047) travelling-wave maser (MUL 8968 012) Interrogation of the alternative spellings (anti-submarine air barrier; traveling-wave maser) was successful. d. Interrogation of expressions which, strictly speaking, cannot be regarded as technical terms Examples: porpoise (GRE 8969 036) ocean passage (GRE 8969 049) stocking (HER 8970 024) e. Uncorrected punching errors Examples: artificial feedback (HER 8970 040) artificial feel (HER 8970 042) f. Inaccuracies in the original text In text 64/18968, line 23, the letters "bL" were interpreted as an abbreviation. However, they are not an abbreviation but the product of the two quantities "b" and "L." For the sake of clarity the product should have been written "b × L." g. All other "missing" notations may be interpreted as blanks in the dictionary Examples: advance radar picket (GRE 8969 019) missile-launcher (GRE 8969 045) stability augmentation (HER 8970 002) artificial feedback feel (HER 8970 039) maser line (MUL 8968 013) gain recovery (MUL 8968 039) In many cases, however, the missing equivalents could have been derived from the information actually printed out. 2. The justification of the warning to the translator not to accept blindly everything printed out by the machine is demonstrated by the following examples: a. Text 64/18969, line 12: "weather beacon." The German equivalents "Wetterboje" and "Wetterbake" (GRE 8969 021) printed out by the machine are not very meaningful in this particular context. A destroyer may rather serve as a "Wetterstation (weather station)" or "Wetterschiff (weather ship)." b. Text 64 /18970, line 18: "loop." What is meant here is a "servo loop" ("Regelkreis"); the word "loop" without a qualifying addition is not specific enough. The equivalents under "loop" (HER 8970 028), therefore, are not applicable. 81
c. Text 64/18970, line 28: "displacement." The equivalents printed out under HER 8970 038 are wrong in this context. The weaknesses pointed up above are not to be blamed on the machine or the procedure but are inherent in the language.
V. OUTLOOK Practical application of the procedure developed so far, a procedure proven in a second successful model test run, now depends on when the Federal Armed Forces computer can be used in order to transfer the entire punch-card information onto magnetic tape. Organizational and programming preparatory work for this significant step are already under way. In addition, work on the new complex "processing of vocabulary passed by the terminology boards" has been initiated.
TEXT-RELATED GLOSSARIES AND MACHINE-PRODUCED ENGLISH-LANGUAGE TECHNICAL TEXTS (1)
(5)
(10)
(15)
(20)
(25)
(30)
One common practice is to credit any ship with a hull number starting with D as being per se an ASW ship. To be sure, destroyers (DD), escorts (DE), and frigates (DL) all have ASW capabilities. So do all other types of ships. The bow of an ocean liner, if it rammed a submarine, would be a mighty ASW weapon. This does not make merchant ships into an ASW force. Is a guided missile destroyer (DDG), or a radar escort picket (DER), any more an ASW craft? Ships are inherently multi-purpose, even when efforts are made to specialize their functions. The versatile destroyer, our traditional ASW surface craft, can and does serve as anti-air screen, advance radar picket, torpedo boat, weather beacon, and even as an emergency power plant for a good-sized city. It even makes an effective transport and cargo ship. Into the "ASW package" (lately broadened into something called undersea warfare, or USW) have gone a hodge-podge of ships. And a potpourri of projects have been labelled ASW, including such things as mines and mine detectors, noisemakers and deception devices, submarine machinery, test barges and calibration ranges, hydrographic and oceanographic surveys, long-range basic programs . . . , bathyscaphs, freak midget craft, and studies of the vocabulary of porpoises. War will demand several rather different ASW missions. The tactics of convoy protection differ from those of a hunter-killer group free to pursue subs wherever they may be found. The problem of guarding an amphibious landing perimeter against coastal or midget-type submarines has little in common with the hunting down of silent missile-launchers hovering deep in unfrequented waters. Maintaining an antisubmarine air barrier across critical ocean passages differs markedly from all these.
82
83
84
85
86
Appendix 13 Machine-Aided Translation at the European Coal and Steel Community, Luxembourg
CONGRES SUR L'UTILISATION DE L 'ACIER KONGRESS ÜBER STAHLVERWENDUNG CONGRESSO SULL'UTILIZZAZIONE DELL'ACCIAIO CONGRES OVER DE TOEPASSING VAN STAAL STEEL UTILIZATION CONGRESS
TERMES TECHNIQUES FACHWÖRTER TERMINI TECNICI VAKTERMEN TECHNICAL TERMS
EUROPEAN COAL AND STEEL COMMUNITY HIGH AUTHORITY Terminological bureau
FOREWORD This glossary has been compiled by the High Authority’s Terminological Bureau for the Congress on Steel Utilization scheduled to meet in Luxembourg from October 28 to 30, 1964. Use has been made of modern data-processing techniques *), which have enabled the difficulties of assembling and analysing material from a variety of countries in a bare three months to be successfully overcome, though, needless to say, in the circumstances, the fivelanguage glossary can make no claim to be exhaustive. In an effort to make for easier consultation, the terms have been grouped under headings corresponding to the items of the Congress programme. A somewhat arbitrary classification has, however, resulted, so that users not finding a term under one heading are recommended to try under a related one. In each case the key word is immediately followed by the search arguments (i.e. key word plus any qualifying matter), and then by the whole phrase from which the term is taken, with the equivalent phrases in the other language; the search argument is, however, of minor importance to the user.
In the assembly of the material the Bureau received most valuable assistance from various information centres in the countries approached, the documentation supplied by whom is listed in the accompanying bibliography. Special thanks for assistance with terminological problems are due to the library of the Technische Hogeschool, Delft, and the Centre Belgo-Luxembourgeois d’Information de l’Acier, Brussels. Although initially intended as an aid for the numerous interpreters and translators who will be called upon to grapple with the highlyspecialised Congress papers and discussions, the glossary may well prove of interest to wider circles. It is issued in five versions, German, French, Italian, Dutch and English, and will be supplied on request. Queries and suggestions will be welcomed, and should be addressed to Mr.J.A.BACHARACH, Head of the High Authority Terminological Bureau. Luxembourg, October 5, 1964
T.F.NOYON Director of Internal Affairs *) Offset reproduction of listings obtained from a KWIC-programmed IBM 1410 computer.
87
Appendix 14 Translation Versus Postediting of Machine Translation
This study reports the results of a small experiment done for the purpose of obtaining some facts regarding the process of postediting machine-translation output as compared with the process of ordinary translation. In particular, information was desired concerning the relative speed and ease (or difficulty) of postediting as compared with those of translation. A variety of translators (i.e., commercial free-lance translators, government in-house translators, government contract translators, and bilingual persons who did not ordinarily engage in translation work) were sent a packet containing (1) a 1,135-word excerpt from a Russian book on cybernetics, Machina i Mysl', which they were to translate and provide typed copy of their translations; (2) a 765-word excerpt from the same book; (3) a print-out of the machine translation of (2), which was to be postedited and typed; and (4) a questionnaire (Exhibit 1, page 99). The translators were to keep a careful record of time spent in translating, editing, postediting, and (for some) typing. Those responding were: (a) three translators employed by commercial translation agencies (Numbers 2, 14, and 23); (b) eleven translators who held contracts with the U.S. Joint Publications Research Service (Numbers 1, 3, 6, 7, 11, 13, 15, 16, 17, 18, and 22); (c) six full-time translators employed, in-house, by an agency of the U.S. Government (Numbers 4, 9, 10, 12, 19, and 21); and (d) three members of the faculty of the Russian department at the Defense Language Institute (Numbers 5, 8, and 20). These three are language instructors and not primarily translators.
EASE OF POSTEDITING Eight translators found postediting to be more difficult than ordinary translation. Six found it to be about the same, and eight found 91
it easier. (One translator indicated that he found the degree of difficulty to lie between "easier" and "the same.") Thus, from the answers received, it can be seen that the translators were almost evenly divided in their opinions on the difficulty of postediting. The point of interest is that the more adept (rapid) translators found postediting more difficult than did the slower translators (see Exhibit 2, page 100). The apparent paradox that those people who thought postediting was more difficult were more proficient at it than those who found it to be "the same" or "easier" is explained by the fact that those who found it more difficult are the same people who are the most adept at translation. From Exhibit 2 one may see that six of the eight translators who found postediting to be more difficult than translating were among the faster half, and that six of the eight translators who found postediting to be easier than translating were in the slower half. The average translation speeds of translators were as follows: those who found postediting more difficult, 11.9 wpm; those who found postediting easier, 6.5 wpm; and those who found postediting about the same, 7.9 wpm. The average postediting speeds of translators were as follows: those who found postediting more difficult, 9.4 wpm; those who found postediting easier, 8.6 wpm; and those who found postediting about the same, 8.0 wpm.
RELIANCE ON THE ORIGINAL Only one translator (number 2) indicated that he seldom had to refer to the original (8a) in order to postedit machine translation. Eight translators indicated that it was almost necessary to translate the original (8b), and 14 translators answered that the degree of reliance fell between answers (8a) and (8b). It is of interest to note that most of those who said they had to translate the original were the fastest translators (and perhaps the best at translation).
POSTEDITING AND TRANSLATION SPEED Translation Speed The fastest translation speed was 19.5 wpm by translator number 1 and the slowest was 4.2 wpm by translator number 23. The difference between the translation rates of the fastest and slowest was 15.3 wpm; the mean speed was 8.7 wpm, the median was 7.6 wpm; the mode was 6.3 wpm (Figure 2). 92
Postediting Speed The fastest posteditor was translator number 5, with a rate of 12.7 wpm. The slowest was translator number 23, with a rate of 3.9 wpm. The difference between the postediting rates of the fastest and slowest translators was 8.8 wpm; the mean postediting speed was 8.7 wpm; the median postediting speed was 9.2 wpm; the mode was 10.2 wpm (Figure 2).
FIGURE 2. Speed (in words per minute) of translation and postediting.
OBSERVATIONS (a) The mean speed for both translation and postediting was 8.7 wpm. (b) Although the fastest translator could translate almost five times as fast as the slowest translator, the fastest translator could postedit only about three times as fast as the slowest posteditor. (c) Of the 23 respondents, ten (3, 6, 7, 11, 13, 14, 15, 16, 17, and 22) indicated that they had had previous experience at postediting machine-translation output (one translator said that he had postedited 93,000 words). Of this group, half had slower rates for postediting than for ordinary translation. Almost exactly the same ratio (number slower:number faster) held overall (11/23 slower: 12/23 faster). (d) The mean postediting speed of the experienced posteditors was 8.6 wpm. The mean postediting speed of those who did not indicate having experience of postediting was 8.8 wpm. 93
(e) 1. The four fastest posteditors had an average postediting rate of 11.8 and an average translation rate of 11.5. 2. The four slowest posteditors had an average postediting rate of 5.3 and an average translation rate of 6.1. 3. The four fastest translators had an average postediting rate of 10.4 and an average translation rate of 16.3. 4. The four slowest translators had an average postediting rate of 8.5 and an average translation rate of 5.3. Thus the difference between the faster and slower of these two groups was only 1.9 wpm for postediting but 11 wpm for translation. 5. The fastest translator's postediting rate was the median for postediting (9.2 wpm). 6. The slowest translator was also the slowest posteditor.
IMPACT OF POSTEDITING ON OUTPUT RATES Figure 3 indicates for each translator his speeds for postediting and translation. It is fairly obvious from a glance at this chart that fast translators will lose productivity if given postediting to do, whereas slow translators will gain. If translators are given postediting to do, then, contrasted with their translation rates: Translators 1-4 will show an aggregate loss of 23.6 wpm or 34 percent in output. Translators 5-8 will show an aggregate gain of 1.7 wpm or 5 percent in output. Translators 9-12 will show an aggregate gain of 2.1 wpm or 3 percent in output. Translators 13-15 will show an aggregate gain of 0.6 wpm or 3 percent in output. Translators 16-19 will show an aggregate gain of 6.3 wpm or 20 percent in output. Translators 20-23 will show an aggregate gain of 12.6 wpm or 37 percent in output. Thus, it may be seen that postediting machine translation tends to impede the rapid translators and assist the slow translators.
94
FIGURE 3. Percentage gain or loss in output from postediting.
TIME SPENT PREPARING THE COPY Practice varied in producing typed translations. Some respondents combined various processes. Ten translators performed translation, editing, and typing as separate operations. The total amount of time these 10 spent on the various processes was as follows: Translation 1,697 min or 63 percent Editing 365 min or 13 percent Typing 645 min or 24 percent Average typing speed of translators was only 18 wpm. Not all translators produced a typed copy.
WILLINGNESS TO POSTEDIT MACHINE TRANSLATION Twenty translators answered question 9a. Of the 20 replies, eight were negative, 11 were affirmative, and one was a qualified affirmative (yes, only if straight translation is not available). Of those who would do postediting at a lower rate than that received for translation, over half (6/11) would be willing to postedit for one half or less than the rate paid for translation.
95
No. of Translators 1 1 4 1 1 1 2
Rate _____ 1/3 1/3 - 1/2 1/2 2/3 2/3 - 3/4 3/4 4/5
It is of considerable interest (especially in a society that is allegedly materialistic) to compare the willingness to postedit at reduced rates with the respondents' speeds of translation and postediting (see Exhibit 2). For example, although translator number 13 indicated that he would accept a rate of 1/3 for postediting, his postediting speed (7.0 wpm) is actually lower than his translation speed (7.3 wpm). Only one translator, number 22, would have broken even. The other 10 would be willing in effect, to do the same number of hours of work for less pay. Of those translators who indicated their willingness to postedit at reduced rates, one out of three were commercial translators, three out of six were government in-house translators. Seven out of 11 were government-contract translators (an eighth gave a qualified "yes").
TRANSLATORS' REACTIONS TO POSTEDITING Twenty respondents took the time to give their reactions to the process of postediting machine-translation output. Although their remarks make interesting reading, for the purpose of this study we will only summarize some of the opinions expressed: Most of the translators found postediting tedious and even frustrating. In particular, they complained of the contorted syntax produced by the machine. Other complaints concerned the excessive number of lexical alternatives provided and the amount of time required to make purely mechanical revisions. A number of the experienced posteditors remarked that, although the material in this study had been carefully keypunched, they had found in their previous experience that careless keypunching was a considerable detriment. Although no translator commented that he really liked to work with the machine output, a number stated that they found the output
96
served as an aid in the translation process, particularly with regard to technical terms. (The difficulty in trying to reflect accurately the opinions of the translators may be appreciated when one reads the following comment made by translator number 23): "In conclusion, the MT was an aid and made translation easier, but when all the time used is figured up, was not as fast or profitable."
TRANSLATORS' RECOMMENDATIONS Several of the respondents were moved to suggest possible improvements in the machine output: Number 21 "I believe it might do well to scan the copy to be translated and provide a translator with a vocabulary and then allow him to translate it directly." Number 15 "Syntax-wise, some time in postediting might be reduced if the editor has knowledge of the degree of dissemination to be given the end product." Number 3 "A major improvement would be a much bolder programming of word-blocks which have a single or at most dual word English equivalent." Number 9 "More space for corrections would be a welcome format modification and would, incidently, help assure accuracy if the text is to be retyped after editing."
CONCLUSIONS In view of the small sample that formed the basis for this study, any conclusions must be tentative. With this in mind, one might draw the following conclusions from this study: 1. An adept translator's skills will probably be wasted on postediting. 2. The slower the translator, the greater the likelihood that his output can be increased by having him postedit machine translation.
97
3. Machine translation is not yet of such quality as to allow postediting to be done without a copy of the original in the hands of the translator. 4. Translators are apt to be rather mediocre typists. 5. Either translators do not consider their time and effort to be overly dear, or our respondents were exaggerating the time necessary to perform postediting, since half indicated their willingness to do the same work for less pay.
98
Exhibit 1. QUESTIONNAIRE 1.
Exactly how much time (hours and minutes) was required to translate document number 2?
2.
Exactly how much time (hours and minutes) was required to edit the translation?
3.
Exactly how much time (hours and minutes) was required to type this translation?
4.
How much time was required to edit document number 3?
5.
How much time was required to edit the edited copy (if this was necessary)?
6.
How much time was required to type document number 3?
7.
How did you find the postediting process to be compared to the process of full translation from the original? Easier? More Difficult?
8.
About the Same? Check the appropriate box: a.
"It was necessary almost to translate the original in order to properly edit the machine output."
b.
"I seldom had to refer to the original."
c.
"I placed not so great reliance on the original as question number 8, but greater than indicated by question number 9."
9.a. Would you be willing to regularly postedit similar machinetranslation output if you were to be paid at a lower rate than you earn for translating from a document in the original language? Yes
No
9.b. If yes, what is the lowest rate you would accept? Circle. 4/5
2/3 3/4 1/2 1/3 1/4 1/5 of the conventional translation rate.
10. Your candid comments and your reactions to the experience of postediting the machine output are invited below.
99
Exhibit 2. Data Compiled from Questionnaires Translator Number
1
2
3
4
5
6
I. Time (minutes) required to translate
58
65
73
87
II. Time (minutes) required to postedit
83
180
75
68
MD
A
7
120
120
60
90
100
MD
S
S
MD
E
E
A
A
A
C
C
C
Yesa
No
120
8
9
125
134
75
75
III.
Postediting was found MD to be more difficult (MD) than translation, about the same (S), or easier (E)
MD
IV.
For postediting (A) it was necessary to translate, (B) seldom had to refer to the original, or (C) between (A) and (B)
C
B
No
No
Yes
19.5
17.4
15.5
13.0
9.4
9.4
9.4
9.1
8.5
9.2
11.1
10.2
11.3
12.7
8.5
7.6
10.2
10.2
ND ND
19 19
56 19
ND ND
113 ND
V. Willingness to regularly postedit MT output if paid at lower rate VI. Amount lower VII. Translation speed (wpm) VIII. Postediting speed (wpm) IX. Editing speed (wpm) X. Typing speed (wpm)
No
—
Yes
2/3
Com 25 Com 19
1/2
Com 227 Com ND
Com: Done in combination with other processes. ND: Not done. a Yes, only if straight translation is not available. b Easier, but not much. c1/2 if typed copy not required, otherwise 3/4 to 4/5. d Between easier and same.
100
—
10
11
12
135
150
150 155
75
90
140 110
MD
S
S
A
C
A
No
No
Yes
3/4
1/3
2/3-3/4c
1/3-1/2 4/5
4/5
8.5
7.6
7.6
7.3
6.7
6.4 6.3
6.3
6.3
6.3
5.9
5.9 5.4 4.2
10.2 8.5
5.4
7.0
6.4
7.6 7.3
12.2
6.1
5.9
9.6
9.6 10.9 3.9
Com 28 10 37
56 17
37 15
37 74 15 23
ND ND
13
14
15
16
17
18
19 20
170
177
180
180
180
180 190
120
100
105
60
S
Eb
E
E
C
C
C
C
Yes
Yes
Yes
Yes
1/2
21
22
190
23
210 270
125
130
80
E
S
MD
MD
C
C
A
C
C
C
A
Yes
—
No
Yes
No
Yes Yes
113 Com
74 14
74 ND
70 195 Ed
E
E
1/2
ND ND
56 32 ND 16
101
15 14
Appendix 15 Evaluation by Science Editors of Joint Publications Research Service and Foreign Technology Division Translations
Five Joint Publications Research Service (JPRS) translations and five Foreign Technology Division (FTD) translations (four postedited machine translations and one unedited rough-draft human translation) were sent to six science editors of the American Association for the Advancement of Science and to one translationagency owner. The translations were ranked according to their quality as scientific writings. The JPRS translations were, in general, ranked higher than the FTD translations. The agreement was almost unanimous that the worst translation of all was the FTD unedited rough-draft human translation. We requested that the Clearinghouse for Federal Scientific and Technical Information provide us with the six most recently acquired Russian-to-English translations from JPRS and FTD. When these arrived, we eliminated three translations—two because of length and one because we wanted to include an unedited roughdraft translation in the sample. The ten translations that formed the sample were keyed as follows: (A) (B)
(C) (D) (E) (F)
Absorption of Radio Waves by Air Behind a Shock Wave, FTD AD605883, FTD-MT-63-74, by T. V. Bazhenova and Yu.S. Lobastov 9/62 Translations on Soviet Construction and Building Materials Industry No. 65, USSR (Large-Scale Building Activity in Process Throughout the Soviet Union) JPRS: 27,267, TT: 64-51522 11/6/64 USSR Industrial Development, Soviet Chemical Industry, No. 188 JPRS: 27,271, TT: 64-51526 11/6/64 Research on Heat Exchange in Vacuum by A. N. Devoyno, FTD-MT-63-09 Edited Machine Translation, 20 Feb. 1964 Testing and Ozokerite Bacillus Culture Liquid for Toxicity by Ch.B.Bayriyev- USSR -JPRS: 27,268, TT: (64-51523 11/6/64 There is Such a Machine by Ye. Temchin, FTD-TT-64-1170/1 27 Oct. 1964 102
(G) (H) (I) (J)
Method of Detection and Identification of Remote Explosions by V. S. Voyutskiy, FTD-MT-64-407, Edited Machine Translation, 6 Oct. 1964 Prevention of Brucellosis by I. N. Ivashurova - USSR JPRS: 27, 269 TT: 64-51524 11/6/64 Investigation of Optical Oscillator on Ruby at Liquid Nitrogen Temperature by V. K. Konyukhov, L. A. Kulevskiy, and A. M. Prokhorov, FTD-MT-63-100, 21 Oct. 1963 Translations on Soviet Agriculture No. 44, JPRS: 27,272, TT: 64-51527 6 November 1964
The translations were then stripped of any identifying markers and photoreproduced. The samples were then sent to the science editors at the American Association for the Advancement of Science and to the owner of a commercial translation agency who did not read Russian but was experienced in the editing of translations. These editors were given the following instructions: What is needed is a rank-ordering of the enclosed materials with the best document being given the number "1" and the worst document number "10." The basis for judgement would be the standards which you as a scientific editor normally apply. What we are after is your rating of excellence or lack of excellence of the writing in these documents. In other words, how does the stuff read? In addition to your rank-ordering of these items (which thus shows their standing relative to each other), we would welcome your comments as to how they impress you on an absolute scale. That is, although number "1" will be the best of the total group, it still may be an example of poor scientific writing. TABLE 7. Ranking of FTD (letters in parentheses) and JPRS Translations Best Editor Number
1
1 (Commercial firm) 2 3 4 5 6 7
H C E E (G) C H
Rating 2
3
(G) (D) H H H C H E
Worst
4
5
6
7
8
9
10
C
(I)
E
(A)
B
(F)
J
B (I) (I) J (I) (I)
(A) J (D) B (A) (J)
(F) (F) (F) (F) (F) (F)
J (G) E (I) (D) C (G) (D) B (A) C B J (G) (A) H E (A) (D) (I) E (G) B J (D) (G) (D) C B (A)
103
Results of the editors' ranking are given in Table 7. In order to obtain a numerical rating of the translations, those appearing in column 1 were given a score of 100; each column was scored 10 points lower so that those in column 10 were given a rating of 10. On this basis the numerical scores of the translations are as follows: Translation H C E G D B I J A F
Score 640 580 550 530 360 310 270 270 260 80
Translating Agency JPRS JPRS JPRS FTD FTD JPRS FTD JPRS FTD FTD
If both FTD and JPRS had had equal numbers of translations on either side of the median (55), their scores would each have been 1,925 (half of the total 3,850 points possible). Actually the JPRS translations scored 2,350 points and the FTD translations scored 1,500 points. Concerning the absolute merit of these translations, some comments of editors might be informative: Number 4. "I consider this (E-JPRS) a paper of average merit, which, from the standpoint of style and clarity, would be acceptable for publication in a technical scientific journal." Number 4. "'What is it all about?' says paper F. What indeed! This one is hopeless." Number 3. "(E and H) could be published as is or with very little rephrasing." Number 2. "As scientific writing, C is acceptable, H, J, G, and E are fair and could be fixed up with a little editing. The rest go from poor to very poor." Although the sample was too small to allow one to generalize with a great deal of confidence, the consensus of the editors concerning the relative worth (or worthlessness) of some of the translations (e.g., H and F) tends to increase one's confidence 104
in the findings of this study; i.e., the JPRS translations are somewhat better than the postedited machine translation, and the unedited rough-draft human translation is the worst of all. This conclusion, when coupled with the report from the Government Printing Office (Exhibit 1) concerning the graphic arts quality of these samples, would tend to indicate that JPRS translations are superior to FTD translations. Statistical reliability figures based on these ratings have been computed by Professor J. B. Carroll. They are as follows: Kendall's W., a coefficient of concordance, based on the JPRSFTD comparison ratings, is 0.724, well beyond the 0.001 level, but not as high as 1.00, the figure indicating perfect reliability. The application of the Mann-Whitney U-test to the summed ratings gives a value of U=4.5. For the case where 5 values are being compared with 5 values, this is significant only with a probability between 0.096 and 0.15. This is not sufficiently significant to reject with any confidence the null hypothesis that the two sets of translation are drawn from the same population. The summed ranks on which the Mann-Whitney test was based are as follows: JPRS H 13 C 19 E 22 B 46 J 50
FTD (G) 24 (D) 41 (I) 50 (A) 51 (F) 69
105
EXHIBIT 1. March 24, 1965
Dr. A. Hood Roberts, Executive Secretary National Academy of Sciences National Research Council 2101 Constitution Avenue Washington, D.C. 20418 Dear Dr. Roberts: In answer to your request for an evaluation of the quality of the printing of the translated material which you left with me, we have arrived at the following breakdown:
1. 2. 3.
F B,C,H,J G,E,D,A,I
Rating Satisfactory Fair Poor
Group 1: This is adequate perhaps only because it is double spaced and seems to be blacker than the rest of the submissions. Group 2: The printing of these is very poor, although not so bad but what the text can be read. The difficulty here seems to be that there has been no attempt to maintain good ink coverage, or good quality camera work and platemaking. The presswork is particularly bad where smudges are permitted to appear across the printing. Group 3: This group contains the illustrations. Most of them are evidently too many times removed from the original, or they were made from duplicator copies (Xerox, Ozalid, etc.) which always lose much of the detail. If the original copy had been used as camera copy, I am sure much better results could have been obtained. If the original copy was used, then the results are simply bad handling or inexperienced personnel. There seems to be little reason for reproductions as poor as this last group. Sincerely yours, JAMES L. HARRISON Public Printer By: Frank H. Mortimer Typography and Design Manager United States Government Printing Office
106
Appendix 16 Government Support of Machine-Translation Research
NATIONAL SCIENCE FOUNDATION Office of Science Information Services, Information Systems Program 1. Cambridge Language Research Unit Grant Number GN 3398 GN 4788 GN 8212 GN 8212.1
Date 3-29-57 12-31-57 4-3-59 5-6-60
NSF $
$
7,100 13,000 15,650 — 35,750
Transferred $ 20,000 (RADC) 20,000 (RADC) 20,000 (RADC) 5,500 (RADC) $ 65,500
Total $
27,100 33,000 35,650 5,500 $ 101,250
2. Georgetown University Grant Number G 2723 G 3867 G 5513
Date 6-29-56 6-6-57 6-6-58
NSF $ 35,000 35,000 36,600 $ 106,600
Transferred $ 65,000 (CIA) 90,000 (CIA) 150,000 (CIA) $305,000
Total $ 100,000 125,000 186,600 $ 411,600
3. Harvard University Grant Number GN 4982 G 5514 G 6400 G 10636 G 15924 G 24833 GN 162 GN 329
Date 1-31-58 6-6-58 9-23-58 12-11-59 12-29-60 6-30-62 6-29-63 6-25-64
NSF $ 14,150 26,200 150,000 100,000 128,500 160,160 235,450 240,500 $ 1,054,960
Transferred $ 15,000 (RADC) 70,000 (RADC) 100,000 (RADC) 21,500 (RADC)
$ 206,500
Total $ 29,150 26,200 220,000 200,000 150,000 160,160 235,450 240,500 $ 1,261,460
107
4. Massachusetts Institute of Technology Grant Number
Date
G 1210 G 2044 G 3031 G 4378 G 6537 G 10130 G 16843 G 24047 GN 244
10-28-54 10-25-55 10-23-56 9-30-57 11-3-58 10-26-59 3-3-61 6-6-62 1-22-64
NSF $ 18,700 24,800 35,200 41,400 90,600 126,000 150,000 225,000 200,000 $ 911,700
Total $ 18,700 24,800 35,200 41,400 90,600 126,000 150,000 225,000 200,000 $ 911,700
5. University of California, Berkeley Grant Number G 6399 G 8737 G 14147 GN 92 GN 306
Date 9-30-58 6-12-59 8-15-60 2-1-63 6-8-64
NSF $ 40,500 57,600 208,000 249,000 167,300 $ 722,400
Total $ 40,500 57,600 208,000 249,000 167,300 $722,400
NSF $ 14,700 40,000 100,000 $154,700
Total $ 14,700 40,000 100,000 $ 154,700
NSF $200,000 244,000 $444,000
Total $200,000 244,000 $444,000
6. Ohio State University Grant Number G 18609 G 25055 GN 174
Date 6-16-61 6-30-62 6-24-63
7. Wayne State University Grant Number GN 159 GN 430
Date 6-15-63 6-11-65
8. Ramo-Wooldridge Contract Number
Date
NSF
Total
C
10-2-61
$119,477
$119,477
152,084 50,223
152,084 50,223
233
Thompson Ramo-Wooldridge C C
233 (Amend) 320
108
3-1-63 8-20-63
Bunker-Ramo Corp. C 372
6-30-64
$240,000
$240,000 $561,784
9. University of Texas Grant Number G 19277 GN 54 GN 208 GN 308
Date 8-18-61 9-27-62 10-24-63 6-18-64
NSF $ 95,000 200,000 150,000 168,200 $613,200
Total $ 95,000 200,000 150,000 168,200 $613,200
10. University of Pennsylvania Grant Number G 3027 G 3397 G 4981 G 6538 G 8217 G 17446 G 24340 GN 311
Date 10-16-56 2-1-57 2-15-58 10-24-58 6-15-59 4-28-61 6-5-62 6-11-64
NSF $
1,950 24,300 42,300 31,450 321,800 180,400 346,000 414,000 $1,362,200
Total $ 1,950 24,300 42,300 31,450 321,800 180,400 346,000 414,000 $1,362,200
11. National Bureau of Standards Grant Number G 17815 G 19659 GN 107 GN 320
Date 6-7-61 10-3-61 3-26-63 6-29-64
NSF
Total
$ 15,000 73,000 75,000 58,200 $ 221,200
$ 15,000 73,000 75,000 58,200 $221,200
12. University of Chicago (Yngve) Grant Number
Date
NSF
Total
GN
5-22-65
$294,000
$294,000
412
13. National Academy of Sciences, Automatic Language Processing Advisory Committee Contract Number
Date
NSF
Transferred
Total
C
4- 20-64
$19,000
$20,000 (CIA) 20.000 (RADC) $40,000
$59,000 __________ $59,000
310 T. O. 80
$19,000
109
14. Linguistic Society of America, MIT (Conference) Grant Number G 11302
Date 2-8-60
NSF $15,000
Total $15,000
15. Wayne State University (Conference) Grant Number G 12887 G 15859 G 22890
Date 5-12-60 12-16-60 3-27-62
NSF $3,938 3,328 357 $7,623
Transferred $1,000 (ONR) 5,000 (RADC) $6,000
Total $ 4,938 3,328 5.357 $13,623
16. Massachusetts Institute of Technology (Conference) Grant Number G 2337 G 2888
Date 5-1956 10-1956
NSF
Total
$1,059 5,351 $6,410
$1,059 5,351 $6,410
17. University of Washington Grant Number G 13579.1 G 13579
Date FY-62 FY-60
NSF $ 1,000 53,700 $54,700
Total $ 1,000 53,700 $54,700
TOTAL NSF SUPPORT:
$6,585,227
TOTAL TRANSFERRED FUNDS:
$623,000
CENTRAL INTELLIGENCE AGENCY Georgetown University Grant Number
Date
NSF G 5513 6-6-58 Supplement XG 2230 7-1-59 XG 2239 7-16-59 XG 2312 7-1-60 XG 2427 9-1-61 Supplement to 3-31-63
Total $
9,890
24,979 153,000 439,000 438,000 250,000 $1,314,869
Note: Other CIA funds in support of the Georgetown machine-translation project (amounting to $205,000) were transferred to NSF. See above.
110
DEPARTMENTOF DEFENSE 1. United States Air Force Fiscal Year 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965
$ 400,000 700,000 800,000 1,500,000 1,400,000 927,000 561,000 600,000 2,045,000 680,000
Total
$9,613,000
2. United States Navy Fiscal Year 1953-1960 1961 1962 1963 1964 1965
$ 416,600 50,000 75,000 130,000 150,000 150,000
Total
$ 971,600
3. United States Army Fiscal Year 1958-1959 1960 1961 1962 1963 1964 1965 Total
$
184,000 223,000 225,000 110,000 175,000 230,000 175,000 $1,322,000
TOTAL DEPARTMENT OF DEFENSE SUPPORT: DOD CIA NSF GRAND TOTAL
$11,906,600
$11,906,600 1,314,869 6,585,227 $19,806, 696
111
The Committee feels that these data form the best estimate now available of government expenditures in support of machinetranslation research. Other estimates could be obtained, however, depending on the extent to which one would include or exclude funds for the support of work in related areas of data processing and information technology and the costs of the operation of the Foreign Technology Division mechanical translation facility. Criteria for what constituted support of mechanical translation research were determined by the individual sponsors.
112
Appendix 17 Computerized Publishing
In the past 3 years, since the first, and unsuccessful, attempt to use computerized typesetting in newspaper production, the advances in this technology have been such that about 200 computers are now in use in or on order by the printing business throughout the world. Nearly all the major U.S. computer manufacturers have entered this field, and competition for the market is keen. Although newspapers have been the primary practitioners of computerized printing, book manufacturers and government agencies have also begun computerized operations. In its newspaper application, a typical system would consist of the following operations: 1. The reporter types his copy in the customary way except that in certain systems the output consists of a punched paper tape in addition to the usual hard copy. 2. The editor indicates on the hard copy what changes he desires to be made. 3. If the reporter's output was a punched tape, only the necessary corrections are punched up. If only the hard copy exists, it is punched up incorporating the editor's corrections. 4. The edited punched paper tape is fed into the computer, where words are hyphenated and lines are justified automatically. 5. The punched tape (sometimes magnetic tape) output from the computer is then used to operate linecasting or photocomposition machines. 6. Subsequent operations are essentially no different from those in the conventional printing process.
LINE JUSTIFICATION The computer is well adapted for the type of computation needed for the justification of printed lines. By simply adding the width of the characters and spaces in each line and comparing the sum with the column width, the computer is able to apply the 113
proper spacing techniques (e.g., insertion of thin spaces, ens, ems, or hyphenation) for justification.
WORD DIVISION Word division still poses a problem in that the two most widely used methods ("logic" and "dictionary look-up") each have certain disadvantages. The logical method, owing to the completely arbitrary nature of English syllabification rules, cannot attain 100 percent accuracy. The dictionary look-up method requires a much larger computer memory than the logical method. Since it is unlikely that the disadvantages of either method can be completely overcome, an entirely different approach has gained the favor of some. This system, to be in operation next year at the CIA's Printing Services Division, justifies without word division hyphenation by using a photocomposer to vary the set size of the type. Exhibit 1 shows an 80 percent reduction of the standard Government Printing Office format, which in its original form is 20 picas wide and set in 10 point Modern at 10 1/2 set. It contains 15 hyphens. Exhibit 2 is the same job reset using a choice of set sizes. No word division hyphenation has been necessary. Exhibit 3 is the same as Exhibit 2 with bullets next to the lines where alternate set sizes were used.
ADVANTAGES OF COMPUTERIZED PRINTING Some of the advantages that have been mentioned by the users of this method of printing are: 1. improved output by typists resulting from elimination of the spacing and hyphenation decisions, 2. reduction of time needed to train new perforator operators, 3. more efficient use of linecasting machines, 4. the ability to set closer deadlines, and 5. increase in production.
PHOTOCOMPOSITION In the future, photocomposing machines will have to be used in order to take full advantage of the computer. The fastest linocasting machines are capable of an output of only 15 newspaper lines a minute, whereas the newest photocomposing machines are capable of printing 1,000-2,000 lines a minute.
114
Appendix 18 Relation Between Programming Languages and Linguistics
EFFECT OF LINGUISTICS ON PROGRAMMING This effect varies from period to period of programming history (which is very short). In pre-Fortran times the effect was almost nil since all programming was in machine language and almost all computation was scientific. In the period from Fortran to ALGOL (1956-1960) the connection was almost totally terminological: words and definitions, but not theory and technique, were borrowed from linguistics, for example, grammar and syntax. The real link was between programming and mathematical logic, as witness the development of ADES language1 based on recursive functions and the development of several Polish prefix-oriented languages. Syntax analysis during this period was a collection of ad hoc techniques. Thus the paper by Sheridan on Fortran2 is enormously complex. Descriptions of even more complex grammars are much more clearly understandable today. The period from ALGOL to the present shows intense borrowing of current mathematical linguistic theory, technique, and notation. The source of this dependency can be traced to the definition of ALGOL 60 syntax production notation. The similarity between this notation and the rewriting rules of some linguistic models caused this theory to be rapidly employed in programming. Still, it is important to note that the definition of the ALGOL language was totally inspired by programming considerations (Fortran, LISP), and not linguistic ones. The effect of this syntax formalism has been enormous and all to the good. Thus ALGOL syntax is "essentially" of Type 2. Hence, parsing mechanisms for Type 2 languages can be applied in the construction of ALGOL translators. Many of the parsing techniques employed were, however, discovered by programmers operating in parallel to, but independent of, similar developments in mathematical linguistics. 118
The existence of a theory has made it possible to define variations on a given grammar that permit the same task specification but in a grammar more efficiently parsed (one push-down stack instead of many, no retracing of paths in a tree of syntax choices), for example, precedence grammars. Certainly it is now the case that the design of programming languages follows a more rational procedure than before because of mathematical linguistics, and proceeds in the following steps: A. A set of tasks is isolated and their informal algorithmic descriptions are specified. B. The data structures inherent in this class of problems are isolated and appropriate computer representations are defined. C. The natural operators on the data are isolated. D. A grammar of increasingly complex units is specified, e.g., atoms, expressions, statements, and programs. E. A parser-recognizer is constructed for the grammar. F. The steps D and E are iterated until a reasonable mixture of flexibility and efficiency is attained. G. A semiformal statement of the evaluation of algorithms described in this language is given, which becomes the basis for a translation process taking this language into some other given language (usually machine code). It is now possible to teach syntax analysis of programming languages, i.e., the basic knowledge is now available in an organized form. It is now possible to construct programs that are generalpurpose syntax analysers in the sense that they parse any programming language of a given type.
EFFECT OF PROGRAMMING ON LINGUISTICS Since programming is an "applied" activity and linguistics a more abstract one, programming has provided linguistics with "real" models that are sufficiently complicated to permit the development of diverse theories. Programming has also led to the definition of linguistic models possessing a theory of their own3 and specifically tailored for use as programming languages.4 The existence of a body of technique in programming has made it possible to develop special programming languages for solving certain linguistic problems, e.g., SNOBOL5 and COMIT.6 Similarly, programming, being concerned with a growing set of demands, provides a pressure on linguistic theory directing it toward problems particularly relevant to computation, e.g., problems of efficiency of representation and speed of computation. 119
FUTURE RELATIONSHIP BETWEEN PROGRAMMING AND LINGUISTICS In programming there will be concentration on developing theories of evaluation, i.e., what is meant by the execution of a program written in language L ? We may call this the semantics of L. Such studies will replace the present ad hoc development of compilercompilers with a theory of their properties and more insight into the design of computing machines. This is the translation problem for computer languages. These languages will become sufficiently complex so that a theory of their semantics or evaluation will be a sufficiently interesting model for the equivalent problems arising in natural language translation. Similarly, there will be a reverse flow from the development of semantic theories within natural linguistics into mathematical linguistic models, which, in turn, will influence programming.
REFERENCES 1. 2. 3. 4. 5.
6.
E. K. Blum, "Automatic Digital Encoding System," NAVORD Rep. 4411 (1956). P. B. Sheridan, "The Arithmetic Translator-Compiler of the IBM Fortran Automatic Coding System," Commun. Assoc. Computing Mach. 2, 9 (1959). S. Ginsburg and R. H. Gordon, "Two Families of Languages Related to ALGOL," J. Assoc. Computing Mach. 9, 350 (1962). R. W. Floyd, "Syntactic Analysis and Operator Precedence," J. Assoc. Computing Mach. 10, 316 (1964). D. J. Farber, R. E. Griswold, and I. P. Polonsky, "SNOBOL, A String Manipulation Language," J. Assoc. Computing Mach. 11, 21 (1964). The Research Laboratory of Electronics and the Computation Center, COMIT Programmers Reference Manual, 2nd ed., The M.I.T. Press, Cambridge, Mass. (1962).
BIBLIOGRAPHY 1.
R. W. Floyd, IEEE Trans. Electron. Computers 13, 346 (1964). This bibliography includes subjects related to the syntax of programming languages insofar as they illuminate the problems of analysis and synthesis of formally defined programming languages.
120
Appendix 19 Machine Translation and Linguistics The advent of computational linguistics promises to work a revolution in the study of natural languages. Hockett is fond of the appellation "computer revolution" or "third human revolution" for the events that are engulfing us [see C. F. Hockett and R. Ascher, "The Human Revolution," Current Anthropol. 5, 135 (1964)]. There was speech, making the aggregate of codwelling animals a conglomerate tribe. There was the tool, the lever with which mankind moved the world. And now there is the computer, the first powerful manipulator of symbols outside the human head. Whether the computer is as great an invention as the first artefact, or only the first intellectual tool, its potential for linguistics is already profound. It can change the level of analysis of natural languages, as the microscope changed biology. It facilitates mathematization as it has aided physics. And it has linked theory, empirical studies, and, perhaps, practical application. Mel'chuk says that computational linguistics is not a field of linguistics, a subspecialty for those who like computation; it is a technique inescapable for any linguist who honors his discipline. In O. S. Akhmanova, I. A. Mel'chuk, R. M. Frumkina, and E. V. Paducheva, Exact Methods in Linguistic Research, University of California Press, Berkeley (1963), p. 46 we read, "MT is simultaneously both a workshop, where the methods of precise linguistic research are perfected independently of the concrete sphere of application of these methods, and an experimental field, where the results are verified by experience." Much of the recent change in linguistics has come from clarification gained through formalizing disciplines, and these changes are surely connected with the developments underlying computer studies, as well as with trends in the growth of contemporary logic and philosophy. Though it seems clear that the computer was not at the center of most of this in a direct causal fashion, it has surely played a significant role, both of interplay and as a tool for validation. 121
Surely the most dramatic recent changes have been caused by Chomsky [see, for example, Proc. 9th Internatl. Cong. of Linguistics, Cambridge, Mass., 1962, Mouton and Company, The Hague Netherlands (1964)] and similar thinkers, and they have explicitly had little to do directly with computers (see page 922 of the abovementioned Proceedings). The fundamental changes that they have brought to linguistics inhere rather in an altered view taken by linguistics of the nature of science, of a scientific theory, and of the relation of empiricism to science. But these changes have been brought about and spurred on not by scholars who live and work in vacuo, but with a good deal of cross-fertilization from areas in close touch with computational activities, and even with machine translation. Moreover, the depth of syntactic analysis has changed. A decade ago, most linguists believed that syntax had to do with word order, inflection, function words (e.g., prepositions and conjunctions), and intonation or punctuation. They also believed that most sentences uttered by native speakers in ordinary contexts were syntactically, even if not semantically, unambiguous. The important difference in their belief of that time was that they thought syntax related only to the surface structure, the visible or audible configurations of the output, and they denied by and large that process-type statements relating to rules that worked on underlying abstract expressions were properly a part of grammar. There can be no doubt that experiments in computer parsing of ordinary sentences, using reasonable grammars as hitherto conceived and programs that expose all ambiguities, have greatly helped many linguists to abandon their earlier inadequate syntactic views. A recent and accessible account of these ambiguities is that of R. A. Langevin and M. F. Owens ["Computer Analysis of the Nuclear Test Ban Treaty," Science 146, 1186 (1964)]. They use the Kuno-Oettinger parser. While it is true that a very new view of syntax has grown up, the interesting result has been that within the last 3 years or so, interest among generative grammarians has been perhaps as lively on questions of phonology as it has come to be on syntax. In fact, this is a natural consequence if one views a grammar as a total set of ordered rules, with components (e.g., phrasestructure and transformational) simply differentiated by type of rule, rather than a set of levels differentiated by the phenomena to which they severally apply, and from which one can then make a choice for the application of one's analytic efforts based on taste. Mathematical linguistics would have had no significance in 1686, if Newton had invented it. The slide rule was the perfect mathematical machine for mechanics and many other branches of 122
physics; with pencil and paper and a slide rule, general theories could be solved abstractly for special cases, and specific examples worked out for observed or proposed parameters. Of course, other branches of physics could not progress far without massive digital calculations: the study of nuclear reactions, for example, or of crystal structure. All of linguistics falls in the latter category. When a mathematical structure is promulgated as a linguistic model, its specific correspondence with any one natural language can be tested, in a serious way, only by the examination of many strings that it generates as sentences [several transformationalists have tried this technique, but the only publications known to use are by V. H. Yngve and his students; e.g., his "Random Generation of English Sentences," in 1961 International Conference on Machine Translation of Languages and Applied Language Analysis, H.M. Stationery Office, London (1962), pp. 65-82], or, conversely, by the study of the structures that it assigns to naturally occurring sentences. This plan has been tried many times. The situation is reviewed by D. G. Bobrow, in his paper "Syntactic Analysis of English by Computer—A Survey," in AFIPS Conference Proceedings, Spartan Books, Baltimore, Md. (1963), Vol. 24. Only a high-speed automatic computer (i.e., symbol manipulator) can serve adequately in empirical tests of such theories. Even today there are linguistic theoreticians who take no interest in empirical studies or in computation. There are also empirical linguists who are not excited by the theoretical advances of the decade—or by computers. But more linguists than ever before are attempting to bring subtler theories into confrontation with richer bodies of data, and virtually all of them, in every country, are eager for computational support. If ever a machine-aided simulation of total linguistic analysissynthesis (or voice-to-ear-to-voice translation) becomes possible, it will not be because of adherence to the type of linguistic theory widely current around 1950. There can be no doubt that the disappointingly slender computer results realized on the basis of such theory must have been important in shaking at least some inquisitive linguists out of their contentment. If machine translation had various negative results, this was one that was potent in a singularly fruitful way.
123
Appendix 20 Persons Who Appeared Before the Committee
June 2-3, 1964 Edmund Glenn, Department of State Jules Mersel, Bunker-Ramo Corporation September 30 - October 1, 1964 Franklin Clark, President, Language Service Bureau, Inc. Theodore Schaeffer, Free-lance translator Kurt Gingold, President, American Translators Association Howard Steensen, Translation Director, F. W. Dodge Company Thomas Miller, Director, Joint Publications Research Service Charles Zalar, National Science Foundation December 9-10, 1964 Vincent Giuliano, Arthur D. Little, Inc. Stephen Pollock, Arthur D. Little, Inc. Ernest R. Sohns, National Science Foundation March 17-18, 1965 Paul L. Garvin, Bunker-Ramo Corporation Gilbert King, The Itek Corporation J. C. R. Licklider, The IBM Corporation David Lieberman, The IBM Corporation Warren Strohm, The IBM Corporation Winfred P. Lehmann, The University of Texas
124
