|<<< Back to Hazard Communication||Printing Instructions|
Hazard Communication: A Review of the Science Underpinning
the Art of Communication for Health and Safety
Submitted to ToxaChemica, International
in a subcontract to
the Occupational Safety and Health Administration
Contract number: J-9-F-5-0050
Task Order No. 2, Option Year 1 Prepared by Barbara Sattler, R.N., Dr. P.H.,
Bruce Lippy, CIH, CSP, and Tyrone G. Jordan, B.S.
The Environmental Health Education Center
The University of Maryland Medical School
May 23, 1997
TABLE OF CONTENTS
This report was commissioned by the United States Occupational Safety and Health Administration to review the state of scientific inquiry supporting our knowledge regarding key elements of chemical hazard communication programs: labeling, warnings, material safety data sheets, and worker training. This endeavor supports the international effort to harmonize laws, regulations, and consensus standards affecting the ways in which information about hazardous chemicals is communicated. The international effort can be divided into three major functions: classifying health and environmental hazards, classifying physical hazards; and communicating hazard information. This last component involves the determination of what information will be communicated to users regarding the hazards and appropriate protective measures, as well as the way in which it will be transmitted, i.e. through symbols, labels, standard phrases, and training.
Internationally, six organizations the World Heath Organization, the International Labour Organization, the United Nations Environmental Program, the FAO, UNIDO, and OECD participate in the Inter-Organization Programme for the Sound Management of Chemicals (IOMC). The IOMC has compiled a bibliography of references and has administered a survey sent to member countries asking for information about their respective requirements on hazard communication. Additionally, the countries responding to the survey were asked to submit any studies and supporting documentation for their country's policies, laws and regulations. The resulting draft report was produced by the Working Group on Harmonization of Chemical Classification Systems which was established by the Coordinating Group of the IOMC. The full text of this important report is contained Appendix A and is referenced throughout this document as the IOMC draft report. The following key chart is taken directly from the report.
There is a strong consensus that a fully harmonized hazard communication system must take into account the scientific findings on comprehensibility, readability, and other human factors regarding the use of labels, warning placards, and safety data sheets. Further, this effort must be built on previous work. It is impractical and ill-advised to ignore the fact that there are existing systems, several used internationally, that should be revised to make them more effective.
An extensive literature search was conducted on hazard communication and the information is summarized within this report. Not all citations entered in the bibliography are cited within the body of the report. Several U.S. agencies have done similar reviews of literature regarding effectiveness of labeling, warning symbols, and hazardous chemical training: the Environmental Protection Agency (EPA), the Consumer Product Safety Commission (CPSC), the National Institute for Occupational Safety and Health (NIOSH), and the National Institute for Environmental Health Science (NIEHS). There is strong international interest in the comprehensibility of information. In addition to summarizing the literature, deficiencies in our knowledge have been identified and recommendations for further studies have been suggested.
2 SEARCH METHODS AND INFORMATION SOURCES
In December 1996, the ILO's Working Group on Harmonization of Chemical Hazard Communication issued a draft document entitled "Report on the responses to the call on chemical hazard communication." The calls for information were actually sent in May and July 1995 to about 270 relevant national and international institutions with a formal questionnaire going to a few organizations selected from this larger body. The requests went out through the ILO's International Occupational Safety and Health Information Centre. Background materials on hazard communication were requested, particularly scientific studies, with an emphasis on comprehensibility studies and procedures used to implement hazard communication systems at the national level as well as by industry. The information was collected as background information for creating a harmonized chemical hazard communication systems.
The work in harmonization is based on a number of international resolutions, conventions, and reports. In June 1992, the United Nations Conference on Environment and Development (UNCED) stipulated that "a globally harmonized hazard classification and compatible labeling system including material safety data sheets and easily understandable symbols, should be available, if feasible, by the year 2000." Part B of Chapter 19 of UNCED's Agenda 21 is devoted to harmonization of classification and labeling.
The Working Group on Harmonization of Hazard Communication addresses issues regarding hazard symbols, colors, and written information used on labels; the preparation of chemical safety data sheets and instructions; the comprehensibility of precautionary statements used on both labels and in chemical safety data sheets; and training related to these areas.
The Coordinating Group for the Harmonization of Chemical Classification Systems brings together experts representing existing national, regional and international systems of classification and labeling of chemicals, as well as representatives of interested international organizations of suppliers, employers, workers, consumers and environmental groups. The objectives are a finalized proposal for harmonized classification criteria by 1997 and possible implementation of a globally harmonized system at the national level by the year 2000.
The field of occupational safety and health is by nature multidisciplinary and the literature describing its varied works must be searched using several tools. Several bibliographic databases were searched back to 1980. No single database identified the necessary literature to accomplish the task at hand. The health and safety databases had little information on labeling. This information was found in a variety of databases including PsycLIT which more extensively identified the human factors and ergonomic literature. The health and safety databases were also deficient in their coverage of training efficacy; that body of literature was found in educational databases and to some degree in the NIOSH database (NIOSHTIC). The use of Internet searching tools uncovered additional information from a variety of sources including international agencies, national agencies, universities, and private organizations. The literature cited and reviewed was limited substantially to peer reviewed documents, such as peer reviewed journals and government reports. Because the literature on the effectiveness of labeling, MSDS, and training specifically in the context of workplace hazard communication was surprisingly sparse, the search was extended to look at other applications of hazard communication, such as the use of labels for hazard warnings on consumer products and in other public safety situations such as transportation safety. In this extended search, a substantial literature was identified regarding the effectiveness of labeling and warning signs.
Additionally, agency staff were interviewed in several of the U.S. agencies responsible for product and chemical labeling, including the Environmental Protection Agency, the Consumer Product Safety Commission, and the Department of Transportation. Staff at the National Fire Protection Association were also contacted. Chairmen and key committee members were contacted from the two main American National Standards Institute (ANSI) standards committees on labeling and MSDS. Members of the Society for Chemical Hazard Communication were also contacted as a means of identifying and locating materials and studies that may have been done within companies or trade associations and therefore not retrievable through bibliographic searches. Some members of the HazCom Workgroup within the National Advisory Committee on Occupational Safety and Health were also contacted for the same reason.
OSHA provided a bibliographic search that had been done by the International Occupational Safety and Health Information Centre at the ILO. Approximately half of the documents listed were written in languages other than English. The majority of the documents listed in this bibliography are governmental reports, guidance, and information documents. Few of the documents are articles or books describing research on the effectiveness of hazard communication.
A substantial bibliographic search had been done as the first phase of a current, U.S., multi-agency project entitled the "Consumer Labeling Initiative." This project has the stated goal of making the health and safety information on labels easier for consumers to understand. This bibliography provided a rich source of literature including agency reports and research on labeling. The second phase of this project will continue in 1997. The Consumer Products Safety Commission received a report in May, 1995 entitled, "Product Labeling Guide" which contained an excellent literature review. The report was prepared by Bari M. Kotwal and Neil D. Lerner of COMSIS Corporation and has been quite valuable for this present report.
The Society for Chemical Hazard Communication produced a compendium of citations entitled, "Published Research on the Comprehensibility of Chemical Hazard Warning Labels and Material Safety Data Sheets." Although this listed a number of articles on labeling, the vast majority of articles described labels in non-workplace settings, such as on consumer products and in transportation contexts. Further, only one article listed referred to Material Safety Data Sheets.
In 1995, a literature review entitled "Assessing Occupational Safety and Health Training" was published by the NIOSH that sought evidence from the literature bearing on two questions: Are OSHA training requirements effective in reducing workplace injury and illnesses? Does the available evidence show certain training factors or practices as being more important than others in having a positive effect on illness and injury prevention? Additionally, a set of papers was collected for a 1996 workshop sponsored by the National Institute for Environmental Health Science, entitled "Measuring and Evaluating the Outcomes of Training."
All of the above-mentioned searches, reviews, and listings are included in this report either as independent documents or subsumed into the comprehensive bibliography.
The following databases were searched for this report using the key words identified for each database:
* Denotes any variation on the root word.
3 REVIEW OF LITERATURE
3.1 LABELS AND WARNINGS
3.1.1 SUMMARY OF RESEARCH WITH RECOMMENDATIONS
The following information represents a summary of the general areas of consensus in the literature on labels and warnings. The summary also includes what the authors of this report consider key findings and recommendations.
One of the most comprehensive review of literature on labels and warnings was performed by COMSIS Corporation under contract to the U.S. Consumer Product Safety Commission (Kotwal and Lerner, 1995). The authors described the state of knowledge on labels and warnings with the following excellent summary:
"Although warning label design may superficially appear to be a simple matter, proper design and use is, in fact, rather complex. It is clear that inappropriately designed or used warnings simply will not be effective, by any standard of evaluation. Yet, it is difficult to find comprehensive, usable and definitive guidance for the design or evaluation of labels, in spite of the fact that there are numerous label standards (e.g., ANSI Z535 standards, ISO standard), label design guides (e.g., Westinghouse and FMC handbooks), general human factors guidance for warnings and labels, and various literature review articles... Unfortunately, the findings of the research literature on labels are often contradictory and frequently weak in terms of face validity or experimental procedures. Label effectiveness is often judged on the basis of subjective opinion or people's reports of what they think they would do if they encountered the label. However, the relative validity and ultimate safety consequences of labeling alternatives is poorly understood.3.1.3 COMPREHENSIBILITY
Comprehensibility refers to the ability of the individual reading a label, warning, or material safety data sheet to understand the information sufficiently to take the desired action. Comprehensibility is different from readability because the latter is simply a measure of the grade level of the written material while the former is a measure of how well the receiver of the information understood it. A warning about incompatible chemicals may be written at the correct reading level for a specific audience but may do such a poor job explaining the concept of incompatibility that the warning isn't understand by most of the intended audience. Additionally, the same warning may be highly comprehensible to a population of chemical workers but poorly understood by firefighters with the same educational level but different work experiences. Finally, achieving comprehensibility does not ensure that the informed individual will take the actions prescribed in the warning or label. This final, behavioral step is affected by a complex mix of attitudes, experiences, motivations, and potential consequences that are specific to each individual in a particular situation.
Two experiments involving warning labels on household products (Godfrey and Laughery, 1993b) indicated that subjects are able to discriminate among products based on overall hazard. The more hazardous they perceived a product, the more likely it was they would look for a warning. Further, females are more likely to look for warnings than males. In summary, perceived hazard, familiarity, and sex influence the consumer's decision to look for a warning message on the labels of potentially hazardous household products.
Other researchers looked at the variables related to people's perceptions of hazard for various types of household pest-control products (Silver and Wogalter, 1991a). Seventy college-aged students, 20 older subjects, and four pesticide experts judged 26 pest-control products on hazard. Results showed that fumigators and foggers were the products perceived as the most hazardous followed by sprays, systems, and traps, respectively. Although several misperceptions of hazard for certain product categories were evident, the judgments of the students and older adults were consistent with those of the experts. Perceived hazard was also found to be positively correlated with a number of objective characteristics of the product labels, including the quantity of chemical ingredients, effectiveness against pests, number of words and sentences on the label, readability, and the presence and location of certain statements on the back label. These finding suggest that people can discriminate the hazard levels of different types of pest-control products, and the presence of various cues on the label may aid this determination.
Desaulniers (1980) examined the effects of warning layout and semantic organization on the readability and recall of warning information. In general, warnings in outline layout and type of hazard organization were ranked as having greater eye appeal, easier to process, and more effective than alternative organization layout conditions (i.e., paragraph layouts). The results indicated that warnings in outline layout were read and complied with by a larger proportion of subjects than warnings in paragraph layout. Collins and Lerner (1982b) also noted other important criteria in addition to understandability in fire safety alerting measures. Variables such as visual range, detectability in smoke, and legibility all were critical.
188.8.131.52 Use of Symbols and Pictograms
There has been a great interest for years in depicting critical safety and health information as succinctly as possible for immediate response by individuals in dangerous situations. The focus has been to avoid the numerous shortcomings of written language, particularly the length of time it takes to read a warning and the proximity the reader must have with the information in order to see it. Additionally, the increasing number of American workers who do not speak English as a first language or who are illiterate argues well for the use of symbols and pictograms. This is also essential for any effort towards international harmonization.
Symbols and pictograms are widely used in warning labels. However, this trend, at least in the United States, is relatively recent. Research on the use and effectiveness of pictorial signage began in the early 1970's and continued into the early 1980's. At the same time, the use of symbols burgeoned for applications such as the highway, public facilities, workplace signs, and product labels. The sizable body of research on symbol effectiveness from this decade of activity primarily focused on signs, for highways, workplaces, and public facilities, rather than labels. Collins' 1982 review of literature through the early 1980's reported that "there has been little if any research on the effectiveness of symbols for product warnings." According to Kotwal and Lerner (1995) although there has been subsequent research specifically on labels, most research still has been based on signs; much of current practice is an extrapolation from sign-related research or is based on logical argument.
One study showed college students 16 symbols from the 1981 Westinghouse Product Safety Label Handbook. For each graphic, the subject was asked to describe the hazard. Recognition rates ranged from nearly 100% to almost completely unrecognizable (Laux, Mayer and Thompson, 1989). Results indicated that there are two questions which must be addressed in the development of symbols and pictorials: 1) will the user recognize the hazard? and 2) will the recognition of the hazard suggest proper precautionary behavior?
The U.S. EPA (1986) received universal agreement from their advisory committee that educational efforts are essential if symbols or colors are to communicate effectively. They cited the experience of the Canadian government which instituted a system of symbols to convey degree of hazard in the mid-seventies. They since undertook an educational effort which resulted in 95% of Canadians understanding the meaning of the symbols. A survey conducted in the U.S. where no similar educational campaign was held revealed comprehension below 25%.
Because of their relatively universal information transmission potential, pictorials have been suggested as common means of safety communication across heterogeneous groups of users and uses (Brelsford, Wogalter, and Scoggins, 1994). The researchers conducted a study and found that easy pictorials were comprehended (both initially and following training) better than difficult pictorials, although the latter showed the most dramatic increase in understandability after training. In addition, they found that the substantial gains measured after training suggest that even brief training - as little as giving the pictorials' verbal meaning once - can have a large impact in facilitating comprehension for pictorials that would otherwise not be understood by many people.
Boersema and Zwaga (1989) identified five warning messages that were used to reduce the hazardous behavior of swimming pool slide users. For each warning, nine symbols were developed. The comprehensibility of the symbols was tested using an evaluation procedure based to a large extent on the International Standards Organization testing procedure to determine the comprehensibility of public information symbols (ISO 9186, Procedures for the Development and Testing of Public Information Symbols). Respondents were swimming pool users between 7 and 19 years of age. Seven acceptable symbols were found referring to four of the five warnings.
Collins and Lerner (1982b) evaluated U.S. participants' understandability of twenty-five internationally proposed symbols for fire-safety alerting. The poor performance of some critical symbols such as "exit" was noted, and some potentially dangerous confusions in meaning were revealed. As a result, the researchers advise incorporating testing procedures as integral parts of the symbol development and standardization process before widespread adoption.
Frantz, Miller, and Lehto (1994) examined the impact of the flame and poison warning symbols prescribed by the Canadian government. The results suggest that although the generic meanings of these two symbols are well understood, people have difficulty inferring the specific safety precautions most necessary for a particular product. In the test case of extremely flammable floor adhesive, users apparently did not realize that adhesive vapors, rather than the adhesive itself, pose the fire hazard.
The authors of the FMC Corporation Product Safety Sign and Label System manual believe that communication effectiveness between a greater cross-section of the population, both nationally and internationally, could be increased by using pictorial or symbolic language in place of, or as a supplement to, written words. The pictorials are combined with words and colors in specifically designed formats intended to present comprehensive hazard information in an orderly and understandable manner.
Jayne and Boles (1993) investigated whether different warning designs, specifically those with symbols, affect compliance rates. Five conditions were tested: a verbal warning, a pictographs warning with a circle enclosing each graphic, a pictographs warning with a triangle on its vertex enclosing each graphic, a warning with both words and pictographs, and a control (no warning). Participants performed a chemistry laboratory task using a set of instructions that contained one of the five conditions. The warnings instructed them to wear safety goggles, mask and gloves. All four warning conditions had significantly greater compliance than the no-warning condition. A significant main effect was found for the "presence of pictographs" variable, suggesting to the authors that the addition of pictographs to a verbal warning will increase compliance rates. The enclosure shape made no difference in compliance rates, despite research that indicates that unstable shapes are preferred as warning enclosures. The authors echoed the recommendations of Horst that conducting behavioral studies was preferable to relying on preference data.
184.108.40.206 Shape of labels and warnings
The proper shape of warning signs has been evaluated. The triangle arranged on its vertex was the preferred shape, with the circle despite its common use scoring among the least preferred warning shapes (Riley, Cochran, and Ballard, 1982a). Similarly, the Consumer Products Safety Commission (Federal Register, May 3, 1996) tested several pictograms for effectiveness in warning about the hazards of carbon monoxide poisoning from burning charcoal briquettes in confined spaces. They followed the ANSI Z535.3-1991 protocol for testing and found that the standard slash recommended by ANSI was not as effective as an "X" for warning consumers of prohibited uses.
Work by Young (1993) focused on increasing the noticeability of warnings by manipulating four variables: pictorial, color, signal icon and border. Subjects viewed 96 simulated alcohol labels on a computer, half with a warning and half without. The results showed that warnings containing a pictorial, color, or an icon had significantly faster response times than warnings without them. However, the addition of a border did not improve response times. More detailed analyses demonstrated that pictorials, color and icons can enhance the noticeability of warning information and interact with each other in such a way to that they should not be used indiscriminately without adequate knowledge of the interactions.
Warning labels are most often rectangular in shape. The orientation of the rectangular shape is determined by the format of the label and the number of elements contained within the label (i.e., signal word, pictograms, message text). The shape of any symbols contained within the label, however, are drawn from the pool of shapes commonly associated with safety, such as octagons, diamonds, triangles, and circles. While the shapes of the elements within labels are somewhat standardized and commonly chosen for prompt user recognition, the effects of the shape of the label itself have not been scrutinized as closely Kotwal and Lerner, 1995). Rectangular shapes are predominantly used because of their simplicity.
Besides allowing for efficient use of space, the shape of a label may enhance the conspicuousness of a warning, even if the targeted user is not actively searching for safety information. This may be because the public may become more sensitive to certain shape stimuli which they have come to associate with danger much as they do with color. As a result of this perceptual learning phenomenon, Cunitz (1992) suggested that standardized shapes (e.g., octagon shaped stop sign, triangle shaped yield sign) can take on greater attention-getting capabilities than other shapes.
ISO 9186, Procedures for the Development and Testing of Public Information Symbols specifically recommends the use of surround shapes. The defined shapes are used to distinguish different alerting categories. However, these shapes only surround the pictogram portion of the warning. If a verbal message is included in the warning, it is not included in the surround shape. ANSI Z535.3 (1991) does state that the perimeter of the label panels may take on the surround shape of safety symbols, although justifications for this allowance are not specified. Safety shapes recommended by ANSI Z535.3 (1991) and ISO 9186, along with their associated meanings are as follows:
220.127.116.11 Use of Multiple Media
There have been several important studies that point to the value of presenting messages in several media simultaneously. For instance, a symbol may be combined with written information or a signal word to reinforce the effect. In one study, fifty-eight subjects were shown randomly-ordered facsimiles of standard danger signs required by OSHA and rated the signs on 13 dimensions related to perceived effectiveness. Signs containing a hazard label and instructions (e.g., GASOLINE - NO SMOKING) were rated as least likely to be recalled at a later time; however, they were rated as easiest to understand, most informative, and most likely to be complied with. Signs containing a hazard label only (e.g., POISON) were rated as least informative and most difficult to understand; however, they were rated as most likely to be recalled, as depicting a high degree of danger, and likely to be complied with. Signs containing instructions only (DO NOT ENTER) were rated as generally least effective.
Dingus, Hunn and Wreggit (1993) found that the inclusion of protective devices in the packaging of hazardous consumer products (e.g. gloves in hair coloring products) decreased the cost of complying with warnings specifying their use and led, consequently, to more compliance. See the discussion in Section 18.104.22.168, Cost of Compliance.
Legibility involves both the typographical elements and the surface upon which the message is printed (Hale, 1991). This was abundantly illustrated when Congress required a health warning on tobacco packages but provided little in the way of specifications. The labels produced could scarcely be read in many instances, due to type size and the contrast between the ink and paper chosen. In 1988, Congress passed a similar requirement for beer and wine but this time included specific requirements, including that the first two words must be "GOVERNMENT WARNING," set in boldface capitals.
A study commissioned by the Public Health Service (1990) looked at the effectiveness of these proscribed warnings and found that when the color of the typography and the color of the background were measured, a Contrast Ratio (CR) could be derived, which proved to be a valid measure of legibility. Specimens with high CR values were less legible, despite containing the same warning in the same size type. Hale (1991) concluded that even with reasonable attempts to mandate legibility there have been a number of factors not specified that can have significant effects. "If regulators and others who wish to formulate rules for legibility are to enjoy even modest success, it is clear that they will need the assistance of well-defined standards covering all the attributes described (Hale, 1991)."
Howett (1983) derived a formula giving the letter stroke-width needed for legibility of words on a sign at any given distance by an observer with any given visual acuity. The stroke width, in turn, determines the letter size, depending upon the characteristics of the type face used. The derivation is strictly mathematical and is based on the assumption that beyond a distance of a few meters, a person's visual acuity is specifiable by a fixed visual angle, independent of the distance. The author does provide graphs for correcting the critical stroke width for nonstandard contrast or background luminance based on a body of data on visual acuity as a function of contrast and background luminance.
22.214.171.124 Reading level evaluations
Dangerous accidents may occur when people misread written materials, consequently materials must match the reading level of the intended audience. There are several internationally recognized instruments for measuring the reading level of written materials but none of them deal with the comprehensibility of the materials, as indicated earlier. Kreindler and Luchsinger (1978) suggest determining the reading-grade level of all safety materials using standardized methods such as the FORCAST formula or the Flesch Reading Ease Index. Laughery and Brelsford (1996) reported commonly finding product warnings written at significantly higher grade level than the grade 4 to 6 range that is usually recommended for general target audiences. A discussion of reading level measures and their application to the design of instructions and warnings can be found in Duffy (1985).
126.96.36.199 Type Fonts and Type Size
Braun and Silver (1992) looked at over-the-counter and prescription drugs and examined certain variables associated with legibility, namely font type, font weight, point size contrast between the signal word and the main body of the warning. The results indicated that participants were more likely to read the warning in Helvetica type (a sans serif face) than in Times or Goudy (both serif faces). Times was more likely to be read than Goudy. Bold type was more likely to be read than Roman type. There was a greater likelihood of reading the warning when the main body was in 10-point size as compared to 8-point size. A 2-point size difference between the signal word and the main body of the warning produced a greater likelihood of reading the warning over a 4-point size difference. One possibility for this result is that the 4-point size difference minimizes the importance of the main body of the warning, therefore making only the signal word salient.
Looking at pesticide labeling, Silver, Kline and Braun (1994) obtained different results from their earlier work with prescription drugs. The variables used were the same: font type, point size contrast between the signal word and the main body of the warning (signal word-text size difference), and point size of the signal word. College students rated 36 insecticide labels that contained a warning which varied across all levels of the three variables. Results showed that Century Schoolbook was perceived as more readable than Bookman or Helvetica. Moreover, greater perceived readability was obtained when there was no difference between the point sizes of the signal word and the body of the warning. Perceived readability and perceived hazardousness decreased as the signal word-text size difference increased. There was greater perceived readability and perceived hazardousness when the signal word was printed in 14-point type than in 12- or 10- point types.
3.1.5 PLACEMENT OF INFORMATION
One of the key variables on warning effectiveness is where the information is placed. This includes placement of information within a label as well as the physical placement of a warning sign in proximity to the hazard the warning describes. In tests on the effectiveness of three different warning label designs for a portable electric heater, researchers Gill, Barbera, and Precht (1993) found that the design requiring the user to interact with the label in order to use the heater was more effective in attracting the user's attention that a traditional label or a color-coded "ski-pass" label attached near the male end of the electric cord. However, none of them were effective in mediating safe user behavior.
An experiment on the noticeability of warnings on alcoholic beverage containers indicated that warnings on the front label were found more quickly than warnings appearing in any other location. Also, warnings printed horizontally were found more quickly than warnings printed vertically. The signal phrase ("Government Warning"), as well as the amount of "noise" or clutter on the surrounding label, significantly influenced warning detection times. The authors suggested that proper manipulation of these design features could make the mandated warning more noticeable (Godfrey et al. 1993).
Godfrey and colleagues (1993c) also looked at the effect of the physical size and location of the warnings. Warnings were posted on a copy machine, a public telephone, a water fountain, and two sets of doors. Most people did not use the copy machine and the telephone when they had the warnings on them. The warning on the water fountain was not effective when a single, small warning was used. When that plus a larger, more forceful warning was used, most people did not drink from the water fountain. The warnings on the doors were not obeyed unless there was a convenient alternative exit available. Warnings with a low cost of compliance are apparently more effective. Additionally, the warnings cannot be too small and must be placed strategically.
A paradigm was developed to examine the effectiveness of warnings in a laboratory task (Wogalter, Fontenelle, and Laughery, 1993). Compliance (use of mask and gloves) was affected by the inclusion of the warning as well as by its location. Greatest compliance occurred when the warning was placed prior to the instructions. The addition of a printed statement placed before the instructions (with warning at the end) to read through the instructions before beginning produced intermediate compliance that was not significantly different from the warning beginning and end conditions. Observation revealed that when the warning message was at the end of the instructions subjects complied only when they saw the warning message before starting the task. These results indicate that if warnings are placed in front of instructions the consumer is more likely to read and comply.
Another chemistry lab format was used by Wogalter, Kalsher, and Racicot (1992) to examine the efficacy of two warning-related factors to produce cautionary behavior. Experiment 1 compared the effects of a posted-sign warning and a within instruction warning on behavioral compliance. The results showed that a warning embedded in a set of task instructions produced significantly greater compliance (the wearing of protective gear) than a similar, larger warning posted as a sign nearby. Experiment 2 reexamined the effect of location and also examined the influence of the presence versus absence of pictorials. The results of Experiment 2 confirmed the location effect of Experiment 1. No influence of pictorials was noted, although there was a nonsignificant increase in compliance when pictorials were added to the within-instruction warning. The results indicated that warning placement is important for eliciting behavioral compliance to safety messages. Explanation such as difference in field of view and perceived relevance are discussed.
FMC Corporation (1985), as part of their safety program, emphasized the appropriate placement of safety signs and labels to reduce the occurrence of accidents. Frantz (1992) found that contrary to current, recommended practice, substantially more subjects read and complied with warnings and instructions that appeared in the directions for use rather than the "precautions" section. On average, moving an instruction from the precautions into the directions increased the reading rate from 37% to 89% and increased the compliance rate from 48% to 83%.
3.1.6 AUDIENCE VARIABLES
188.8.131.52 General considerations
McCarthy and colleagues (1984) examined approximately 400 articles on the effectiveness of on-product warning labels and were able to categorize the literature based on the manner in which the warning issue was addressed:
Qualitative - editorial. These appear to be based on the expectation that providing safety information is related to an increase in safety behavior.The authors concluded for nonquantitative studies that not one of the design variables discussed in label creation "is related in any known way to the actual effectiveness of warnings in changing behavior and reducing injuries." Further, they felt that all available quantitative studies showed the warnings under investigation to have no impact on safety. Horst (1993) reviewed the literature of experimental psychology and communication theory and pointed out that people learn to filter out most of the information that bombards our senses and to attend to things that they have learned to view as important or interesting. If safety information is perceived as familiar or redundant, it may simply be dropped from short term memory, and have no further effect on behavior. Alternatively, if the information is perceived as inconsistent with existing knowledge and beliefs, it may be rejected as not credible. Even if the information is previously unknown and is accepted as true, the recipient may be insufficiently motivated to alter behavior.
Similarly, Miller and Lehto (1986) looked at 388 reference sources on the subject of warnings and found only 10 sources that they considered true analyses of warnings effectiveness, and of these, only 6 actually cite any experimental results. Purswell, Krenek, and Dorris (1993) do not find this paucity surprising because of the difficulties associated with performing these studies. There is the general problem of collecting experimental observations and the specific problem of inadvertently introducing an experimental bias because the subject is influenced to read the warning and behave safely if he or she perceives that the purpose of the study is to evaluate warning effectiveness.
Horst (1993) also reported that there are widely encountered guidelines - based primarily on preference studies that warnings should comprise four elements: a standard signal word, statements of the hazard, the potential consequences, ad how to avoid the hazard. Horst et al. claim it is erroneous to suggest that all of this information must be provided to affect behavior or that inclusion of all of this information will guarantee effectiveness. The authors gave the examples of stop signs and restroom signs neither of which carry all four elements but are effective warnings. Stop signs are effective because the negative consequences of failing to respond appropriately are universally familiar and highly probable. Similarly, the high probability of social embarrassment is clearly a motivator in the effectiveness of restroom signs.
Laughery and Brelsford (1993) reviewed the literature and reported "reasonably clear trends" that indicate women report a greater likelihood than men to look for and read warnings. They are also more likely to comply with warnings.
Kotwal and Lerner (1995) concluded from their comprehensive and excellent review of literature on warning labels that the age of the user appears to have an effect on the perceived level of hazard conveyed by common signal words. Leonard, Hill, and Karnes (1989) found significant effects between younger college students and older college students (over age 25). Kotwal and Lerner (1995) summarized their findings with the following statement:
"Older subjects generally used signal words that implied greater hazards to represent the amount of risk involved in a given situation. Since older users have indicated that a given signal word implies a lower level of hazard than the same word implies for younger users, it may be necessary to apply relatively strong signal words for older users in order to connote a given level of hazard."Desaulniers (1991) reported that older people, 40 and above, are more likely to take precautions in response to warnings as reflected in their behavioral intentions. However, Collins & Lerner, (1982) found that older subjects exhibited lower levels of comprehension for safety signs involving pictorials. For Laughery and Brelsford (1993) these studies taken together suggest that older receivers are more influenced by warnings, but greater attention to issues of comprehension may be necessary.
184.108.40.206 Hazard and risk perception
A consistent finding in warning research is that people's perception of the hazardousness associated with a product or situation can determine the effectiveness of a warning or label (Laughery and Brelsford, 1993). Many researchers have acknowledged that the actual design of the warning label may not be as important as the expectations that the user brings to the situation (Kotwal and Lerner, 1995). Research has indicated that perceived severity of consequences has a strong influence on behavioral intentions. Numerous studies have shown that the greater the a priori perception of hazardousness, the more likely people will look for and read a warning and the more likely they will comply by taking precautions (Donner and Brelsford, 1988; Friedman, 1988; Otsubo, 1988). Additionally, the more explicit the warning is about the severity of injury consequences, the greater the perceived hazardousness and recall of warning information.
Purswell, Schlegel, Kejriwal, and Sashi (1993) reported that there have been surprisingly few attempts to develop a useful measure of risk-taking behavior, given the importance of this factor in safe behavior. They developed a model to predict whether a consumer would use a product safely as a function of sixteen different individual variable. Subjects were presented with four consumer products to use in an experimental setting where the true purpose of the study was concealed. Discriminant analysis was used to develop a prediction model to classify subjects into categories of safe or unsafe behavior. Prediction accuracy ranged from 68 - 86 percent for different types of behavior.
Other researchers looked at people's willingness to read warnings on household pest-control products and their likelihood of purchasing these products (Silver and Wogalter, 1991b). Seventy college-aged and 20 older participants examined a set of pest-control products and responded to a questionnaire assessing perceptions of the products, the packaging, and the warnings. Results show that product hazardousness, warning understandability, and warning attractiveness were strongly related to people's willingness to read the warnings. A different set of variables was related to purchasing intentions. Participants reported greater willingness to purchase products that were more familiar and had more attractive packaging. Participants were more willing to read warnings that contained more statements and had readability scores at higher grade levels, a result that appeared to be due to their common relationship with perceived hazard. The findings suggested that manufacturers can place appropriate and effective warnings on pest-control products without necessarily reducing buying intentions.
Bogett and Rodriguez (1987) explored the influence of a perception of danger particularly regarding product-borne warnings and safety instructional programs. The results of their study, together with supporting data from the literature implied that a perception of danger (i.e., an unacceptable risk of loss or injury) must exist in order to elevate a person's safety behavior.
Following Pages: Table from Kotwal and Lerner (1995) Summary of Research Findings Regarding User Age, Gender, Perceived Hazardousness, and Familiarity With Product
Along with the perception of hazard, another important external factor with respect to warning effectiveness is stress. Magurno and Wogalter (1994) used a chemistry task format in which the stressor was a combination of two kinds of stress: time pressure and social evaluation by another person. There were four conditions evaluated: stress (lower versus higher) and warning location (posted sign vs. within task instructions). The results showed that higher stress produced significantly lower compliance. Warning placement was also found to produce a main effect on behavioral compliance. More participants complied with the within-instruction warning than the posted-sign warning, even though the sign was over 40 times larger in terms of area.
220.127.116.11 Product familiarity
A study of one hundred students using hammers in a laboratory setting resulted in none even noticing the warning labels on the hammers (Dorris and Purswell, 1977). The authors concluded that a warning label on a familiar product is not necessarily a salient item and may be filtered out by the user's attentional mechanisms and never processed at a conscious level.
Godfrey and Laughery (1993c) found that women were not so likely to notice warnings about toxic shock syndrome on or in tampon packages when they switched products. The authors found a real biasing effect from product familiarity.
DeJoy (1989) reviewed the literature on warnings and found seven studies where familiarity decreased the likelihood of noticing, reading, or complying with warnings. Dejoy noted, however, that these factors do not appear to be as important as the user's product-related expectations. Similarly, interview research into the effectiveness of warning signs on scaffolding showed that both inexperienced and experienced subjects would react to a new warning on scaffolding they were unfamiliar with but not to the same new warning on equipment that was familiar to them (Johnson, 1993). This study, however, was based on measuring behavioral intention through interviews, not behaviors.
Many other studies have looked at the effects of familiarity on whether or not subjects will look for, notice, and read a warning. Godfrey, et al. (1989), Godfrey & Laughery (1993c), LaRue & Cohen (1987), and Otsubo (1988) have all reported that people are less likely to look for, notice and read warnings on consumer products with which they are familiar than they are with unfamiliar products. Laughery and Brelsford (1993) considered the familiarity effect consistent and robust and suggested a possible reason: the more people use a product without experiencing a safety problem, the less hazardous they perceive the product to be. In the authors' words: "products that are used repetitively pose special warning problems."
Ayres, et al. (1994) suggested, after an extensive literature review, that warnings are unlikely to be effective unless a series of conditions are met. The failure of many intended warnings, including most on-product warning labels, to reduce accidents reflects the difficulty of overcoming problems inherent in their use.
Color affects behavior. Braun, Sansing, and Silver (1994) pointed to an extensive literature on the effects of color but note that vast majority of product warning research has not considered color beyond the application of standards and guidelines. Dunlap, Granda, and Kustas (1986) surveyed 1169 subjects across several different language groups including English, German, Scandinavian, and Spanish. Subjects rate the color words red, orange, yellow, blue, green, and white according to the level of perceived hazard. The results demonstrated that the hazard information communicated by different colors followed a consistent pattern across language groups. Red resulted in the highest hazard ratings followed by orange, yellow, blue, green, and white, respectively.
Researchers have examined a variety of attributes that influence a warning's ability to communicate important product hazards but Kline, Braun, Peterson, and Silver (1993) consider the attribute of color to have been neglected in the literature on warnings. The authors looked at the appropriateness of achromatic stimuli in product labels and found that color labels were perceived as more readable and hazardous than achromatic labels.
Braun and Silver (1994) examined the effect of color on compliance with printed warnings. The participants in the study indicated a higher likelihood of injury associated with products printed in red than green or black. Barbera and Gill (1987) attempted to assess the ability of two warning label designs on a consumer product to: (1) attract the user's attention, and (2) alter the user's behavior. For one condition, the warning label consisted of a one by three inch black and white tag affixed to the back of a portable electric heater. The alternative condition involved the same product but with a three by five inch red and black color coded warning label with pictographs. This second warning label was mounted at the end of the heater's electrical cord in the form of a "ski-pass" tag. Results revealed neither warning label to be effective. Even the "proper" warning label could not induce the subjects to read it nor alter their behavior.
Building on their previous work, Gill, Barbera, and Precht (1993) moved away from solely passive label designs. Three different warning label designs for a portable electric heater were tested: (1) a traditional non-human factored label; (2) a color-coded "ski-pass" label attached near the male end of the electric cord; and (3) a color-coded "interactive" label that required the user to interact with the label in order to use the heater. The results showed that the interactive design was most effective in attracting the user's attention, but none of them were effective in mediating safe user behavior, the ultimate goal.
Braun, Sansing, and Silver (1994b) examined the interaction of signal words and colors. A sample of 30 undergraduates rated the perceived hazard of 105 signal word/color combinations printed in specific hazard colors. Of the colors used, red conveyed the highest level of perceived hazard followed by orange, black, green, and blue. More importantly, however, it was noted that a signal word such as DEADLY connotated less hazard when printed in green than red ink.
Additional studies in the area of color and warnings support the earlier findings that the level of hazard communicated by signal words varies as a function of the color in which they are printed (Braun, Kennedy, and Silver, 1994c). Importantly, these findings suggest that signal word and color combinations create a continuum of perceived hazard. This model of a continuum may prove helpful to designers of warnings.
18.104.22.168 Cost of Compliance
Cost of compliance refers to the amount of effort a user must exert in order to comply with a warning (Kotwal and Lerner, 1995). The cost to the user of complying with a warning can be reduced, for example, by including protective wear (e.g., gloves, mask, ear plugs, etc.) with the hazardous product. Kotwal and Lerner point out that the issue of cost of compliance as a formal research topic is relatively new, as most of the literature which addresses this issue has been published within the past six years. Therefore, it is not surprising that, although several recent articles have demonstrated how strongly cost correlates with compliance, cost of compliance is not even mentioned in major industry standards (ANSI, 1991; Westinghouse Printing Division, 1981; ISO, 1984). Kotwal and Lerner conclude that the amount of effort required by the user to comply with the warning has not been a factor during the process of designing a warning, but rather, an afterthought to the design. However, the changes in compliance rates which result from lower cost to the user provide strong evidence for the inclusion of cost of compliance as a design factor in new labeling guidelines.
Intuitively, the lower the effort required by the user to comply with the warning, the more impressive the increase in compliance rates. In a field study (Dingus et al.,1991) university students agreed to evaluate the marketing potential of a "new" cleaning product. Compliance involved wearing rubber gloves while using the product. Subjects in the low cost condition had gloves provided with the product, while those in the high cost condition had no gloves provided. The results showed a 25% compliance rate for the high cost condition, increasing to 87-88% compliance for the low cost condition. Hunn et al. (1992) found that "in addition to reducing cost of compliance, the inclusion of gloves as part of product packaging increased the perception of danger associated with the product use." Therefore, increased compliance, in some cases, is probably a result of lower cost, as well as an increased perception of danger.
Similar increases in compliance rates were reported by Kotwal and Lerner during their literature review (1995). Increases in compliance rates range anywhere from 23% to almost 94%, depending on the given experimental situation. Wogalter, McKenna, and Allison, (1988) used a chemistry laboratory setting to determine high and low cost compliance rates (17% and 73%, respectively) for wearing protective gloves and masks. Studies on a racquetball court found high costs resulted in no compliance with wearing protective eyewear, while low cost gave 60% compliance (Hathaway and Dingus, 1992; Dingus, Hathaway, and Hunn, 1991).
Relating the cost of compliance to other warning features, Hunn et al. (1992) found that low cost of compliance had a greater effect on compliance than either warning content or warning interactivity. Additionally, Hathaway et al. (1992) found that the benefits of a low cost of compliance could be increased by adding specific consequence information to the warning. The researchers concluded that providing the user with information regarding frequency and severity of injuries associated with the hazard (in this case, eye injuries while playing racquetball), as well as providing the tools required to exhibit safe behavior (e.g., protective eyewear) can markedly improve warning effectiveness.
Table from Kotwal and Lerner (1995)
Summary of Research Findings Regarding Cost of Compliance
3.1.7 MEASUREMENT AND PROTOCOLS
Over time the focus of research on warnings has shifted from a debate on whether warnings work to systematic investigation of the factors that do or could influence the behavior of users of safety product. Ayers and his colleagues ( 1992) pointed out that the logical test of a warning must be reduction of the frequency and/or severity of accidents and injuries. This should be the key measurement.
Delegates from eight societies including: the American Society for Testing and Materials, American Association of Textile Chemists and Colorists, American Pharmaceutical Association, American Psychological Association, Illuminating Engineering Society, Optical Society of America, Textile Color Card Association of the United States, and U.S. Pharmacopoeia Convention collectively assembled into a group known as the Inter-Society Color Council, or ISCC (Billmeyer, 1991). The ISCC originally assembled to standardize color terms and color names used to describe the colors of pharmaceuticals. Even after the color standardization problem was solved, the ISCC continued to exist with the intent to advance the knowledge and use of color in art, science, and industry. This organization may be able to provide important assistance with the international harmonization efforts.
The Commission Internationale de l'Eclairage (CIE) sanctions the internationally accepted method of color specification and measurement that is designed to mimic human color perception (Hale, 1991). ANSI Z535 is a set of standards to guide technical communicators in the development of effective hazard alert messages. ANSI Z535.3-1991 is the ANSI standard entitled "Criteria for Safety Symbols" which contains a test method for determining the effectiveness of a pictogram and a criterion for success of 85% correct responses with no more than 5% critical confusion. The latter term refers to when the message conveyed is the opposite of the intended message, which for safety warnings could result in death. A score below this ANSI level does not mean the pictogram cannot be used but that it cannot be used alone and must be coupled with some other medium, such as a written warning. ANSI Z535.4 is the ANSI standard for Product Safety Signs and Labels.
The International Standards Organization issued a standard, ISO 9186, Procedures for the Development and Testing of Public Information Symbols, that recommends testing methodologies to evaluate symbols intended to be used internationally. This standard sets a lower level of acceptability compared to the ANSI standard. "For critical referents (e.g. safety symbols), the 66% criterion should be rigorously adhered to." These standards represent a major starting point for international harmonization but Kemnitz (1991) includes a reminder of the considerable time and effort required to understand and use any of these standards.
3.1.8 STANDARD PHRASES
The literature contains much investigation into the role of standardized phrases for warnings, particularly in the use of "signal words" such as DANGER or WARNING. Research previously cited by Godfrey and colleagues (1993) indicated that the use of a signal phrase such as "Government Warning" significantly influenced warning detection times. Consequently, the repetition of words or phrases through government mandate may be an important way to avoid confusion and increase recognition. Other studies on standard phrases are included in the MSDS portion of this report (Section 3.3).
22.214.171.124 Signal words
ANSI Z535.4-1991 entitled "Product Safety Signs and Labels" indicates when the following signal words should be used:The importance that ANSI and the U.S. military attaches to signal words may not be shared by the population at large. Given the lack of training, it is questionable how the public interprets different signal words. Leonard and Matthews (1993) used questionnaires with college student test subjects to determine how the population in general differentiates levels of warnings. No differences were found in ratings of perception of risk to different signal words. Further, size of the signal word and color of the signal word had no effect on perception of risk. Statements of consequences of disregarding the warnings and type of risk situation did affect rated perception of risk. Also, circumstances in which the subjects might be placed affected ratings of likelihood of disregarding warnings.
Silver and Wogalter (1993) tested the arousal effects of signal words on college students and found that DANGER connoted greater strength than WARNING and CAUTION. The results failed to show a difference between WARNING and CAUTION. Among other words tested, DEADLY was seen as having the strongest arousal connotation, and NOTE the least. From a long list of 84 terms, a "short" list of 20 signal words was developed based on understandability, low variability. shortness of word, and frequency of use. The authors suggested that an expanded list of signal words might alleviate potential problems of habituation from overuse of the currently recommended terms. This research was built upon similar, earlier work (Wogalter and Silver, 1990).
Although many organizations have guidelines for the determination of what signal words are to be used with specific hazards, these are usually unknown to the public. Leonard, Hill, and Karens (1989) developed information about how the general public perceives the degree of danger represented by signal words in warnings. For 15 items that had been rated for the seriousness of risk, 288 subjects were asked to indicate which signal word they would use to inform others of the hazard. Signal words that had been found to rate high in seriousness by Leonard, Karens, and Schneider (1988) tended to be used more with items rated as higher risks. Differences were found among age groups with older subjects using signal words that carried more serious connotations.
There are two systems for categorizing flammable liquids, one promulgated by the NFPA in 1986 (NFPA 321) and the other found in the Federal Hazardous Substances Act, 16 CFR 1500. Implicit in both systems is that consumers will recognize the degree of hazard associated with the terms "flammable" and "combustible." Main, Rhoades, and Frantz (1994) showed 46 drivers of step vans two photos of lighter fluid that were identical except one contained the warning "flammable" and the other "combustible." The subjects were asked which presented the greatest fire hazard. Only 23.9% of the subjects correctly identified the fluid marked "flammable" as the greater fire hazard while 47.8% incorrectly identified "combustible" as presenting the greater A second evaluation in the same study looked at how well subjects could use the generic flame symbol to choose correct specific actions to take with an extremely flammable adhesive. Less than 40% of the subjects reported that vapor flammability was a reason to ventilate a work area and only 4 out of 100 specifically mentioned that they would avoid using the product near pilot lights, the most frequently documented cause of accidents with these adhesives and the cause of a ban by the CPSC. This research indicates that the terminology and symbols of two current systems do not convey critical information to consumers, according to Main, Rhoades, and Frantz. The CPSC ban was initiated due to a study by Nelson (1976) that found the required labeling was not effective. The CPSC concluded that there was not a labeling scheme which would achieve the desired outcome and moved to a ban.
Wogalter et al. (1992) investigated the influence of warning signal words and a signal icon on perceptions of hazard for consumer products. Under the pretext of a marketing research study, 90 high school and college students rated product labels on variables such as product familiarity, frequency of use, and perceived hazard. Results showed that the presence of a signal word increased perceived hazard compared to its absence. Between extreme terms (e.g. NOTE and DANGER), significant differences were noted, but not between terms usually recommended in warning design guidelines. The presence of the signal icon had no significant effect on hazard perception.
The standard wisdom of whether four components are needed for warnings was evaluated by Wogalter, Desaulniers, and Godfrey (1993). Four-statement signs contained a signal word, a hazard statement, a consequence statement, and an instruction statement, as generally recommended. Four additional three-statement signs, each with a different component systematically removed from the four-statement sign, were used, for a total of 5 signs for each hazard situation. The results of Experiments 1 and 2 indicated that removing content statements reduced perceived effectiveness. Hazard and instruction statements were the most important, showing the greatest decrease in effectiveness when deleted. Signs for the most hazardous situations were perceived as the most effective warnings. Other results suggested that the deletion of redundant statements, particularly signal words, had less influence on effectiveness. The hazard statement showed the lowest redundancy consistent with it producing the greatest effectiveness decrement when deleted.
Ley (1995) looked at effectiveness of label statements for the Australian government. This work was provided to the IOMC as part of their call on chemical hazard communication. The study indicated that the following signal words were not well understood: combustible, flammable, hazardous, irritant, lethal, and toxic.
126.96.36.199 Standardization of label format
The Environmental Protection Agency has wrestled with whether standardizing pesticide labels is an effective course (EPA, 1986). After meeting with numerous stakeholder groups and establishing an advisory committee, three arguments were considered against standardization:
The EPA (1986) received an overwhelming response from stakeholders and experts in their evaluation of pesticide labeling that the required phrase, "Keep out of the reach of children" had no impact because of overuse. It was considered the least read section of the labels. A follow up report the next year (U.S. EPA, 1987) recommended discontinuing the child warning and replacing it with fresh statements that might be rotated every two to three years to attract attention.
3.1.9 ADDITIONAL RESEARCH NEEDS
Dorris and Purswell (1978) presented a list of needed research, including:
Stimulus: Energy Level and Contact
Most users of a product will not read the warning information because of a well documented "filtering" process.
The following variables affect filtering:
Few warnings are stored in long-term memory if they are more than six or seven lines long or address more than this number of hazards in using a product. More research is needed to better understand the reasons for the lack of long-term recall.
The Decision Making Process
Purswell, Krenek, and Dorris conclude their list of needed research with this item, which they identify as the most important. They hypothesize a threshold of perceived probability of an injury which must be reached before a person's behavior will be influenced by risk information. This threshold may be relatively high, i.e. 1/100 before most persons will respond to knowledge about hazards as presented in a warning. Since few products carry such risks, most warnings go unheeded. Purswell et al. cite Godfrey et al. (1985) concept of "cost of compliance." There may be benefits of noncompliance that are of significant importance to the individual. Consequently, individuals appear to construct some type of utility function for making the trade-off. This is another important area for research.
3.2.1 SUMMARY OF FINDINGS AND RECOMMENDATIONS
The following information represents a summary of the general areas of consensus in the literature on training. The summary also includes what the authors of this report.
Hazard communication is accomplished through a process that includes several tools for transferring information and influencing behavior. Laugher and Brelsford (1993) note that "numerous studies have shown the greater the a priori perception of hazardousness, the more likely people will look for and read a warning, and the more likely they will comply by taking safety precautions (Donner and Brelsford, 1988; Godfrey et al, 1983; Friedmann, 1988; LaRue and Cohen, 1987; Leonard et al., 1986; and Wogalter et al.)". Training programs are one of the components of a hazard communication program that can prime the workers' pump so that they are receptive to the important messages from other sources of communication.
3.2.3 NIOSH LITERATURE REVIEW
In 1995, a NIOSH published a report entitled "Assessing Occupational Safety and Health Training: A Literature Review" (Cohen and Colligan, 1995) The following is the author's abstract in its entirety.
"More than 100 Occupational Safety and Health Administration (OSHA) standards for hazard control in the workplace contain requirements for training aimed at reducing risk factors for injury or disease, and others limit certain jobs to persons deemed competent by virtue of special training. A literature review was undertaken to assess the merits of such training rules in achieving this objective and to sort out factors of consequence. The review focused heavily on reports of studies where training was used as an intervention effort to reduce risk of work related injury and disease. This literature yielded much evidence to show the merits of training. However, the findings had to be tempered because a) the work did not address OSHA rules per se b) knowledge gain and safe behavior measures as opposed to actual injury/disease indicators were used in many evaluations, and c) the training was in some instances coupled with other forms of intervention so as to make attribution difficult. Reports from select surveys and investigations of worker injuries and workplace facilities were also accessed and gave mixed results with regards to the rule that training deficits may have played as a contributing factor. Regarding other workplace training activities of relevance, reduced worker injury rates were found after the introduction of worksite first aid training programs in several studies, suggesting that this kind of instruction increases consciousness to job safety concerns. Results from analyzing the hazard control programs of companies with exemplary safety and health performance records found more time devoted to training and the greater involvement of supervisors in such efforts to be import characteristics.Cohen and Colligan looked at training intervention efforts designed to "enhance workers' knowledge of workplace hazards, affect behavior changes to ensure compliance with safe work practices or prompt other actions aimed at reducing the risk of occupational injury or disease." The report established a clear link between training and the establishment of healthy and safe working conditions. The authors noted that the studies reviewed were "near unanimous" in the benefits that training can achieve such as increased hazard awareness and safe workplace practices. Two caveats must be noted. First, the types of training programs that were evaluated were targeted to the specific workplace, not necessarily to an OSHA training requirement and second, the evaluation methodologies could not pinpoint the precise benefit that the "training" per se accomplished in the overall health and safety programs.
The authors noted several other variables that appear critical to accomplishing the overarching goal of healthy and safe workplaces including, "Managements role and support of safety training and its transfer to the jobsite, setting goals, and providing feedback to motivate use of the knowledge gained, and offering incentives, rewards for reinforcing safe performance" (Cohen and Colligan, 1995). These are all critical pieces in ensuring a successful program. In an evaluation of the implementation of the Hazard Communication Standard in manufacturing plants in Maryland, Sattler (1990) found that the most important variable in determining better compliance was explicit support for the program by plant-level management.
Some of the studies reviewed by Cohen and Colligan did not evaluate the quality of the training, merely the presence of training. The data were not as strong in defining the factors that contribute to an effective training.
3.2.4 RIGHT TO UNDERSTAND
The Labor Occupational Health Program (LOHP) at the University of California at Berkeley, produced The Right to Understand: Linking Literacy to Health and Safety Training, as part of a project funded by the NIEHS (Szudy and Arroyo, 1994). The document is a peer reviewed, well-referenced, and practical work that provides guidance for health and safety trainers. It emphasizes the challenges in training adult learners who may be illiterate or have reading difficulties (because of literacy issues or English as a second language). It takes into account basic principals of adult education, health education, and training methodologies that have been expressly adapted for health and safety subject matter. Additionally, it provides a review of literacy terms and statistics, including the fact that an estimated one-third of the U.S. workforce reads at or below the eighth-grade reading level.
The book has a chapter specifically devoted to creating materials that are easy to read, visually appealing, and illustrated to help explain the text. There is a chapter, as well, on field testing materials. The book contains several general recommendations that are appropriate for consideration in improving hazard communication training:
A side issue is the fact that in programs preparing health and safety professionals (at both undergraduate and graduate levels), very little coursework prepares the students for their roles as communicators or trainers. This has been corroborated in the Public Health Service Report on the Preparation of the Environmental Health Workforce (1991).
Wallerstein has written extensively on worker training and in a special issue of the American Journal of Industrial Medicine: Empowerment Approaches to Worker Health and Safety Education (1992), she outlines issues associated with low literacy and English as a second language and encourages the use of participatory training techniques to overcome these challenges. Literacy and language issues have not been substantially explored regarding hazard communication and therefore should be evaluated.
3.2.5 ADDITIONAL RESEARCH NEEDS
Suggestions were made by Cohen and Colligan for further work in the area of health and safety training:
3.3.1 SUMMARY OF FINDINGS AND RECOMMENDATIONS
The following recommendations represent the consensus taken from the literature on MSDS research, as well as key points the authors of this report felt needed emphasizing.
188.8.131.52International Use of Safety Data Sheets
The IOMC draft report resulted from contact with 270 national and international institutions. A "few selected organizations" (IOMC, 1996) were sent a questionnaire to ascertain more information. Of the ten nations that responded, all but two had a regulatory requirement or recommendation that safety data sheets be used. Seven of the eight required or recommended specific subjects to be covered in the document. As of May, 1997 Denmark, Finland, France, Germany, Norway, Sweden, Switzerland, the United States, and the United Kingdom required the production and use of MSDS but only Finland required producers to use a specific format (Phillips, 1997). The Canadian format is well known and is called the Workplace Hazardous Materials Information System (WHMIS). The system represents a good example of the difficulties facing attempts at international harmonization. The WHMIS could be used in the United Sates but the reciprocal is not true (West, 1991). The creation of the European Economic Community gave strong impetus to requiring preparation of MSDS for all chemicals and mixtures (Campbell, 1992). Worksafe Australia adopted a national MSDS format in 1989.
Some European countries have already adopted the ANSI Z400 format. This sixteen-section CMA/ANSI format was coordinated with major chemical associations of Canada, Europe, and Japan in reaching an agreement on section content and titles. In 1990, the General Conference of the International Labor Organization adopted Convention 170 and Recommendation 177 concerning safety in the use of chemicals at work and a classification scheme using the 16 sections of the ANSI format. The Commission of European Communities in a June 5, 1991 Directive delineated its safety data sheet format, which included the same 16 section headings as the ANSI Z400 document recommends. This directive made use of the document mandatory and it was to be available in appropriate user languages. A summary of the data elements of the OSHA Hazard Communication Standard, Canada's WHMIS, and the EC Directive can be found in Annex A of the ANSI Z400.1-1993 standard.
Another important international development in hazard communication has been the creation of International Chemical Safety Cards (ICSCs). This effort through the International Programme on Chemical Safety (IPCS) is a joint activity of three cooperating international organizations: the United Nations Environment Program, the International Labour Office, and the World Health Organization. Their work has been focused on providing information to developing countries. The goals were to develop a standardized format that was shorter (one double-sided page) and less complex than ANSI approach. Another goal was to identify standard phrases from the literature and then incorporate them into the card (Niemeier, 1997).
The International Chemical Safety Cards summarizes essential health and safety information on chemicals for their use on the "shop floor" level by workers and employers in factories, agriculture, and construction. They are verified and peer reviewed by internationally recognized experts and could serve as the principle information source in less developed areas or in small and medium size enterprises. The IPCS intends to create nearly 2000 cards in the next six years (NIOSH, 1997). Of particular importance for the harmonization efforts, the IPCS has computerized approximately 1200 standard phrases and will be testing a Windows 95 compatible software package soon that allows designers to create these cards using standard phrases and then translate them into any of the available languages automatically. The ICSCs and the software package should be available on the Internet soon (Niemeier, 1997).
Due to concerns about literacy levels among its workforce, Dow Chemical, in 1989, began using icons instead of standard phrases to inform workers of hazards. NIOSH has picked up this systems and has converted the standard phrases in its database to icons. These will be tested on workers employed as painters through the Center to Protect Workers Rights. They will use a CD-Rom multimedia approach providing the standard phrases on a screen along with the associated icon while pronouncing the phrases as part of training (Niemeier, 1997).
The MSDS is one of the prime tools for information transfer in the implementation of hazard communication programs. A study contracted by U.S. OSHA, evaluated the comprehensibility of MSDS to workers, the prime audience for hazard communication. The study (Kolp et al. 1993) was done prior to the development of the new ANSI Z400.1-1993 Standard for Hazardous Industrial Chemicals - Material Safety Data Sheet - Preparation (hereafter ANSI Z400.1) and examined the comprehensibility of a sample of MSDSs to a group of about 100 unionized workers in manufacturing industries located in the state of Maryland.
The purpose of the study was to assess the ability of the workers to understand, for each MSDS reviewed, the route of entry of the substance, the type of health hazard presented, what needs to be done to avoid or protect against the hazard, and where to go if additional help is necessary. A literature review indicated that the readability of an MSDS, i.e. the reading grade level at which the MSDS is written, is an important factor in determining the comprehensibility of an MSDS. To control for this variable, the investigators chose MSDSs with the average readability level at about a twelfth grade level, as determined by evaluating the readability of 100 randomly selected MSDSs.
Based on the recommendations of a reading expert in health messages (Freimuth 1979), the SMOG Grading Formula was applied in the study to determine the readability levels of the MSDSs. This formulation is a simple and fast test based on the proportion of polysyllabic words in a document. The SMOG formula, developed by McLaughlin, was the assessment tool of choice selected by the Office of Cancer Communication, at the National Cancer Institute. The MSDS format could not be controlled because the MSDS were randomly selected. Among one group of workers, an International Chemical Safety Card (ICSC) was also tested. When tested for readability, the ICSC read at an 11th grade reading level, and on the average was better understood by workers.
Before reporting the results, it must be noted that the workers who volunteered for this study understood that it relied on reading comprehension. This presented a selection bias and created a best-case-scenario for the results. Workers with reading difficulties or English as their second language would not have volunteered for the study. (Approximately 22% of the US adult population is functionally illiterate.) Nonetheless, it was found that on average the workers only understood about 60% of the health and safety information on the MSDSs associated with the hazardous chemical. Though the scope of the study was limited, the results were a cause to reconsider the role of MSDS as a communication tool. In reporting this study to OSHA, the investigators made the following recommendations for continued work and research on MSDSs:
At the time, the PIA made the suggestion that a standardized format be used for MSDS, as well as standardized color-coded sections for easier access to emergency information, and standardized signs and symbols for vital information. They further suggested that MSDS be produced in other languages (this is not required by OSHA) and be written at a reading level no higher than twelfth grade.
In a Canadian journal on occupational safety and health, Nore (1990) expressed the need to be alert to the relationship between literacy and hazard communication. Nore cited a Canadian study finding that seven percent of a general adult population could not use printed documents commonly found at work, home, or in the community, and that 20% could not read the instructions that come with aspirin. Lack of reading skill makes full, safe and productive workplace participation difficult when workers are dependent on the written word to communicate. Nore had several recommendations: offering literacy programs in workplaces; incorporating literacy programs into health and safety training; using a variety of methods for communicating, in addition to written materials; "buddying" new employees with veteran employees; and making health and safety documents more readable.
In an article about functionally illiterate workers, Samways (1988) raised the concern that access to MSDSs may satisfy the employees right-to-know, however it may not satisfy their right to understand. Samways suggested the use of simplified factsheets. Samways also suggested training methodologies that incorporate variety in the presentation media, that provide an opportunity to ask questions, and provide training in languages responsive to the audiences' language needs.
A major effort to improve the quality of the MSDS is represented by the consensus-built development of the ANSI Z400.1 Standard for the development of MSDSs to be used under "industrial occupational conditions." However, it recognizes in its section titled "Audiences for MSDSs" that the following individuals may use MSDSs as their source of information: community member, emergency responder, employee, employer, environmental professional, medical professions, and occupational health and safety professional.
The following is extracted directly from Section 3.2 of the ANSI Z400.1 Standard:
The ANSI Z400.1 Standard does not provide a standardized format but rather standardizes the steps to be taken in preparing an MSDS and creates a structure (16 section headings) in which to work. The standard is currently under its first revision. No study has ever been done to determine whether MSDSs prepared following the ANSI-Z400.1 recommendations were more or less comprehensible than typical MSDSs.
This raises an important question regarding the effectiveness of the following recommendations made to OSHA by the National Advisory Committee on Occupational Safety and Health - HazCom Workgroup:
OSHA should endorse the order and section titles as described in ANSI Z400.1-1993, and strongly support the use of other recommendations from this voluntary standard. This can be done through release of an OSHA memorandum, an interpretive letter, interpretive notice and /or listing of the order and section titles as a non-mandatory appendix to the HCS (Hazard Communication Standard).As mentioned before, the ANSI Z400.1 Standard provides guidance for the preparer of MSDSs to include the information in a standardized sequence and gives guidance on addressing readability issues. However, according to a member of the Society for Chemical Hazard Communication who is now providing national training on MSDSs preparation using ANSI Z400.1, if several professionals were asked to use the standard to develop an MSDS on a specific chemical, the resulting MSDSs could look quite different (Ignatowski, 1997).
A selection of standard phrases recommended in the ANSI Z400.1 were formally evaluated in a study commissioned by the Chemical Manufacturers Association (Lehto, 1993). In this study, 63 standard phrases were evaluated on the basis of syntactic analysis (using computerized readability methods); semantic analysis (determining the use and understandability of individual words); and a Structured Focus Group analysis determining the comprehensibility by a group of chemical workers. It was found that the chemical workers rated their understanding of nearly all of the phrases somewhere between "understand completely" and "understand mostly."
In the second study, workers enrolled in a study in which they knew they would be tested on reading comprehension. Their education levels were as follows:
Other characteristics of the workers enrolled in the study which may make the study more difficult to generalize includes the following:
In the final analysis of their study evaluating the content, structure and format of MSDSs, Cohen et al. (1989) suggest that MSDS, by themselves, are a poor means of informing workers of hazards to which they may be exposed for the following reasons: 1) much of the technical information has little meaning to the average worker and may even frustrate the workers' ability to read other portions of the MSDSs that have information pertinent to hazard recognition and safe practices; 2) information depicting hazardous conditions, signs or symptoms of exposure, and safe handling procedures are written generically and workers may have difficulty seeing the connection between their own use of the chemical and the information on the MSDS; and 3) the information may contain terms too difficult to understand or too brief and vague to actually generate the concern that workers should have regarding safe use of the chemicals.
The most recent study evaluating the comprehensibility of MSDS and comparing the efficacy of the International Chemical Safety Card, the ANSI Z400 format, and an OSHA type form was done with a population of unionized and non-union workers at a large national (U.S.) research laboratory, Phillips, (1997). The employees represented a large number of crafts, such as painters, carpenters, truck drivers, and general laborers. Ninety-five percent of the union workers studied were trained on MSDS. Thirty-nine percent of the workers found the MSDS "difficult to understand". One third of the workers found the MSDS format to be confusing. Each worker was pre and post-tested for knowledge regarding a hazardous chemical before and after reading an MSDS. All three formats improved the workers' knowledge to a certain degree. In rank order, the International Chemical Safety Card (ICSC) faired the best, the OSHA type form second, and the ANSI Z400 ranked last. Variability existed on specific questions. For example,
The following is a selection of recommendations made by Phillips in light of his study:
Important to note, is the fact that ICSC are prepared with the explicit intent that they "be used at the 'shop floor' level by workers." The ICSC were field tested in the initial stages of development. With regard to market testing the warning system, principle #4, Laughery and Brelsford further note that "Despite the designer's knowledge of receiver characteristics and efforts to apply that knowledge, warnings should be market tested" to assess comprehension and behavioral intentions. They also observe that "such minimal efforts are seldom part of the warning design process."
Another study noting the challenges that must be overcome, examined the accuracy of the data presented within the MSDS (Kolp et al, 1994). The study evaluated the accuracy and completeness of data on 150 randomly selected MSDS in five categories of information:
The human health effects data are generally thought to be sparse given the number of "hazardous chemicals" being used in workplaces. However, in some instances it has been noted that MSDSs are particularly faulty. Paul and Kurtz (1994) note that the reproductive health hazards were deficient in a limited study of MSDSs when they looked at MSDSs for lead and ethylene glycol ethers, two substances with well documented risks to reproductive health.
3.3.3 HAZARD COMMUNICATION AS A WHOLE AND DYNAMIC PROCESS
The literature on the elements of hazard communication have been reviewed above. The relative merits of these parts as they contribute to the specific goals of hazard communication in the workplace has not been evaluated. The Office of Cancer Communication at the U.S. National Cancer Institute publishes Making Health Communication Work: Planner's Guide (NIH, 1992) (hereafter Guide) which provides an excellent framework for evaluating overall hazard communication policies and practices, as well as hazard communication programs within workplaces. In addition, this framework can be useful in making recommendations for the future direction of hazard communication both within the U.S. and internationally. The Guide divides the health communication process into six stages. These stages are used below to elaborate questions that may be useful in directing future research needs and policy analysis. The questions which are posed in italics are taken directly from the Guide. The additional questions are posed by the authors of this report.
STAGE 1 Planning and strategy selection The first stage is the foundation for any communication program. A careful assessment of the problem must be outlined. Regarding hazard communication, the "problem" is that workers need meaningful information about the hazardous chemicals with which they work or to which they may be exposed during the course of their work so that they can make informed decisions about their behavior. The following questions should be posed to assist in planning:
Overall, what change is planned to solve or lessen the problem?
What measurable objectives can be established to define success? The only measurable objectives for success that have been applied in most workplaces are whether or not component parts of the hazard communication standard requirements are in place. Another objective, at the worksite level, may be to improve the knowledge and change the behavior of management and workers as they apply to hazardous chemicals.
How can progress be measured? At the worksite level, measures of progress should be developed.
What should the target audience be told? Although there is little debate as to what information is critical for workers to have, there has been much written about the disadvantages of including complex technical information which then diminishes the overall goals to achieve comprehension and ultimately behavioral changes. In planning the International Chemical Safety Cards, a process was created to establish what should and should not be placed on the card, that have workers as the primary audience. The ANSI Z400.1 Standard created a document for multiple audiences that, in turn, may not be the best tool to communicate information expressly to workers.
STAGE 2 Selecting a Medium and Materials In selecting the medium and materials to communicate the message, the following questions should be posed:
Are there any existing materials which could be adapted for the program? This question goes to the heart of the international harmonization efforts. The materials selected should be based on sound principles of communication and should be substantially evaluated before the final selection is made.
Which media are most appropriate for reaching the target audience? The following questions should be answered regarding the medium:
Is it appropriate for your target audience in format, style, and readability level?
Is it available?
Could it be modified to become appropriate?
Were the materials tested?
How have they been used?
Were they effective?
What formats will best suit the medium and the messages? The scientific literature regarding hazard communication formats is fairly prescriptive and should be followed in reformulating the thinking regarding MSDS in the U.S. especially as international harmonization is considered. It appears evident that a simpler information sheet with information specific to workers' immediate needs could be effective. Any new format should be extensively tested with a variety of worker populations
STAGE 3 Developing Material and Pretesting The message should be evaluated and the target audience's reactions to the message should be assessed. During the course of the ANSI Z400.1 Standard development a study was done to determine the comprehensibility of selected hazard communication phrases to be used on labels. Although this was a useful study, the comprehensibility of an MSDS in its entirety was not evaluated.
There is a set of four questions that the Guide recommends pertaining to the audience:
DOES THE AUDIENCE:
Accept it as important?
Agree with the value of the solution (safe behavior)?
STAGE 4 Implementation The implementation phase must include the coordination of all the stakeholders in the process so that the overall program goals can be achieved. Program components should be periodically reviewed and revised, if necessary. During this phase, it is important to know whether the message is making it through the intended channels of communication. When hazard communication programs are being implemented, a key player in the process is the line supervisor. In a review of hazard control programs with exemplary safety and health performance records, it was found that the time devoted to training and the involvement of supervisors were important factors (Cohen and Colligan, 1995). The supervisors must understand the link they are responsible for in the chain of hazard communication.
Is the target audience paying attention and reacting? The answer to this question is largely dependent on the quality of the training that is offered in support of the other communication tools, such as labels and MSDSs. The use of video training, in the absence of any other communication process will not effectively capture the audience's attention, especially if the video training is offered, as it often is, at the end of a worker's shift.
STAGE 5 Assessing Effectiveness The program should be assessed by analyzing the results of measurements established in the planning stage. Were the program's objectives met? Most employers will translate this into determining whether or not the various compliance activities were implemented. The literature reporting that knowledge or improved work practices resulted from hazard communication programs is limited. This is largely because it is not required to evaluate the effectiveness of a program (and because research funding is virtually non-existent for such endeavors to be done by university and research institutions and others who might publish the results). These type of evaluations may be taking place "in-house" within industries, however reports of the results of such evaluations are not getting published.
In the U.S. auto industry, hazard communication has been taken very seriously and a substantial investment has been made to implement their programs, which were created and implemented by the combined participation of the industry, the United Auto Workers, and the University of Michigan. In an evaluation of the Ford Motor Company, hazard communication training program, it was determined that "more interactive, trainer intensive delivery methods to smaller groups were associated with more positive effects on reported training usefulness and changes in work practices and working conditions" (Robins et al. 1990). Work must continue to be supported to identify the critical variables that will predict training successes.
STAGE 6 Feedback to Refine Program In some ways, Canada, the U.S., the European Community and others who have developed some hazard communication mechanisms are now in stage 6, a time of evaluating feedback and refining their programs. This is, once again, especially true as harmonization is considered. The more information that can be gathered, the more likely it is that the following questions will be answered:
Are there program changes that should be made to increase the likelihood of success or to address changes in the audience, or problem or situations?
Are there lessons learned that could make future programs more successful?
Based on the review of the literature on the major aspects of hazard communication, it appears that there are two significant research needs. The first is understanding how we can significantly improve MSDS so they can become a more meaningful tool for communication. And the second is how and where the "communication" in hazard communication occurs. What are the variables that will predict that a worker will receive and understand the information necessary to make informed decisions in his/her workplace? What are the variables determining safe and informed behaviors regarding hazardous chemicals?
Regarding the quality of MSDS, we may need to consider whether the MSDS in its current form is the best written communication tool and, if not, look seriously for how it might be altered. Regardless of what ultimate form it takes, it should be in a standard format. There should be some minimum qualification for the person preparing the document.. There should be a system of random, periodic review of a selection of MSDS by a third party to determine the accuracy of the data represented on the sheets and sanctions applied for non-compliance.
As we move into the 21st Century, it is important to consider who and what are moving with us. There are immense changes in information technology that have erupted in the last decade for the transmission of information, factsheets, checklists, training curriculum and other of the basic tools of communication. The international public health and safety community could be convening a variety of groups of experts to help form the vision and strategic plan for bringing high quality, effective hazard communication programs into the next century. For example, an expert panel in computer and information transfer technology could be convened to discuss how to deliver a global hazard communication program. Such information can help to inform the longer term goals that should be considered in the next phase of international hazard communication development and harmonization.
National campaigns have helped to change attitudes and behaviors regarding seatbelts, helmets, and recycling. Social marketing has proven to be a powerful force for changing behavior on a grand scale. It might be helpful to elicit the support of experts in social marketing to hear their ideas for the best messages to achieve hazard communication goals.
Most importantly, it must be stressed that sound science and proven efficacy provide the foundation for our next stages in the very important mission of achieving worker and community health and safety.
American National Standards Institute. 1987. Accident Prevention Tags. ANSI Z535.5, draft. New York: National Electrical Manufacturers Association.
American National Standards Institute. 1978. American National Standard Manual on Uniform Traffic Control Devices for Streets and Highways. ANSI D6. New York: National Electrical Manufacturers Association.
American National Standards Institute. 1972. American National Standard Specifications for Accident Prevention Signs. ANSI Z535.1. New York: National Electrical Manufacturers Association.
American National Standards Institute. 1987. Criteria for Safety Symbols. ANSI Z535.3, draft. New York: National Electrical Manufacturers Association.
American National Standards Institute. 1987. Environmental and Facility Safety Signs, ANSI Z535.2, draft. New York: National Electrical Manufacturers Association.
American National Standards Institute, 1988. Hazardous Industrial Chemicals Precautionary Labeling. ANSI Z129.1. New York: National Electrical Manufacturers Association.
American National Standards Institute. 1987. Product Safety Signs and Labels, ANSI Z535.4, draft. New York: National Electrical Manufacturers Association.
American National Standards Institute. 1987. Safety Color Code, ANSI Z535.1, draft. New York: National Electrical Manufacturers Association.
American National Standards Institute. 1993. American National Standard for Hazardous Industrial Chemicals Material Safety Data Sheets Preparation: Z400.1-1993. New York: American National Standards Institute. (June 3).
American National Standards Institute. 1994. American National Standard for Hazardous Industrial Chemicals Precautionary Labeling: Z129.1. New York: National Electrical Manufacturers Association.
Anonymous. 1987. "MSDS: Required Reading", Ohio Monitor (June).
Ayres, T.J.; Gross, M.M.; Wood, C.T.; Horst, D. P.; Beyer, R. R.; and Robinson, J.N. (Failure Analysis Associates, Inc.), 1994. What is a Warning and When will it Work?, Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society.
Ayres, T.J.; Gross, M.M.; Horst, D.P.; and Robinson N. 1992. A Methodological Taxonomy for Warning Research, Proceedings of the Human Factors Society 36th Annual Meeting 499-503.
Babbitz, M.A., 1986. Hazard Communication. Workers' Right to Know Nurses' Need to Know. AAOHN Journal 34(6):261-263.
Barbara, C., and Gill, R. 1987. Human Factors and Warning Label Design, Interface 87 91-94.
Bettman, J.; Payne, J.; and Staelin, R. 1986. Cognitive Considerations in Designing Effective Labels of Presenting Risk Information, Journal of Public Policy and Marketing 5:1-28. Cited in Andrews, Netemeyer and Durvasula 1991.
Billmeyer, F. 1991. Inter-Society Color Council and ASTM, ASTM Standardization News. (March).
Bogett, W.R., and Rodriguez, L. 1987. On the Influence of a Perception of Danger on the Effectiveness of Warnings for Product Users. Hazard Prevention 23(4):19-24.
Braun, C.C.; Kennedy, R. S.; and Silver, N.C. 1994. Signal Word and Color Specifications for Product Warnings: An Isoperformance Application. Proceedings of the Human Factors and Ergonomics Society 38th Meeting:1104-1108.
Braun, C.C., and Silver, N.C. 1995. Interaction of Signal Word and Colour on warning Labels: Differences in Perceived Hazard and Behavioural Compliance. Ergonomics 38(11):2207-20.
Braun, C.C., and Silver, N.C. 1994. Differences in Behavior Compliance as a Function of Warning Color. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting:379-383.
Braun, C.C.; Sansing, L.; and Silver, N.C. 1994. The Interaction of Signal Word and Color on Warning Labels: Differences in Perceived Hazard. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting:831-835.
Braun, C.C.; Silver, N.C.; and Stock, B.R. 1992. Likelihood of Reading Warnings: The Effect of Fonts and Font Sizes. Proceedings of the Human Factors and Ergonomics Society 36th Annual Meeting:926-930.
Brelsford, J.W.; Wogalter, M.S.; and Scoggins, J.A. 1994. Enhancing Comprehension and Retention of Safety-Related Pictorials. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting:836-840.
Bresnahan, T.F., and Bryk, J. 1975. The Hazard Association Values of Accident-Prevention Signs", Professional Safety (January):17-25.
Brimmell, C. 1987. Study Reveals that Many Safety Symbols Warning of Hazards are Not Understood. Consensus (14)1:5-6.
Broadbent, D.E. 1977. Language and Ergonomics. Applied Ergonomics 8(1):15-18.
Cahill, M.C. 1975. Interpretability of Graphic Symbols as a Function of Context and Experience Factors. Journal of Applied Psychology 60(3):376-380.
Cahill, M.C. 1976. Design Features of Graphic Symbols Varying in Interpretability. Perceptual and Motor Skills (42)2:647-653.
Cairney, P. and Sless, D. 1982. Communication Effectiveness of Symbolic Safety Signs with Different User Groups. Applied Ergonomics (13)2:91-97.
Campbell, S.L. 1992. A New Look for the MSDS. Occupational Health and Safety (61)10:78-80.
Caron, J.P.; Jamieson, D.G.; and Dewar, R.E. 1980. Evaluating Pictographs Using Semantic Differential and Classification Techniques. Ergonomics 23(2):137-146.
Carriere, R. 1985. Chemical Handling Training Aided by MSDS Information Chart. Occupational Health and Safety 54(3):36-39.
Christ, R.E. 1975. Review and Analysis of Color Coding Research for Visual Displays. Human Factors (17)6:542-570.
Cohen, A.L.; Schmitt, N.; and Colligan, M.J. 1989. Aspects of Health Risk Communication: Material Safety Data Sheets for Informing Workers of Chemical Hazards at Work. Paper presented at the National Safety Council Annual Congress, Chicago.
Cole, B.L., and Brown, M. NIEHS Hazardous Waste Worker Training Grantees Evaluation Procedures Executive Summary.
Colligan, M.J., Sinclair, R.C. 1994. The Training Ethic and the Ethics of Training. Occupational Medicine: State of the Art Reviews 9(2):127-134.
Collins, B.L. 1983. Evaluation of Mine-Safety Symbols. Proceedings of the Human Factors Society 27th Annual Meeting:947-949.
Collins, B.L., and Lerner, N.D. 1982. Assessment of Fire-Safety Symbols. Human Factors 24(1):75-84.
Collins, B.L., Lerner, N.D. and Pierman, B.C. June 1982. Symbols for Industrial Safety. National Bureau of Standards, Washington, D.C., 1-102, NSBIR 82-2485.
Collins, B.L. 1983. Evaluation of Mine-Safety Symbols. Proceedings of the Human Factors Society 27th Annual Meeting:947-949.
Consumer Reports. 1990. What's Wrong with This Label? (May):326-327.
Consumer Product Safety Commission Federal Register. 1996. 16 CFR Part 1500, Requirements for Labeling of Retail Containers of Charcoal. (61)87: 19818-19829.
Covello, V.; Winterfeldt, D.; and Slovic, P. 1986. Risk Communication: A Review of the Literature. Risk Abstracts 3(4):171-182.
Dejoy, D.M. 1989. Consumer Product Warnings: Review and Analysis of Effectiveness Research. Human Factors Perspectives on Warnings:936-940.
Desaulniers, D.R. 1991. An examination of consequence probability as a determinant of precautionary intent. Ph.D. diss., Rice University, Houston.
Desaulniers, D.R. 1993. Layout, Organization, and the Effectiveness of Consumer Product Warnings. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society.
Deutsch, S. 1996. Building a Trainer's Community: Innovations in Worker Health and Safety Training. New Solutions:68-72.
Dewar, R.E. 1976. The Slash Obscures the Symbol on Prohibitive Traffic Signs. Human Factors 18(3):253-258.
Dewar, R.E. and Ells, J.G. 1977. The Semantic Differential as an Index of Traffic Sign Perception and Comprehension. Human Factors 19(2):183-189.
Dingus, R.A.; Wreggit, S.S.; and Hathaway, J.A. 1993. Warning Variables Affecting Personal Protective Equipment Use. Safety Science (16) 5/6:655-673.
Dionne, E.D. 1979. Effective Safety Signs and Posters. National Safety News (120)4:48-52.
Dixon, P. 1982. Plans and Written Directions for Complex Tasks. Journal of Verbal Learning and Verbal Behavior 21:70-84.
Doblin, D. A Structure for Nontextural Communication. no other info. 89-112..
Donner, K.A. and Brelsford, J.W. 1988. Cueing hazard information for consumer products. Proceedings of the Human Factors Society 32nd Annual Meeting, Human Factors Society:532-535.
Dorris, A.L. 1991. Product warnings in theory and practice. Some questions answered and some answers questioned. Proceedings of the Human Factors Society 35th Annual Meeting 2:1073-1077.
Dorris, A.L., and Purswell, J.L. Warnings and Human Behavior: Implications for the Design of Product Warnings. Journal of Product Liability, no. 1 (1977):255-264.
Duffy, T.M., 1985. Chapter 6: Readability formulas: What's the use? In Designing Usable Texts, T.M. Duff, and R. Wailer (Eds.) Orlando: Academic Press, Inc., 113-140.
Dunlap, G.L., Granda, R.E. and Kustas, M.S., 1986. Observer perceptions of implied hazard: safety signal word and color words. IBM Technical Report (T 00.3428)
Easterby, R.S., and Hakiel, S.R. 1981. Field Testing of Consumer Safety Signs: The Comprehension of Pictorially Presented Messages. Applied Ergonomics 12(3):143-152.
Easterby, R.S. and Hakiel, S.R. December, 1977. Safety Labelling of Consumer Products: Shape and Colour Code Stereotypes in the Design of Signs. AP Report No. 75:1-43. University of Aston in Birmingham.
Easterby, R.S., and Zwaga, H.J.G. 1976. Evaluation of Public Information Symbols ISO Tests: 1975 Series. A.P. Report No. 60: 1-62. University of Aston in Birmingham, (March).
Evaluation of the Ford Motor Company/United Automobile Workers Hazard Communication Training Program Final Report, March 21, 1989.
Flesch, R. 1974. The Art of Readable Writing. Harber & Row, Publishers, Inc.
FMC Corporation. 1985. FMC, Product Safety Sign and Label System, 3rd ed., Santa Clara, California.
FMC Corporation. 1985. Product Safety Signs and Label System (national and international use of pictorials and symbols).
Frances, W.N. and Kucera, H. 1982. Frequency Analysis of English Usage. Houghton Mifflin Company, Boston.
Frantz, J.P.; Miller, J.M.; and Lehto, M.R. 1994. Must the Context be Considered When Applying Generic Safety Symbols: A Case Study in Flammable Contact Adhesives.
Frantz, J.P. 1993. Effect of Location and Presentation Format on Attention to and Compliance with Product Warnings and Instructions. Journal of Safety Research 24:131-154.
Frantz, J.P. 1992. Effect of Location, Procedural Explicitness, and Presentation Format on User Processing of and Compliance With Product Warnings and Instructions. Ph.D. diss., University of Michigan.
Freimuth, V.S. November December 1979. Assessing the Readability of Health Education Messages. Public Health Reports.
Friedman, K., 1988. The Effect of Adding Symbols to Written Warning Labels on User Behavior and Recall. Human Factors (30)4:507-515.
Galer, M. 1980. An Ergonomics Approach to the Problem of High Vehicles Striking Low Bridges. Applied Ergonomics 11(1):43-46.
Garbo, M.J.; Comeau, N.A.; and Swanson, K.A., 1988. OSHA Hazard Communication Standard. Helping Prevent Chemical Hazards. AAOHN Journal. 36(9):366-371.
Gill, R.T.; Barbera, C.; and Precht, T. 1993. A Comparative Evaluation of Warning Label Design. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:50-52.
Gill, R.T.; Barbera, C.; and Precht, T. 1993. A Comparative Evaluation of Warning Label Design. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:50-52.
Godfrey, S.D.; Rothstein, P.R.; and Laughery, K.R. 1993b. Warnings: Do They Make a Difference?. Human Factors Perspectives on Warnings. California. The Human Factors and Ergonomics Society:67-71.
Godfrey, S.S., and Laughery, K. R. 1993c. The Biasing Effects of Product Familiarity on Consumers Awareness of Hazard. Human Factors Perspective on Warnings. California: The Human Factors and Ergonomics Society:58-61.
Godfrey, S.S.; Laughery, K.R.; Young, S.L.; Vaubel, K.P.; Brelsford, J.W.; Laughery, K.E.; Horn, E. 1993a. The New Alcohol Warning Labels: How Noticeable Are They?. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:62-66.
Godfrey, S.S.; Allender, L.; Laughery, K.R.; and Smith, V.L. 1983. Warning Messages: Will the Consumer Bother to Look?. Proceedings of the Human Factors Society 27th Annual Meeting, Human Factors Society:950-954.
Green, P., and Pew, R.W. 1978. Evaluating Pictographic Symbols: An Automotive Application. Human Factors 20(1):103-114.
Hadden, S.G. 1989. Providing Citizens with Information about Health Effects of Hazardous Chemicals. Journal of Occupational Medicine 31(6):528-534.
Hadden, S. 1986. Read the Label: Reducing Risk by Providing Information, Westview/AAAS: Boulder, CO.
Hale, W. 1991. Does it Look O.K.?. ASTM Standardization News. (March):50-53.
Hartley, J. 1981. Eighty Ways of Improving Instructional Text. IEEE Transactions on Professional Communication 24(1):17-27.
Hartley, J. 1978. Designing Instructional Text. Nichols Publishing Company, New York, 1-125.
Hartley, J. 1978. Designing Instructional Text. Nichols Publishing Company, New York, 1-125.
Hartley, J. 1981. Eighty Ways of Improving Instructional Text. IEEE Transactions on Professional Communication 24(1):17-27.
Horst, D.P.; McCarthy, G.E.; Robinson, J.N.; McCarthy, R.L.; Krumm-Scott, S. 1993. Safety Information Presentation: Factors Influencing the Potential for Behavior. Human Factors Perspective on Warnings. California: The Human Factors and Ergonomics Society:86-90.
Howett, G.L. 1983. Size of Letters Required for Visibility as a Function of Viewing Distance and Observer Visual Acuity. Building Physics Division, Center for Building Technology, National Engineering Laboratory, National Bureau of Standards, Washington, D.C., July.
Hurst, R. 1986. Using Federal Standards to Determine Adequacy of Consumer Products's Precautionay Labeling. Veterinary and Human Toxicology 28:557-562.
International Chamber of Commerce. 1991. Environmental Labelling Schemes (ELS), ICC position paper, Paris, June.
Israel, B.A.; Cummings, M.; Dignan, M.B.; Heaney, C.A.; Perales, D.P.; Simons-Morton, B.G.; and Zimmerman, M.A. 1995. Evaluation of Health Education Programs: Current Assessment and Future Directions. Health Education Quarterly 22:364-389.
Jaynes, L.S., and Boles, D.B. 1990. The Effect of Symbols on Warning Compliance. Proceedings of the Human Factors Society 34th Annual Meeting (2):984-987.
Johnson, D. 1993. A Warning Label for Scaffold Users. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:95-99.
Kaminski, M.; Graubarth, R.; And Mock, A. 1995. Using Grant-Based Training as a Vehicle for Lasting Change: Strengthening the Role of Local Health and Safety Activists. New Solutions:6 - 14.
Kammann, R. 1975. The Comprehensibility of Printed Instructions and the Flowchart Alternative. Human Factors 17(2):183-191.
Kanouse, D.E., and Hayes-Roth, B. Cognitive Considerations in the Design of Product Warnings in Product Labeling and Health Risks. Morris, L.A. Mazis, M.B. and Barofsky, I. (Eds.), 1980, Cold Spring Harbor Laboratory. Banbury Report 6. 147-163.
Kemnitz, C., 1991. How to Write Effective Hazard Alert Messages. Technical Communications, First Quarter:68-73.
Kieras, D. 1989. A Methodology for Computer Interface Design in the Handbook of Human-Computer Interaction.
Kieras, D., and Bovair, S. 1986. The Acquisition of Procedures from Text: A Production-System Analysis of Transfer of Training. Journal of Memory and Language 25:507-524.
Kieras, D.E. and Deckert, C. 1985. Rules for Comprehensible Technical Prose: A Survey of Psycholinguistic Literature. Technical Report No. 21 (TR-85/ONR-21). (June):1-21.
Klare, G.R. 1974. Assessing Readability. Reading Research Quarterly 10(1):62-102.
Kline, P.B.; Braun, C.C.; Peterson, N.; and Silver, N.C. 1993. The Impact of Color Warnings Research. Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting:940-943.
Kolp, P.W.; Williams, P.L.; and Burtan, R.C. 1995. Assessment of the Accuracy of Material Safety Data Sheets. Journal of the American Industrial Hygiene Association 56:178-183.
Kolp, P.; Sattler, B.; Blayney, M.; Sherwood, T. 1993. Comprehensibility of Material Safety Data Sheets. American Journal of Industrial Medicine (23)1:135-141.
Kotwal, Bari, and Lerner, Neil. Product Labeling Guide Literature Review. Comsis Corporation. Report generated for the Consumer Product Safety Commission. Contract CPSC-C-93-1132. 1995.
Kreindler, R.J., Luchsinger, V.P. 1978. How Readable are Your Safety Publications. Professional Safety (23)9:40-42.
Krenzelok, E.P.,and Dean, B.S. 1988. Hazardous Substance Center; A Poison Center's Workers' Right to Know Program. Veterinary and Human Toxicology 30(1):18-20.
Kurtz, Sr. P.M. 1994. Analysis of Reproductive Health Hazard Information on Material Safety Data Sheets for Lead and the ethylene glycol ethers. American Journal of Industrial Medicine 25(3):403-405.
LaRue, C., and Cohen, H. 1987. Factors influencing consumers' perceptions of warning: an examination of the differences between male and female consumers. Proceedings of the Human Factors Society 31st Annual Meeting:610-614.
Laughery, K.R. and S.L. Young. 1991. Eye scan analysis of accessing product warning information. Proceedings of the Human Factors Society 35th Annual Meeting 1:585-589.
Laughery, K.R., and Brelsford, J.W. 1993. Receiver Characteristics in Safety Communications. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:120-124.
Laughery, K.R..; Rowe-Hallbert, A. L.; Young, S.L.; Vaubel, K.P.; and L.F. Laux. 1991. Effects of explicitness in conveying severity information in product warnings. Proceedings of the Human Factors Society 35th Annual Meeting 1:481-485.
Laux, L., Mayer, and DL, Thompson, N.B. 1989. Usefulness of Symbols and Pictorials to Communicate Hazard Information. Proceedings of INTERFACE 89:79-83.
Lehto, M.R. 1992. Designing Warning Signs and Warning Labels: Part I Guidelines for the Practitioner. International Journal of Industrial Ergonomics (10)1-2:105-113.
Lehto, M.R., and Miller, J.M. 1986. Warnings: Fundamentals, Design, and Evaluation Methodologies, Fuller Technical Publications, Ann Arbor, Michigan.
Lehto, M.R. 1992a. Designing Warning Signs and Labels: Part I - Guidelines for the Practitioner. International Journal of Industrial Ergonomics.
Lehto, M.R. 1992. Designing Warning Signs and Warning Labels: Part II Scientific Basis for Initial Guidelines. International Journal of Industrial Ergonomics (10)1-2:115-138.
Lehto, M.R. and DeSalvo, T.E. 1992. Evaluation of the Comprehension of Hazard Communication Phrases by Chemical Workers. A technical report prepared for the Chemical Manufacturers Association by Consumer Research, Inc., West Lafayette, IN, (April 6).
Lehto, M.R., and DeSalvo, T.E. 1993. Evaluation of Proposed Labeling Phrases. A report prepared for the Chemical Manufacturers Association by Consumer Research, Inc., West Lafayette, IN, (July 20).
Lehto, M.R. 1992b. Designing Warning Signs and Labels: Part II Scientific Basis for Initial Guidelines. International Journal of Industrial Ergonomics.
Lehto, M.R. and Miller, J.M. 1986. Warnings Volume I: Fundamentals, Design, and Evaluation Methodologies, Ann Arbor, MI, Fuller Technical Publications.
Lehto, M.R., and Papastravrou, J.D. 1993. Models of the Warning Process: Important Implications Towards Effectiveness. Safety Science 16.
Lehto, M.R. 1992a. Designing Warning Signs and Labels: Part I Guidelines for the Practitioner. International Journal of Industrial Ergonomics.
Lehto, M.R. 1992b. Designing Warning Signs and Labels: Part II Scientific Basis for Initial Guidelines. International Journal of Industrial Ergonomics.
Lehto, M.R. 1992. Designing Warning Signs and Warning Labels: Part II Scientific Basis for Initial Guidelines. International Journal of Industrial Ergonomics (10)1-2:115-138.
Lehto, M.R., and Miller, J.M. 1986. Warnings Volume I: Fundamentals, Design, and Evaluation Methodologies. Fuller Technical Publications, Ann Arbor.
Lehto, M.R. 1992. Designing Warning Signs and Warning Labels: Part I Guidelines for the Practitioner. International Journal of Industrial Ergonomics (10)1-2:105-113.
Leonard, S.D.; Karnes, E.W.; Otani, H.; and Hastins, DL 1987. Effects of Information Content on Warning Signals. Trends in Ergonomic/Human Factors IV, Proceedings of the Annual International Industrial Ergonomics and Safety Conference, Miami, Florida, June 9-12, Asfour, S.S., Editor, Amsterdam, Netherlands, Elsevier Science Publishers B.V., Part A:493-490.
Leonard, S.; Hill, G.; and Karnes, E. 1989. Risk Perception and Use of Warnings. Proceedings of the Human Factors Society 33rd Annual Meeting:550-554.
Leonard, S.D.; Matthews, D.; and Karnes, E.W. 1993. How Does the Population Interpret Warning Signals?. Human Factors Perspective on Warnings. California: The Human Factors and Ergonomics Society:140-144.
Leonard, S.D.; Creel, E; and Karnes, E.W. 1991. Adequacy of responses to warning terms, Proceedings of the Human Factors Society 35th Annual Meeting (2):1024-1028.
Lerman, S.E., Kipen, H.M. 1990. Material Safety Data Sheet. Caveat Emptor. Archives of Internal Medicine 150(5):981-984.
Ley, P., 1995. Effectiveness of label statements for drugs and poisons, University of Sydney, Australian Health Ministers' Advisory Council, Australian Government Publishing Service, GPO Box 84, Canberra ACT 2601,ISBN 0644 45335 4, as quoted in the Inter-Organization Programme for the Sound Management of Chemicals Draft Report on the responses to the call on chemical hazard communication, December 1996.
Lirtzman, S.I. 1984. Labels, Perception, and Psychometrics, in Handbook of Chemical Industry Labeling. O'Connor, C.J. and Lirtzman, S.I. (Eds.), Noyes Publications, Park Ridge, NJ.
Luskin, J.; Somers, C.; Wooding, J.; and Levenstein, C. 1992. Teaching Health and Safety: Problems and Possibilities for Learner-Centered Training. American Journal of Industrial Medicine 22:665-670.
MacCollum, D.V. 1985. Readings in Hazard Control and Hazardous Materials. Park Ridge, IL. American Society of Safety Engineers.
Madden, M.S. Premises Hazard Signs and Products or Toxic Tort Litigation.
Magat, W., and Viscusi, K. 1992. Informational Approaches to Regulation, MIT Press: Cambridge, MA and London.
Magurno, A.B., and Wogalter, M.S. 1994. Behavioral Compliance With Warnings: Effects of Stress and Placement. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting:826-830.
Main, B.; Rhodes, P.R.; and Frantz, P. 1994. Are Current Product Labeling Systems Effective?. NFPA Journal (88)1:71-76.
Maines, John, 1992. Who Reads the Warning Label? American Demographics September:13-14.
Marsick, D.J., and Byrd, III, D.M. 1990. Resources for Material Safety Data Sheet (MSDS) Preparation. Fundamental & Applied Toxicology 15(1):1-5.
McCarthy, R.L.; Ayres, T. J.; Wood, C.T.; and Robinson J.N. 1995. Risk and effectiveness criteria for using on-product warnings. Ergonomics Vol. 38(11):2164-2175.
McCarthy, R.; Finnegan, J.; Krumm-Scott, S., and McCarthy, G. Failure Analysis Associates. 1984. Product Information Presentation, Use Behavior and Safety. Proceedings of the Human Factors Society, 28th Annual Meeting.
McCormick, E.J., and Sanders, M.S. 1982. Human Factors Engineering and Design, McGraw-Hill, New York.
McLaughlin, G.H. May 1969. "SMOG Grading - New Readability Formula", Journal of Reading.
McNeely, E. 1990. An Organizational Study of Hazard Communication. The Health Provider Perspective. AAOHN Journal 38(4):165-173.
McQuiston, T.H., 1996. Multi-program Evaluation: A Descriptive Review.
Miller, J.M., Lehto, M.R., and Frantz, P. 1990. Instructions and Warnings: The Annotated Bibliography, Ann Arbor, MI, Fuller Technical Publications.
Miller, H.G., 1986. Consistent, Accurate Data Needed for Preparing the MSDS. Occupational Health and Safety 55(4):31-34.
Montante, W.M. 1996. Effective Training: the Missing Links. Professional Safety 41(1):32-34.
Moore, M.G. 1991. Product Warning Effectiveness: Perception Versus Reality. Professional Safety. American Society of Safety Engineers. (April):21-24
Mueller, William, 1991. Who Reads the Label?. American Demographics (January):37-40.
National Institutes on Alcohol Abuse and Alcoholism (NIAAA). 1987. Review of the Research Literature on the Effects of Health Warning Labels prepared by Macro Systems Inc., (June).
National Institute for Occupational Safety and Health (NIOSH). 1997. International Chemical Safety Cards: An International Programme on Chemical Safety Project. [online] Available: NIOSH website: http://www.cdc.gov/niosh/homepage.html. May 20, 1997.
NBS Building Science Series 141, Collins, Belinda, 1982. The Development and Evaluation of Effective Symbol Signs.
Nelson, T.M. 1976. Hazard Analysis on Contact Adhesive Fires. U.S. Consumer Product Safety Commission, Bureau of Epidemiology.
Niemeier, R. 1997. NIOSH, Education and Information Division. Personal Communication with Bruce Lippy. ( May 20, 1995).
Nore, L.G., December 1990. Literacy and the Right-to-Know. At the Centre.
Oldenburg, Don, 1992. Words of Warning: Taking a New Look at Health Advisories. Washington Post. (March 17):B5.
Otsubo, S.M. 1988. A Behavioral Study of Warning Labels for Consumer Products: Perceived Danger and Use of Pictographs. Proceedings of the Human Factors Society 32nd Annual Meeting:536-540.
O'Connor, C.J. and Lirtzman, S.I. (Eds.), 1984. Handbook of Chemical Industry Labeling. Noyes Publications, Park Ridge, NJ.
Parker Brown, M., and Nguyen-Scott, N. 1992. Evaluating a Training-for-Action Job Health and Safety Program. American Journal of Industrial Medicine 22:739-749.
Patterson, M.R. 1987. The Hazards of Illiteracy in the Workplace. Professional Safety 32(11):41-45.
Phillips, C. C. 1997. The Efficacy of Material Safety Data Sheets and Worker Acceptability. Dissertation presented for the Doctor of Philosophy Degree at the University of Tennessee at Knoxville, (May).
Pollack-Nelson, C. 1991. Estimated Effectiveness of Warning Labels. U.S. Consumer Product Safety Commission, Division of Human Factors, (March).
Polzella, D.J.; Gravelle, M.D.; and Klauer, K.M., 1992. Perceived Effectiveness of Danger Signs: A Multivariate Analysis. Proceedings of the Human Factors and Ergonomics Society 36th Annual Meeting:931-934.
Powers, M.; Ruttenberg, R.; and Weinstock, D. The Railway Workers' Hazardous Materials Training Programs, 1991 - 1995, A Review of Evaluation Results.
Printing Industries of America. August 13, 1990. Comments on the OSHA Hazard Communications Standard. Docket H-022G.
Public Health Service, National Institute on Alcohol Abuse and Alcoholism, 1990. Grant No. 1 R-1 AA08383-01.
Purswell, J.L.; Krenek, R.F.; and Dorris, A. 1993. Warning Effectiveness: What Do We Need to Know. Human Perspectives on Warnings. California: The Human Factors and Ergonomics Society:174-178.
Purswell, J.L.; Schlegel, R.E.; and Kejriwal, S. 1993. A Prediction Model for Consumer Behavior Regarding Product Safety. Human Factors Perspectives on Warnings:179-182
Pyrczak, F., and Roth, D.H. 1976. The Readability of Directions on Non-Description Drugs. Journal of the American Pharmaceutical Association NS 16(5):242-243, 267.
Reale, M.J. 1996. Making HazCom Labels Effective. Occupational Health & Safety (65)6:39-40.
Riley, M.W.; Cochran, D. J.; and Ballard, J.L. 1982a. An Investigation of Preferred Shapes for Warning Labels. Human Factors (24):737-742.
Riley, M.; Cochran, D.; and Ballard, J., 1982b. Designing and Evaluating Warning Labels. IEEE Transactions on Professional Communication (PC-25)3:127-130.
Robins, T.G., and Klitzman, S., 1988. Hazard Communication in a Large U.S. Manufacturing Firm: the Ecology of Health Education in the Workplace. Health Education Quarterly 15(4):451-472.
Robins, T.G.; Hugentobler, M.K.; Kaminski, M.; and Klitzman, S. 1994. A Joint Labor Management Hazard Communication Training Program: A Case Study in Worker Health and Safety Training. Occupational Medicine 9(2):135-145.
Robins, T.G.; Hugentobler, M.K.; and Kaminski, M.; and Klitzman, S. 1990. Implementation of the Federal Hazard Communication Standard: Does Training Work?. Journal of Occupational Medicine 32(11):1133-1140.
Rodriquez, M.A. 1991. What makes a warning label salient?. Proceedings of the Human Factors Society 35th Annual Meeting (2):1029-1033.
Ruttenberg, R., and Weinstock, D. 1995. Evaluation HAZMAT Transportation Awareness Training Program, September 1993 - August 1994.
Saari, J.; Bedard, S.; Dufort, V.; Hryniewiecki, J.; and Theriault, G. 1994. Successful Training Strategies to Implement a Workplace Hazardous Materials Information System. An Evaluation Study at 80 Plants. Journal of Occupational Medicine 36(5):569-574.
Samways, M. January 1988. Functionally Illiterate Worker Also Has the 'Right to Understand. Occupational Health and Safety.
Sattler, B. 1990. Implementation of the Worker Right to Know Law in Maryland: An Analysis of Contemporary Occupational Health Policies. Ph.D. diss., Johns Hopkins University.
Sell, R.G. 1977. What Does Safety Propaganda Do for Safety? A Review. Applied Ergonomics December:203-214.
Sherer, M., and Rogers, R.W. 1984. The role of vivid information in fear appeals and attitude change. Journal of Research in Personality. 18:321-334.
Silver, N.C., and Wogalter, M.S. 1993. Broadening the Range of Signal Words. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:186-190.
Silver, N.; Leonard, D.; Ponsi, K.; and Wogalter, M. 1991b. Warnings and Purchase Intentions for Pest-Control Products. Forensic Reports (4):17-33.
Silver, N.C.; Kline, P.B.; and Braun, C.C. 1994. Type Form Variables: Differences in Perceived Readability and Perceived Hazardousness". Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting:821-825.
Silver, N.C., and Wogalter, M. 1991a. Pest-control Products: Hazard Perception, Product Type and Label Characteristics. Interface 1991, Human Factors Society, Santa Monica, CA:106-110.
Silver, N.; Leonard, D.; Ponsi, K.; and Wogalter, M. 1991b. Warnings and Purchase Intentions for Pest-Control Products. Forensic Reports (4):17-33.
Slatin, C., and Wooding, J., 1994. Evaluating Participatory Education, Technical Knowledge, The University and Worker Trainer.
Smith, S.L. 1979. Letter Size and Legibility. Human Factors 21(6):661-670.
Solomon, C.J. 1986. Hazard communication and Right-to-Know: Passion, Progress, Practice. AAOHN Journal 34(6):264-268.
Solomon, C.J. 1988. Understanding and Using the MSDS. Material Safety Data Sheets. AAOHN Journal (9):376-379.
Szudy, E., and Arroyo, M.G. 1994. The Right to Understand: Linking Literacy to Health and Safety Training.
Tompkins, N.C. 1991. Labor-supported Committees Advocate Workers' Right to Understand an MSDS. Occupational Health and Safety 60(7):23-25.
U.S. EPA, 1987. Pesticide Label Criteria and Recommendations - Draft Report, Prepared by ICF, Inc. and SHR Communications, October.
U.S. EPA, 1989. Environmental Labeling in the United States: Background Research, Issues, and Recommendations - Draft Report. Prepared by Applied Decision Analysis, Inc., (December 5).
U.S. EPA, 1986. Pesticide Label Utility Project Draft Report, Office of Pesticide Programs, (April 11th version).
U.S. EPA, 1987. Pesticide Label Criteria and Recommendations - Draft Report, U.S. EPA, 1986. Pesticide Label Utility Project Draft Report, Office of Pesticide Programs,
(April 11th version). Prepared by ICF, Inc. and SHR Communications, October.
U.S. EPA 1993. Status Report on the Use of Environmental Labels Worldwide, prepared by Abt Associates, Inc. EPA 742-R-9-93-001, (September).
United Nations Environmental Programme, Industry and Environment Office, 1991. Global Environmental Labelling: Invitational Expert Seminar, Lesvos, Greece, 24-25 September 1991, Working Group on Policies, Strategies and Instruments of the UNEP/IEO Cleaner Production Programme.
University of Michigan Evaluation Group, 1996. Summary Progress Report: Highlights of an Evaluation of the UAW Hazardous Materials Training Project.
US GAO, 1991, Report to Congressional Requesters: Occupational Safety and Health, OSHA Action Needed to Improve Compliance with Hazard Communication Standard.
(GAO/HRD-92-8) OSHA Hazard Communication Standard.
Venema, A. 1989. Product Information for the Prevention of Accidents in the Home and During Leisure Activities. Institute for Consumer Research, SWOKA, Research report 69, Leiden, The Netherlands, 1989.
Viscusi, W.K., and Magat, W.A. 1987. Learning About Risk: Consumer and Worker Responses to Hazard Information. Cambridge, MA. Harvard University Press.
Weidner, L., and Gotsch, A. Worker Health and Safety Training Assessing Impact Through Interview.
Westinghouse Electric Corporation. 1981. Product Safety Label Handbook. Westinghouse Printing Division, Trafford, PA.
West, A.S. 1991. Worker Hazard Communications, the Regulatory Framework. Plant/Operations Progress: 10(1), 13-16.
Wogalter, M.S.; Kalsher, M.J.; and Racicot, B.M. 1992. The Influence of Location and Pictorials on Behavioral Compliance to Warnings. Proceedings of the Human Factors Society:1029-1033.
Wogalter, M.S., and Silver, N.C. 1990. Arousal Strength of Signal Words. Forensic Reports:407-420.
Wogalter, M.S.; Racicot, B.M.; Kalsher, M.J.; and Simpson, S.N. 1993. Behavioral Compliance to Personalized Warning Signs and the Role of Perceived Relevance. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:239-243.
Wogalter, M.S., and Barlow, T. 1990. Injury severity and likelihood in warnings. Proceedings of the Human Factors Society 34th Annual Meeting:580-583.
Wogalter, M.S., Allison, S.T., McKenna, N.A., 1989. Effects of Cost and Social Influence on Warning Compliance. Human Factors (31)2:133-140.
Wogalter, M. S.; Fontenelle, G.A.; and Laughery, K.R. 1993. Behavioral Effectiveness of Warnings. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:220-224.
Wogalter, M.S., Jarrad S.W., and Simpson S.N. 1994. Influence of Warning Labels Signal Words on Perceived Hazard Level. Human Factors (36)3:547-556.
Wogalter, M.S.; Brelsford, J.W.; Desaulniers, D.R..; and Laughery, K.R. 1991. Consumer Product Warnings: The Role of Hazard Perception. Journal of Safety Research (22)2:71-82.
Wogalter, M.S., and Silver, N.C., 1995. Warning Signal Words: Connoted Strength and Understandability by Children, Elders, and Non-native English Speakers. Ergonomics 38(11):2188-2206.
Wogalter, M.S.; Godfrey, S.S.; Fontenelle, G.A.; Desaulniers, D.R.; Rothstein, P.R.; and Laughery, K.R., 1987. Effectiveness of Warnings. Human Factors (29)5:599-612.
Wogalter, M.S.; Jarrad, S.W.; and Simpson, S. N., 1992. Effects of Warning Signal Words on Consumer-Product Hazard Perceptions. Proceedings of the Human Factors Society, 36th annual Meeting:935-939.
Wogalter, M.S.; Desaulniers, D.R.; and Godfrey, S. S. 1993. Perceived Effectiveness of Environmental Warnings. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:215-219.
Work, D.R. 1990. Improving Drug Labels with Pictograms. International Pharmacy Journal 4:153-157.
Wright, P. 1981. Five Skills Technical Writers Need. IEEE Transactions on Professional Communication 24(1):10-16.
Wright, P. 1979. Concrete Action Plans in TV Messages to Increase Reading of Drug Warnings. Journal of Consumer Research 6:256-269.
Wright, P. 1981. 'The Instructions Clearly State...' Can't People Read?. Applied Ergonomics 12(3):131-141.
Young, S.L. 1991. Increasing the noticeability of warnings. Effects of pictorial, color, signal icon and border. Proceedings of the Human Factors Society 35th Annual Meeting 1:580-584.
Young, S.L., and Wogalter, M.S. 1990. Comprehension and Memory of Instruction Manual Warnings: Conspicuous Print and Pictorial Icons. Human Factors (32)6:637-649.
Young, S.L. 1993. "Increasing the Noticeability of Warning Effects of Pictorial, Color, Signal Icon and Border. Human Factors Perspectives on Warnings. California: The Human Factors and Ergonomics Society:249-253.