<< Back to State Occupational Safety and Health Plans

Guidance for FY 2009 Enhanced FAME Reports

U.S. Department of Labor Occupational Safety and Health Administration
Washington, D.C. 20210

Reply to the attention of:
Department of Labor Seal
November 24, 2009    
     
MEMORANDUM FOR: REGIONAL ADMINISTRATORS
STATE PLAN DESIGNEES
 
     
FROM: /s/ JORDAN BARAB
Acting Assistant Secretary
 
     
SUBJECT: Enhanced FAME Reports and Baseline Special Evaluations  
     
Last month I testified before the House Committee on Education and Labor about the recent Special Study that OSHA conducted in Nevada, as well as our plans for increasing our oversight and conducting a baseline special evaluation in all other State Plans. During that hearing we received clear direction that OSHA’s monitoring of the State Plans needs to change from our current process. I made the commitment, even before this hearing, that this would happen. I hope you will join me in viewing this as a unique opportunity to work together to identify and mutually resolve problems, as well as to identify effective performance which may achieve the goals of the national OSHA program through means that may differ from the Federal.

Attached you will find guidance for preparation of an Enhanced Federal Annual Monitoring and Evaluation (FAME) report incorporating Baseline Special Evaluations for each State Plan (excluding Illinois and Nevada). These evaluations will include a Case File Review component, review of State performance statistics in comparison to Federal performance, and will look at areas not recently reviewed, such as the operation and outcome of State contested case review procedures. These Baseline Special Evaluations will focus on FY 2009 and should begin as soon as possible, but no later than the second quarter of FY 2010. The findings will be incorporated in the FY 2009 FAME reports. In order to ensure that we are looking at each State consistently, this guidance includes a framework for gathering and reporting your findings; however, each Region may adjust the presentation of information to reflect the circumstances in its States. State participation in the monitoring and evaluation efforts will assure that all pertinent facts are considered and that the conclusions are accurate and fair. States are encouraged to include elements from the Enhanced FAME format in their State Occupational Safety and Health Annual Report (SOAR) as well. Regions should consider using resources on their monitoring teams in addition to those already assigned to State Plan monitoring, both for their expertise and for a new perspective. Regions may choose to share resources, as appropriate.

The draft Enhanced FAME report must be shared with the State for review and comment, revised as appropriate, and submitted, together with a draft transmittal letter to the State, to the Directorate of Cooperative and State Programs (DCSP), Office of State Programs for coordination of National Office review and approval prior to issuance. Our goal is to have all final Enhanced FAME Reports for FY 2009 issued by April 30, 2010, although requests for extensions will be considered by the Assistant Secretary. All reports, and each State’s formal response to the report (due 30 days after issuance), will be made available publically.

If you have any questions or concerns about this guidance, please let me or Steven Witt know. The OSHSPA Board and the Federal State Plan Steering Committee will discuss the Enhanced FAME at their upcoming meeting on December 2, 2009 in Salt Lake City, Utah. We are hopeful that our mutual implementation of this guidance will help us identify the changes that need to be made to our State Plan monitoring procedures.


Attachment


FY 2009 State Plan Enhanced FAME Reports
To Include Baseline Special Evaluations


Background

As a result of an agreement reached with the Occupational Safety and Health State Plan Association (OSHSPA) several years ago, OSHA’s current State plan monitoring system focuses on achievement of State strategic goals and performance on a limited number of statistical activities measures. The results of a Special Study of the Nevada State Plan recently completed by a team of Federal monitors with the cooperation of the State demonstrate that this approach may not capture and address performance problems that may exist in some States. In addition, the recent Federal Annual Monitoring and Evaluation (FAME) reports do not always present complete pictures of the State programs. While the current monitoring system does not preclude on-site monitoring or other in-depth evaluations of a State plan, the focus on strategic goals and activities measures has in some cases led to limiting monitoring to those areas.

Interim Monitoring Guidance was issued on August 31, 2009, to reiterate the available monitoring options and suggest improvements in preparation of the FY 2009 FAME reports. OSHA has announced to the States and the Congress, following the issuance of the Report on the Special Study done in Nevada, that baseline Special Evaluation Studies would be done for each State Plan.

FY 2009 Enhanced FAME Reports – New Guidance

As part of the preparation of the FY 2009 FAME reports, each Region will conduct a Special Evaluation Study in each of its State Plan States (excluding Illinois and Nevada) to better assess the current performance of each State program and identify any structural or performance issues of concern. The findings of these baseline Special Studies will be incorporated into the FY 2009 FAME report when issued. Guidance for the content and format for the expanded report based on the current FAME format in Chapter 9 of the State Plan Policies and Procedures Manual (CSP 01-00-002) is attached. These studies will also aid OSHA in the future revision of the monitoring system by identifying the most effective monitoring techniques.

Each Region will design and conduct a baseline Special Evaluation for each of its State Plans during the first and second quarters of FY 2010. State cooperation and participation in the Special Study is to be encouraged. Each Special Evaluation Study will, in addition to looking at broad statistical measures from the State Plan Activities Measures (SAMM and SIR reports) and IMIS management reports, focus on known areas of concern in the State which may be performance and/or structurally related. Any performance review should look at underlying policies, procedures, regulations and statutory authority to determine not only State conformance with its procedures but the "at least as effectiveness" of those procedures. Studies should focus on enforcement effectiveness, including the handling of major accidents, fatalities and complaints, targeting, hazard recognition, violation classification, citation and penalty issuance, employee involvement, and training of compliance officers. The outcome of the State’s contested case review process, including settlement of cases and the procedures and findings of the review authority and their transparency must be reviewed. In addition, issues receiving public attention, CASPAs, legislative or regulatory changes or initiatives, and funding and staffing changes, must be addressed. Interviews with State inspectors and supervisors are encouraged. Stakeholders should be contacted and their views on State Plan performance sought. The State’s response to recommendations contained in previous CASPAs, Annual Evaluations and other reports should be reviewed.

Case File Reviews

Each Special Study must include a Case File Review component directed either at specific known areas of concern, or a statistically valid random sample of enforcement cases, as determined by the Regional Administrator. In designing the Special Study for each State Plan, the procedures used for Federal field audits as set out in Management and Accountability Program Directive (EAA-01-000-003), and the more detailed on-site monitoring procedures contained in the 1994 State Plan Monitoring Manual may be used as guidance. (The 1994 Manual will be posted on the "O" drive.)

The cases reviewed should be from FY 2009. If monitoring that included a Federal audit with case file reviews has already been conducted for FY 2009, those findings may be used as part of this report, and those specific topic areas do not need to be reviewed again. Additional focused case file reviews covering areas not previously addressed may be necessary. If Accompanied Visits of State CSHO’s are the only means available to evaluate a case file or other monitoring finding, they may be done on a very limited basis.

The findings from these Special Studies will be incorporated into the FY 2009 Enhanced FAME Report. As a result, the scope of that report will go beyond the current requirements found in Chapter 9 of the State Plan Policy and Procedures Manual (SPPM) to include a description and the findings of the study with concrete recommendations for correction, and a timeframe for expected completion. The States are encouraged to expand the focus of their State Occupational Safety and Health Annual Report (SOAR) to include a broader assessment of their enforcement, going beyond the goals in their Annual Performance Plan, and including the results of their internal audits.

Consultation

Regions need not consolidate their evaluation of the 21(d) Consultation Program in the FY 2009 Enhanced FAME. The Consultation evaluation report may be issued separately.

Study Teams

The Special Studies shall be conducted using a team approach. Each team should consist of members with an adequate level of expertise to thoroughly review and analyze the State plan’s performance. Ideally, that expertise would include experience in the areas of field enforcement, case file review, whistleblower, CSHO training, cooperative programs and outreach. A team of four (4) was used for the Nevada Special Study. Regions may want to consider "sharing" resources in order to obtain the most effective and consistent reviews. Other Regional staff may be helpful for specific areas of review, e.g., members of the Regional audit team, experienced field CSHOs/Team Leaders, Whistleblower Investigators, etc.

Due Date

The goal is to issue the final Enhanced FAME Reports for FY 2009 by April 30, 2010. Reasonable extensions may be requested for good cause and may be granted, after review by the Assistant Secretary. In addition to providing the State an opportunity to review and comment on the draft report, reports must be submitted to the National Office for review and approval prior to issuance.


Appendix A
General Guidance for Preparation of the State Plan
FY 2009 Enhanced Fame Report
  • The expanded outline provided in the attachment incorporates new information that should be included as part of the Baseline Special Evaluation Study along with existing requirements from the State Plan Policies and Procedures Manual (SPPM). General requirements for each Enhanced FAME report are identified, as well as measures for additional focus areas that the Region may decide to include as part of the Special Study. Regions may modify the format of the report and topics addressed as appropriate to reflect the issues in each State Plan.
     
  • This report will likely be read by a diverse audience, including members of the public. Keep this in mind when introducing technical issues or specific policy documents, and remember to spell out all acronyms.
     
  • Names of Federal and State personnel (other than program officials) should not be specifically mentioned, nor should names of companies, company representatives, or employees from inspection files.
     
  • Look for root causes when identifying issues. For example, if the case file review indicates that hazards are being cited using the wrong standard, the root cause may be a lack of initial training, and/or that supervisors are not given adequate time to perform case file review.
     
  • The evaluation, with the exception of the review of State standards that are different from Federal standards, should be based to the greatest extent possible on the analysis of available data. In addition to evaluating the State's performance against either the national or State standards or guides, as identified in the State Activity Mandated Measures (SAMM) report, States will also be compared to nationwide Federal performance statistics. FY 2009 IMIS data (standard INSP and ENFC reports) for both State and Federal will be posted on the "O" drive and should be used to the extent possible as the primary source of standardized State/Federal data comparisons, rather than generating new IMIS reports. A State by State table extracting the major performance measures in comparison to Federal performance will also be available.
     
  • The Region must provide a general conclusion regarding the acceptability of State performance on mandated activities and describe in detail those elements for which further analysis beyond review of the SAMM was conducted.
     
  • The State will be provided an opportunity to comment on the draft report and their comments incorporated in the final report, as appropriate, to provide clarifications, correct errors or misstatements, etc. The formal transmittal of the Enhanced FAME Report to the State must include a requirement for a written response to the recommendations and submission of a corrective action plan within 30 days. Each Enhanced FAME report and transmittal letter must be cleared by the National Office prior to issuance.
     
Appendix B
FY 2009 Enhanced FAME Report
Suggested Format and Content

(to incorporate findings of Special Baseline Evaluation Studies)


Executive Summary

This section provides an overview of the State's progress towards achieving its performance goals, including results accomplished, and summarizes any significant issues or changes related to the annual performance plan, legislation, enforcement activities, funding, events that receive public/media attention, or CASPAs in the last year and during the period covered by the Special Study that have significantly impacted the State plan. The Executive Summary must also indicate whether the State continues to meet its State plan requirements. The Executive Summary should not include all of the details of the State program's performance, which are to be included in the body of the report, but should include a summary listing of the significant findings/general conclusions in the report. The purpose of this summary is to provide an overview of what is working well and to highlight those areas where corrective action is necessary. A list of significant recommendations also should be included. Recommendations should be specific as to outcome, with time frames for response. Related recommendations should be grouped together, rather than repeating similar recommendations several times. A comprehensive listing of all recommendations should be included as a separate section of the report.

Introduction

This section will include a brief profile of the State plan, including the department or division within the organizational structure of the State government in which the program is operated and the identity of the State Plan Designee and manager of the State program. It must also include a brief description of the size of the program in terms of the number of full- and part-time staff and the State program budget. In addition, this section must include a brief description of the program and its major unique features or significant differences from the Federal program. Describe how onsite consultation is provided in the state. If private sector consultation is provided under 23(g), it must be assessed in the report.

Summary of Recommendations and State Actions from the FY 2008 FAME

Recommendations may be grouped together where appropriate, and indicate what actions the State has taken to address the issues raised or improve performance. Where the State did not take any corrective action, or where the action taken did not follow the Region’s recommendations, discuss why corrective action did not occur and whether the issue was identified in the current evaluation.

Major New Issues

This section should discuss any new issues during the evaluation period that have had significant impact on the program and program performance. This section expands on the discussion of issues summarized in the Executive Summary. These may include, but are not limited to changes in the annual performance plan, legislation, enforcement activities, funding, events that receive public/media attention, or Complaints About State Plan Administration (CASPAs). Include all issues that arose during FY 2009, as well as those identified during the Special Study. If an issue relates to performance and/or a recommendation, reference the appropriate section above or below.

Assessment of State Performance 
  1. Assessment of State Progress in Achieving Annual Performance Goals. This section provides an assessment of whether the State has made sufficient progress toward achieving its annual performance goals and is thus on track to accomplish its five-year strategic goals. The Region will review the State's performance results, including activity, intermediate outcome, and outcome measures. The Region should base its analysis on the data provided by the State in the SOAR.

    The Region's assessment must address the status of all State goals with particular emphasis on the State's annual performance goals that are not on target for successful completion and identify recommendations for improvement.

    The Region's assessment must also include a review of the actions the State took in response to previous OSHA recommendations on the implementation and achievement of the State's strategic and performance goals or measurement system.
     
  2. Assessment of State Performance on Mandated and Other Related Activities. This section will address both State performance on the required (mandated) program items and related enforcement activities:
     
    • For each category below, briefly describe state policy, and whether it is identical to or different from Federal policy.
       
    • Use all available information – SAMM, SIR, other IMIS data, case file reviews, CASPAs which raised program concerns, interviews, and any other relevant information.
       
    • Compare the State Plan’s performance to Federal performance, using FY 2009 closeout data from standard reports wherever possible.
       
    • When performance problems are identified that suggest the need for staff training, the effectiveness of the State’s overall CSHO training program should also be assessed.
       
    • Include positive and negative findings.
       
    • The National Office will post standard performance measures with State by State comparisons to Federal data on the "O"drive; State and Federal data is available in INSP and ENFST reports; FY 2009 closeout data will be posted. Include a general discussion of the State’s data as compared with last year’s, in comparison to Federal performance, and in comparison with the State’s performance goals.
       
    1. Enforcement (including complaint investigations, right of entry, first instance sanctions, abatement of violations, and employee involvement)

      Complaints
       
      • Timeliness of State response and notifications to complainant
      • Appropriateness of State response
      Fatalities
       
      • Contact and involvement of families of victims, including notification of enforcement action
      • Appropriate use of "no inspection" or "no jurisdiction"
      • Timeliness of notification and opening conference
      Targeting/Inspections
       
      • Goals, emphasis programs, participation in NEPs
      • Violations per inspection
      • Number and percentage of Serious, Willful, Repeat Violations
      • Hazard identification
      • Violation classification; grouping
      • In-compliance inspections
      Employee and Union Involvement
       
      • Adequate policies and procedures addressing employee involvement in inspection process
      • Employee interviews; adequacy of documentation of employee interviews
      • Union or other labor representative participation in opening/closing conferences, walkaround, settlement and contests
      Citations and Penalties
       
      • Adequate evidence to support violation
      • Appropriateness of violation classification
      • All apparent violations cited
      • Average serious penalties
      • Appropriateness of penalties
      • Appropriate use of willful and repeated violation classifications
      • Appropriate use and level of review for Notice of Violations, if applicable
      Abatement
       
      • Appropriate abatement periods
      • Adequate verification/evidence of abatement
      • Follow-up inspections when indicated
      Review procedures

      Informal Conferences
       
      • Adequacy of procedures and whether followed
      • Violations vacated and/or reclassified
      • Penalties reduced/retained; initial vs. final penalties
      • Appropriateness of changes made and reasons for any changes
      • Reasons for modifications; whether changes are the result of problems with original citations
      • Patterns of settlements
      Formal review of citations (Independent Review Boards/Commissions/Other)
       
      • Adequacy of State defense
      • Violations vacated and/or reclassified
      • Penalties reduced/retained; initial vs. final penalties
      • Whether changes due to problems with original citations
      • Appeal of adverse decisions by State
      • Transparency of process; availability of decisions to public
      • Quality of decisions; consistency with Federal precedence
      • Procedural issues (timely hearings; full-time members/staff)
      Public employee program (address in above sections when appropriate)
       
      • Percentage of total inspections in the public sector
      • If no monetary penalties are imposed, adequacy/effect of State’s alternatives
      • Any other issues specific to public sector
      Information management
       
      • State use of IMIS reports for program management
      • Any issues concerning accuracy and integrity of data
      • Timeliness of data entry
      • Timeliness of updates after case settlement, penalty payment, abatement, etc.
      BLS Rates (Illness, injury and fatality)
       
      • Review the last five years of State-specific rates to determine trends
      • Compare to targeting and special emphasis programs to determine correlation
      • Public Sector rates
         
    2. Standards and Plan Changes

      Standards Adoption
       
      • Standards with adoption due during evaluation period and any delinquent from earlier; timely notice of intent provided
      • Adopted on time (within 6 months of Federal promulgation); submitted on time with comparison document if "different"
      • Status of those not adopted on time
      • Different State standards and State-initiated standards
      • Any other standards issues (effectiveness issues, challenges, initiatives, etc.)
      • Standards interpretations
      • Standards adoption process
      Federal Program/State Initiated Changes
       
      • Non-standards Federal Program Changes due during evaluation period and any delinquent from earlier; timely notice of intent
      • Adopted and submitted on time with comparison document if "different"
      • Adoption of revised FOM; submission of detailed comparison identifying different State policies
      • Status of those not adopted on time
      • State-initiated changes
      • Timely submission of legislative and regulatory changes
      • Submission of comparison documents for different policies
      • Any effectiveness issues
      • Any other initiatives being developed if not discussed elsewhere
         
    3. Variances
       
      • Number of permanent and temporary variances requested and granted
      • Variances entered into ATS
      • Adequacy of alternate protection afforded to employees (permanent variance)
      • Temporary variances granted only prior to effective date of standard/provision
         
    4. Consultation activities (for Kentucky, Puerto Rico, and Washington only). (See the Federal Program Change section in CSP 02-00-002 for fundamental program requirements relevant to 23(g) private sector consultation activities)
       
      • Number of visits
      • Percentage of visits with hazards abated within the required time frame
      • Conformance with 1908 and CPPM requirements as detailed in 21(d) Cooperative Agreements
      • Safety and Health Achievement Recognition Program and impact on enforcement
         
    5. Discrimination Program
       
      • Appropriateness of State findings and decisions (would OSHA likely have reached same conclusions)
      • Timeliness of investigation and response
      • Outcome of referrals/deferrals
         
    6. CASPAs. In addition to a general discussion of CASPAs, findings that relate to any of the topic areas above should be included in those discussions.
       
      • Number of CASPAs received by Region
      • Number and percentage of CASPAs that were determined to have valid allegations
      • CASPAs where State followed its procedures, but procedures were inadequate, improper
      • Timeliness and appropriateness of written response from State
      • Any concerns regarding State’s follow-up actions in response to findings
      • Any CASPAs not discussed elsewhere
         
    7. Voluntary Compliance Programs
       
      • Availability of written policies and procedures for voluntary and cooperative programs, as well as State Plan changes
      • Appropriateness of State programs that provide exemptions and other enforcement incentives
      • Review and assessment of State’s Voluntary Protection Program
         
    8. Program Administration. (In addition to a general discussion about program administration, findings that relate to any of the topic areas above should be included in those discussions.)
       
      • Impact of State funding or other fiscal issues
      • For States with final approval, ability to meet compliance staffing benchmarks – allocated vs. on-board; reduced staffing issues
      • Furloughs, office closures or other changes in services
      • Assessment of CSHO Training Program; response to revised Federal directive
         
    9. Appendices. The FAME report must include at least the following two appendices:
       
      • State OSHA Annual Report (SOAR)
      • Data printout of the end-of-year State Activity Mandated Measures (SAMM) report
      • Data printout of the end-of-year State Information Report (SIR)
Resources (to be stored on "O" drive at: O:\DCSP\Office of State Programs (OSP)\Monitoring Tools)

Note: Regions are encouraged to post Monitoring and Special Study tools and best practices to the "O" drive.

 
Appendix C - Performance Data

  • FY 2009 closeout data – INSP and ENFST reports (State and Federal)
  • State Plan FY 2009 Selected Enforcement Measures by State in comparison to Federal performance
  • BLS Rates, by State, for 2004-2008
     
Appendix D – Monitoring Tools 

  • Case File Review Tracking Tool - Under development (Bonita Winingham)
  • State Annual Comparison of Enforcement Data as of (date).xls (presentation graphic) Region I (Mary Shannon)
  • State Annual Comparison of Inspection Data as of (date).xls (presentation graphic) Region I (Mary Shannon)
     
Appendix E – Procedures for Performing Case File Reviews, Random Sampling, etc.

  • 1994 Monitoring Manual


Page last updated: 02/24/2010