EAL

MHFS: Food Safety (2012-2013)

Citation:

Study Design:

Class:

- Click here for explanation of classification scheme.

Quality Rating:

Research Purpose:

The objective of this study was to develop and validate a set of self-reported behavior questions that could be used with confidence as a measure of actual behavior on self-report questionnaires designed to evaluate food safety behaviors and to identify, through direct observation and interviews, those areas where education and skill development are needed to encourage performance of recommended food-handling behaviors.

Inclusion Criteria:

Nationally recognized food safety experts: Food microbiologists, epidemiologists, food safety educators and food safety policy makers
Graduates of a nutrition education program completed four food preparation tasks while being observed and videotaped.

Exclusion Criteria:

Not described.

Description of Study Protocol:

Recruitment

Food-safety experts were recruited from nationwide listings of food microbiologists, epidemiologists, food safety educators and food safety policy makers who completed the four-round Delphi process.

Design

Diagnostic, reliability or validity study.

Statistical Analysis

To assess the reliability of individual questions, paired T-tests and paired percentage agreement in question responses between test and re-test (without education intervention) were calculated
Individual questions were considered reliable if the percentage agreement was at least 75% and the P-value over 0.05
Overall questionnaire stability over time was assessed using the paired T-test and Cronbachs coefficient alpha
Criterion validity was established by comparing questionnaire responses to observed behavior and interview responses.

Data Collection Summary:

Timing of Measurements

A web-based Delphi process was used to determine which home food-handling behaviors were most important to teach consumers, based on incidence and severity data of food-borne illnesses associated with common pathogens
During a kitchen activity session, graduates of a nutrition education program completed four food preparation tasks while being observed and videotaped
Individuals also participated in an in-depth interview to validate behaviors that could not be observed during the food preparation activity.

Dependent Variables

The Delphi process yielded a prioritized and validated list of 29 safe food-handling behaviors, rank-ordered within five pathogen control factors:
- Practice personal hygiene
- Avoid cross-contamination
- Cook foods adequately
- Keep foods at safe temperatures
- Avoid foods from unsafe sources.
Eight food safety experts, an evaluation expert and the research team participated in a face-to-face conference to develop one or more behavioral questions related to each of the 29 behaviors. The questions developed were reviewed for content validity by the experts, research team and other professionals.
In the revised form, the questionnaire contained 52 questions with one or more questions addressing each of the 29 food safety behaviors. 45 questions addressed food safety behaviors of concern to the general public and seven questions targeted food safety issues specific to pregnant women.
The questionnaire contained a variety of question types, including 20 statements with five-point Likert scale responses, 31 dichotomous response questions and one multiple-choice question.

Independent Variables

24 food safety experts recruited from nationwide listings of food microbiologists, epidemiologists, food safety educators and food safety policy makers completed the four-round Delphi process
Graduates of a nutrition education program completed four food preparation tasks while being observed and videotaped.

Description of Actual Data Sample:

Initial N

79 EFNEP and FSNEP class participants.

Attrition (Final N)

70 EFNEP and FSNEP class participants (64 women and six men).

Age

The majority were between the ages of 20 and 39

40 through 49 years: 20%
Over 50: 11%
Under 20: 6%.

Ethnicity

White: 58%
African-American: 14%
Hispanic: 13%
Other ethnicities: 15%

Other Relevant Demographics

Not reported.

Anthropometrics

Not reported.

Location

Colorado State University, Fort Collins, CO.

Summary of Results:

Key Findings

The 20 participants in the Test-Retest Group were similar in demographic characteristics to the Criterion Validity Study Group. Based on test-results, 41 of the 52 questions met the reliability criteria P>0.05 and at least 75% agreement.
Overall the questionnaire was considered stable over time (R=0.83 and P=0.52)
Behaviors from three pathogen control factors [practice personal hygiene (N=1), avoid cross-contamination (N=12) and cook foods adequately (N=4)] were observed during the kitchen activity
Of the original 41 questions considered reliable over time, 28 (68%) met the criterion for validity (at least 70% agreement between observed and interviewed responses and self-reported responses). Among the 11 items on the questionnaire that were observable, three did not meet the reliability criterion.
Observation assessments revealed that hand-washing was more likely to be performed prior to beginning food preparation than between working with raw meats and fresh produce
Errors in methods of washing hands, utensils and preparation surfaces between food preparation tasks were common
Most participants did not use thermometers to evaluate doneness, but still cooked to safe internal temperatures.

Author Conclusion:

The results from this study provide a tool that educators can use to evaluate food safety education programs and will help guide the development of more effective food safety education programs targeting needed improvements in behavioral skills.

Funding Source:

Government:

US Department of Agriculture Grant #99-35201-8126, Cooperative State Research, Education, and Extension Services

Reviewer Comments:

Inclusion and exclusion criteria for experts and students was not described.

Quality Criteria Checklist: Primary Research
Relevance Questions
	1.	Would implementing the studied intervention or procedure (if found successful) result in improved outcomes for the patients/clients/population group? (Not Applicable for some epidemiological studies)	N/A
	2.	Did the authors study an outcome (dependent variable) or topic that the patients/clients/population group would care about?	Yes
	3.	Is the focus of the intervention or procedure (independent variable) or topic of study a common issue of concern to dieteticspractice?	Yes
	4.	Is the intervention or procedure feasible? (NA for some epidemiological studies)	N/A

Validity Questions
1.	Was the research question clearly stated?		Yes
	1.1.	Was (were) the specific intervention(s) or procedure(s) [independent variable(s)] identified?	Yes
	1.2.	Was (were) the outcome(s) [dependent variable(s)] clearly indicated?	Yes
	1.3.	Were the target population and setting specified?	Yes
2.	Was the selection of study subjects/patients free from bias?		???
	2.1.	Were inclusion/exclusion criteria specified (e.g., risk, point in disease progression, diagnostic or prognosis criteria), and with sufficient detail and without omitting criteria critical to the study?	No
	2.2.	Were criteria applied equally to all study groups?	Yes
	2.3.	Were health, demographics, and other characteristics of subjects described?	Yes
	2.4.	Were the subjects/patients a representative sample of the relevant population?	???
3.	Were study groups comparable?		Yes
	3.1.	Was the method of assigning subjects/patients to groups described and unbiased? (Method of randomization identified if RCT)	Yes
	3.2.	Were distribution of disease status, prognostic factors, and other factors (e.g., demographics) similar across study groups at baseline?	Yes
	3.3.	Were concurrent controls or comparisons used? (Concurrent preferred over historical control or comparison groups.)	Yes
	3.4.	If cohort study or cross-sectional study, were groups comparable on important confounding factors and/or were preexisting differences accounted for by using appropriate adjustments in statistical analysis?	N/A
	3.5.	If case control study, were potential confounding factors comparable for cases and controls? (If case series or trial with subjects serving as own control, this criterion is not applicable.)	N/A
	3.6.	If diagnostic test, was there an independent blind comparison with an appropriate reference standard (e.g., "gold standard")?	N/A
4.	Was method of handling withdrawals described?		Yes
	4.1.	Were follow-up methods described and the same for all groups?	Yes
	4.2.	Was the number, characteristics of withdrawals (i.e., dropouts, lost to follow up, attrition rate) and/or response rate (cross-sectional studies) described for each group? (Follow up goal for a strong study is 80%.)	Yes
	4.3.	Were all enrolled subjects/patients (in the original sample) accounted for?	Yes
	4.4.	Were reasons for withdrawals similar across groups?	Yes
	4.5.	If diagnostic test, was decision to perform reference test not dependent on results of test under study?	N/A
5.	Was blinding used to prevent introduction of bias?		N/A
	5.1.	In intervention study, were subjects, clinicians/practitioners, and investigators blinded to treatment group, as appropriate?	N/A
	5.2.	Were data collectors blinded for outcomes assessment? (If outcome is measured using an objective test, such as a lab value, this criterion is assumed to be met.)	N/A
	5.3.	In cohort study or cross-sectional study, were measurements of outcomes and risk factors blinded?	N/A
	5.4.	In case control study, was case definition explicit and case ascertainment not influenced by exposure status?	N/A
	5.5.	In diagnostic study, were test results blinded to patient history and other test results?	N/A
6.	Were intervention/therapeutic regimens/exposure factor or procedure and any comparison(s) described in detail? Were interveningfactors described?		Yes
	6.1.	In RCT or other intervention trial, were protocols described for all regimens studied?	N/A
	6.2.	In observational study, were interventions, study settings, and clinicians/provider described?	Yes
	6.3.	Was the intensity and duration of the intervention or exposure factor sufficient to produce a meaningful effect?	N/A
	6.4.	Was the amount of exposure and, if relevant, subject/patient compliance measured?	N/A
	6.5.	Were co-interventions (e.g., ancillary treatments, other therapies) described?	N/A
	6.6.	Were extra or unplanned treatments described?	N/A
	6.7.	Was the information for 6.4, 6.5, and 6.6 assessed the same way for all groups?	N/A
	6.8.	In diagnostic study, were details of test administration and replication sufficient?	N/A
7.	Were outcomes clearly defined and the measurements valid and reliable?		Yes
	7.1.	Were primary and secondary endpoints described and relevant to the question?	Yes
	7.2.	Were nutrition measures appropriate to question and outcomes of concern?	Yes
	7.3.	Was the period of follow-up long enough for important outcome(s) to occur?	Yes
	7.4.	Were the observations and measurements based on standard, valid, and reliable data collection instruments/tests/procedures?	Yes
	7.5.	Was the measurement of effect at an appropriate level of precision?	Yes
	7.6.	Were other factors accounted for (measured) that could affect outcomes?	Yes
	7.7.	Were the measurements conducted consistently across groups?	Yes
8.	Was the statistical analysis appropriate for the study design and type of outcome indicators?		Yes
	8.1.	Were statistical analyses adequately described and the results reported appropriately?	Yes
	8.2.	Were correct statistical tests used and assumptions of test not violated?	Yes
	8.3.	Were statistics reported with levels of significance and/or confidence intervals?	N/A
	8.4.	Was "intent to treat" analysis of outcomes done (and as appropriate, was there an analysis of outcomes for those maximally exposed or a dose-response analysis)?	No
	8.5.	Were adequate adjustments made for effects of confounding factors that might have affected the outcomes (e.g., multivariate analyses)?	N/A
	8.6.	Was clinical significance as well as statistical significance reported?	Yes
	8.7.	If negative findings, was a power calculation reported to address type 2 error?	N/A
9.	Are conclusions supported by results with biases and limitations taken into consideration?		Yes
	9.1.	Is there a discussion of findings?	Yes
	9.2.	Are biases and study limitations identified and discussed?	Yes
10.	Is bias due to study's funding or sponsorship unlikely?		Yes
	10.1.	Were sources of funding and investigators' affiliations described?	Yes
	10.2.	Was the study free from apparent conflict of interest?	Yes