NSCR: Quick and Easy Nutrition Screening Tools: Adult Patients in Acute and Ambulatory Care Settings (2009)

Citation:
Keller HH, Goy R, Kane SL. Validity and reliability of SCREEN II (Seniors in the community: risk evaluation for eating and nutrition, Version II). Eur J Clin Nutr. 2005 Oct; 59(10): 1149-1157.
PubMed ID: 16015256
 
Study Design:
Cross-Sectional Study
Class:
D - Click here for explanation of classification scheme.
Quality Rating:
Neutral NEUTRAL: See Quality Criteria Checklist below.
Research Purpose:

Study 1: The purpose of this study was to conduct a construct validation study that compared the index score from SCREEN II to a dietitian's rating of nutritional risk based on medical and nutritional history, dietary intake and anthropometry. An abbreviated version of SCREEN II was also developed.

Study 2: The purpose of this study was to assess the test-retest reliability of SCREEN II.

Study 3: The purpose of this study was to assess the inter-rater reliability associated with SCREEN II.

Inclusion Criteria:

Study 1

  • 55 years of age or older
  • Lived in the community (Guelph area of Ontario)
  • No cognitive impairment as determined by Standardized Mini Mental State Exam; scores greater than or equal to 26.

Study 2

  • Seniors from the community subsample from Study 1, OR
  • Seniors who lived in the community (Guelph area of Ontario).

Study 3

  • Seniors who required assistance from others to live independently in the community (Guelph area of Ontario)
  • No cognitive impairment.
Exclusion Criteria:

Study 1

  • Less than 55 years of age
  • Did not live in the community
  • Cognitive impairment.

Study 2

  • Did not participate in Study 1, OR
  • Did not live in the community.

Study 3

  • Did not require assistance from others to live independently in the community
  • Cognitive impairment.
Description of Study Protocol:

Recruitment

Study 1:

  • Participants recruited from the community in the Guelph area of Ontario via newspaper ads, posters, invitational letters and displays at various community programs and clubs
  • Participants also recruited from a geriatrician's clinic. Clinic staff identified patients who were eligible and asked them to participate.

Study 2:

  • 45 seniors from the community subsample from Study 1
  • 100 seniors from the community using displays at a recreation center, a mall and places of worship.

Study 3:

Participants were recruited from malls, recreation and day programs in the Guelph area.

Design

Study 1:

  • Geriatrician's Clinic
    • SCREEN II, weight and blood work collected at clinic visit
    • Demographic and medical questionnaire, anthropometry and 24-hour dietary recall completed in person's home (time period from clinic visit not specified)
    • Two telephone calls within a week following completion of home visit to obtain further dietary information.
  • Community
    • SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
    • Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms
    • Two telephone calls within week following assessment appointment to collect two more days of dietary intake data.
  • Development of abbreviated version of SCREEN II
    • Development used mixed methods
      • 21 dietitians with expertise in senior nutrition reviewed SCREEN II and ranked each item on its importance to the construct of nutrition risk
      • Items ranked highest were candidates for inclusion in the abbreviated version.

Study 2:

  • Community sample from Study 1
    • SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
    • Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms; a second SCREEN II was form completed during assessment appointment.
  • Additional community sample
    • First administration of SCREEN II and demographic information completed in presence of research assistant
    • Second SCREEN II mailed, completed and returned within two weeks of first administration.

Study 3:

  • Telephone administration of SCREEN II and a demographic and functional assessment questionnaire
  • Second telephone administration completed by second trained researcher (time from first administration not specified).

Statistical Analysis

Study 1:

  • Descriptive statistics for all samples and subgroups for comparison purposes on demographics, health and functional measures
  • Receiver operating characteristics (ROC) curves were created using the median of the dietitian's risk rating as the cut-point for risk
    • Area under the curve (AUC) equals consistent with scoring of the criterion (risk/no risk)
    • Higher AUC equals increased consistency
  • Development of abbreviated SCREEN II: Using dietitian ranking as dependent variable, individual SCREEN II items were regressed to determine those most predictive of the risk rating.

Study 2:

  • Intra-class correlation (ICC) of scores for the entire index and individual index items.

Study 3:

  • Intra-class correlation (ICC) of scores for the entire index and individual index items.
Data Collection Summary:

Timing of Measurements

Study 1:

  • Geriatrician's Clinic:
    • SCREEN II, weight and blood work collected at clinic visit
    • Demographic and medical questionnaire, anthropometry and 24-hour dietary recall completed in person's home (time period from clinic visit not specified)
    • Two telephone calls within week following completion of home visit to obtain further dietary information
  • Community:
    • SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
    • Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms
    • Two telephone calls within week following assessment appointment to collect two more days of dietary intake data.

Study 2:

  • Community sample from Study 1:
    • SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
    • Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms. Second SCREEN II form completed during assessment appointment.
  • Additional community sample:
    • First administration of SCREEN II and demographic information completed in presence of research assistant
    • Second SCREEN II mailed, completed and returned within two weeks of first administration.

Study 3:

  • Telephone administration of SCREEN II and a demographic and functional assessment questionnaire
  • Second telephone administration completed by second trained researcher (time from first administration not specified).

Dependent Variables

  • Study 1: SCREEN II index; development of abbreviated SCREEN II index used the dietitian ranking as the dependent variable
  • Study 2: Second completion of SCREEN II index
  • Study 3: Second telephone interview completion of SCREEN II index.

Independent Variables

Study 1: Dietitian's rating of nutritional risk based on medical and nutritional history, dietary intake and anthropometry (one- point scale; one = low risk, 10 = high risk).

  • Medical and Nutritional History questionnaire
    • Demographics
    • List of medical problems, including vision and hearing
    • Use of medications, vitamins, minerals, herbal remedies, meal supplements and alcohol
    • Perceived health status
    • Quality of life using SF-20
    • Doctor visits and hospitalizations in the past six months
    • Dentition
    • Swallowing problems
    • Smoking
    • Gastrointestinal problems
    • Appetite, taste and smell changes
    • Specialized diets
    • Activities of daily living
    • Use of community services (e.g., Meals on Wheels, home care, nursing, etc.)
    • Income
  • Anthropometry
    • Triceps and subscapular skinfolds
    • Knee height
    • Weight
    • Mid-arm, wrist and calf circumferences
  • Dietary intake: 24-hour recalls (two weekdays, one weekend day)
  • Laboratory data
    • Prealbumin, N=44
    • C-reactive protein, N=50
    • Hematocrit and hemoglobin, N=57
  •  Development of abbreviated SCREEN II
  • individual SCREEN II items.

Study 2: First completion of SCREEN II index.

Study 3: First telephone interview completion of SCREEN II index.

Description of Actual Data Sample:

Initial N

Study 1:

  • 61 participants from geriatrician's clinic
  • 132 participants from community.

Study 2:

  • 45 participants from community subsample of Study 1
  • 100 participants (additional) from community.

Study 3:

97 participants: 69 for inter-rater reliability, 28 for intra-rater reliability.

Attrition (final N):

Study 1:

  • 61 participants from geriatrician's clinic
  • 132 participants from community.

Study 2:

  • 45 participants from community subsample of Study 1
  • 100 participants (additional) from community
  • Nine additional participants, not specified from which group.

Study 3:

97 participants: 69 for inter-rater reliability, 28 for intra-rater reliability.

Age

Study 1:

  • N=193
    • Less than 64 years, 1.6% (N=3)
    • 65 to 74 years, 48.4% (N=93)
    • 75 to 84 years, 43.8% (N=85)
    • 85+ years, 6.3% (N=12).

Study 2:

  • N=149
    • Less than 64 years, 16.1% (N=24)
    • 65 to 74 years, 44.3% (N=66)
    • 75 to 84 years, 32.2% (N=48)
    • 85+ years, 7.4% (N=11).

Study 3:

  • N=97
    • Less than 64 years, 3.1% (N=3)
    • 65 to 74 years, 17.5% (N=17)
    • 75 to 84 years, 53.6% (N=52)
    • 85+ years, 25.8% (N=25)

Other Relevant Demographics

 

Study 1

N=193

Study 2

N=149

Study 3

N=97

Characteristic % % %
Gender (female) 62.2 69.1 85.5

Living situation

Lives alone

Lives with spouse

 

33.7

56.5

 

37.8

48.6

 

NA

NA

Marital status

Married

Widowed

 

60.1

26.9

 

NA

NA

 

28.9

58.8

Education level

Primary school or less

High school graduate

Post-secondary degree

 

5.2

26.4

37.8

 

8.7

30.9

34.2

 

17.5

22.7

33.0

Income less than $20,000/year 26.9 14 NA

General health rating

Good to excellent

Fair to poor

 

76.7

23.3

 

87.9

12.1

 

NA

NA

Health-related QOL

Health worse than last year

Limited in vigorous activity

Limited in moderate activity

Limited walking stairs

Limited bending/stooping

Limited walking one block

Limited ADL

Receives help with ADL/IADL

 

20.8

76.7

44.1

46.1

42.7

26.9

9.3

43.5

 

NA

61.0

22.3

29.3

21.8

10.9

4.7

NA

 

NA

NA

NA

NA

NA

NA

NA

64.9

Dietitian rating

Risk level         Low (rank one to four)

Moderate (rank five to seven)

High (rank eight to 10)

 

14.5

66.8

18.7

 

NA

NA

NA

 

NA

NA

NA

Location:

Guelph area of Ontario, Canada

Summary of Results:

 Proportion of Participants with Specific SCREEN II Item Scores that Indicate Increased Risk (Item Score <2), Study 1-3

Question

Study 1

(%)

Study 2

(%)

Study 3

(%)

More than 5-lb weight change

 23 25.5 18.6 
Unintentional weight change

 4.7

4.7

14.4 

Think excess weight

 60.2

 57.7

 60.4

Think weight less than should be  8.3  4.7  4.2
Often/always skip meals  4.2  5.4  8.2
Limits foods  38.7  36.9  30.9
Fair/poor appetite  12.6  7.4  19.6

Less than or equal to three fruits and vegetables per day

 33.0  43.0  8.2

Less than or equal to one meat and alternatives per day

 22.5  39.6  12.4
Less than or equal to two meat and protein per day  43.9  55.0  28.9
Tree to four cups fluid per day  23.8  28.4  4.1
Sometimes/often swallow difficulty  17.5  15.4  13.4
Sometimes/often chewing difficulty  19.6  10.7  19.6
Sometimes/often use meal replacements  14.7  24.8  3.0
Sometimes/never eat with others  33.0  42.3  55.7
Prepares own meal  59.2  70.5  67.0
Shares meal prep with others  14.7  12.8  9.3
Other cooks  26.2  16.8  23.7
Sometimes cooking is a chore  29.6  30.0  23.7
Usually cooking a is a chore  9.5  7.0  15.5
Dissatisfied with others' cooking  0.5  0  1.0
Often/always difficulty with groceries  12.6  15.4  4.1
       
TOTAL SCREEN II SCORE      
  Median 95% Confidence Interval  
Study 1 sample  51.0  48.28, 50.37  
Study 2 sample  49.0  47.27, 49.37  
Study 3 sample  52.0  50.07, 52.41  

 

Other Findings

Study 1:

  • Seniors recruited from the geriatrician's clinic had significantly higher dietitian risk rating (7.3±6.8 vs. 5.7±1.5, T=-8.0, P=0.000) and lower SCREEN II scores (45.9±7.0 vs. 50.8±6.9, T=4.46, P=0.000) than seniors recruited from the community
  • Participants who perceived their health to be excellent had significantly higher SCREEN II scores than those who considered their health to be poor (53.7±4.3 vs. 42.4±8.0, F=12.2, P=0.000)
  •  SCREEN II was significantly correlated with the dietitian nutritional risk rating (P=-0.62, P=0.01)
  • The ROC curve using the median dietitian risk rating 96.0 on a scale of one to 10 indicated that SCREEN II was an adequate index for comparison to this criterion
  • The AUC was 82% and a score less than 54 was considered a good cut-point with a sensitivity of 84%, specificity of 62%, positive predictive value of 85% and a negative predictive value of 61%
  • The eight-item abbreviated SCREEN II was also an adequate index when compared with the criteria. The AUC was 79% and a score less than 43 out of a possible 48 total had a sensitivity of 84%, specificity of 58%, positive predictive value of 83% and a negative predictive value of 59%
  • A 'high-risk' category (ranking eight to 10 by dietitian risk rating) was also determined. The AUC was 82% and the best cutoff was less than 50 (SE=86%, SP=66%)
  • A similar lower cut-point (score less than 38) for the abbreviated version was also determined (SE=77%, SP=64%, AUC=78%).

Study 2:

  • Total SCREEN II score for test and retest were significantly and highly correlated [ICC=0.83, 95% CI (0.75, 0.88)]
  • The test-retest reliability of the abbreviated SCREEN II is similar [ICC=0.84, 95% CI (0.79, 0.89)].

Study 3:

  • Intra-class correlations indicate that inter-rater [ICC=0.83, 95% CI (0.74, 0.89)] and intra-rater reliability [ICC 0.90, 95% CI (0.80, 0.95)] were adequate for the full index
  • Abbreviated SCREEN II had slightly lower reliability [ICC=0.79, inter-rater, ICC=0.85 intra-rater)
  • Individual SCREEN II items were compared for test-retest, and inter-rater and intra-rater reliability
    • There was a range of ICC based on these analyses
    • There was no consistency among the three types of reliability testing for items with low correlations (ICC<0.5) suggesting that no specific items warranted removal or rewriting.
Author Conclusion:
  • Based on dietitian rankings for the validation sample, nutritional risk based on a detailed nutritional assessment and clinical judgement is prevalent in community-living seniors
  • SCREEN II has a higher sensitivity, specificity and reliability than SCREEN I
  • Recommended cut-off of less than 54 on the SCREEN II be used for health promotion and nutrition education efforts
  • Recommended cut-off of less than 50 on the SCREEN II be used for a screening process
  • SCREEN II is valid and reliable.
Funding Source:
Government: Canadian Institutes of Health Research
Reviewer Comments:
  • A relatively well-conducted set of studies to construct validate and test the reliability of SCREEN II
  • More information on exclusion criteria would be desired
  • More information on the timing of events in Study 3 would be helpful
  • Information on the ethnicity of the participants would enhance the validity, reliability and applicability of the data.
Quality Criteria Checklist: Primary Research
Relevance Questions
  1. Would implementing the studied intervention or procedure (if found successful) result in improved outcomes for the patients/clients/population group? (Not Applicable for some epidemiological studies) Yes
  2. Did the authors study an outcome (dependent variable) or topic that the patients/clients/population group would care about? Yes
  3. Is the focus of the intervention or procedure (independent variable) or topic of study a common issue of concern to dieteticspractice? Yes
  4. Is the intervention or procedure feasible? (NA for some epidemiological studies) Yes
 
Validity Questions
1. Was the research question clearly stated? Yes
  1.1. Was (were) the specific intervention(s) or procedure(s) [independent variable(s)] identified? Yes
  1.2. Was (were) the outcome(s) [dependent variable(s)] clearly indicated? Yes
  1.3. Were the target population and setting specified? Yes
2. Was the selection of study subjects/patients free from bias? ???
  2.1. Were inclusion/exclusion criteria specified (e.g., risk, point in disease progression, diagnostic or prognosis criteria), and with sufficient detail and without omitting criteria critical to the study? No
  2.2. Were criteria applied equally to all study groups? ???
  2.3. Were health, demographics, and other characteristics of subjects described? Yes
  2.4. Were the subjects/patients a representative sample of the relevant population? Yes
3. Were study groups comparable? Yes
  3.1. Was the method of assigning subjects/patients to groups described and unbiased? (Method of randomization identified if RCT) N/A
  3.2. Were distribution of disease status, prognostic factors, and other factors (e.g., demographics) similar across study groups at baseline? N/A
  3.3. Were concurrent controls or comparisons used? (Concurrent preferred over historical control or comparison groups.) N/A
  3.4. If cohort study or cross-sectional study, were groups comparable on important confounding factors and/or were preexisting differences accounted for by using appropriate adjustments in statistical analysis? Yes
  3.5. If case control study, were potential confounding factors comparable for cases and controls? (If case series or trial with subjects serving as own control, this criterion is not applicable.) Yes
  3.6. If diagnostic test, was there an independent blind comparison with an appropriate reference standard (e.g., "gold standard")? N/A
4. Was method of handling withdrawals described? No
  4.1. Were follow-up methods described and the same for all groups? Yes
  4.2. Was the number, characteristics of withdrawals (i.e., dropouts, lost to follow up, attrition rate) and/or response rate (cross-sectional studies) described for each group? (Follow up goal for a strong study is 80%.) No
  4.3. Were all enrolled subjects/patients (in the original sample) accounted for? Yes
  4.4. Were reasons for withdrawals similar across groups? N/A
  4.5. If diagnostic test, was decision to perform reference test not dependent on results of test under study? N/A
5. Was blinding used to prevent introduction of bias? No
  5.1. In intervention study, were subjects, clinicians/practitioners, and investigators blinded to treatment group, as appropriate? N/A
  5.2. Were data collectors blinded for outcomes assessment? (If outcome is measured using an objective test, such as a lab value, this criterion is assumed to be met.) ???
  5.3. In cohort study or cross-sectional study, were measurements of outcomes and risk factors blinded? No
  5.4. In case control study, was case definition explicit and case ascertainment not influenced by exposure status? N/A
  5.5. In diagnostic study, were test results blinded to patient history and other test results? N/A
6. Were intervention/therapeutic regimens/exposure factor or procedure and any comparison(s) described in detail? Were interveningfactors described? Yes
  6.1. In RCT or other intervention trial, were protocols described for all regimens studied? N/A
  6.2. In observational study, were interventions, study settings, and clinicians/provider described? N/A
  6.3. Was the intensity and duration of the intervention or exposure factor sufficient to produce a meaningful effect? Yes
  6.4. Was the amount of exposure and, if relevant, subject/patient compliance measured? Yes
  6.5. Were co-interventions (e.g., ancillary treatments, other therapies) described? No
  6.6. Were extra or unplanned treatments described? No
  6.7. Was the information for 6.4, 6.5, and 6.6 assessed the same way for all groups? Yes
  6.8. In diagnostic study, were details of test administration and replication sufficient? N/A
7. Were outcomes clearly defined and the measurements valid and reliable? Yes
  7.1. Were primary and secondary endpoints described and relevant to the question? Yes
  7.2. Were nutrition measures appropriate to question and outcomes of concern? Yes
  7.3. Was the period of follow-up long enough for important outcome(s) to occur? Yes
  7.4. Were the observations and measurements based on standard, valid, and reliable data collection instruments/tests/procedures? Yes
  7.5. Was the measurement of effect at an appropriate level of precision? Yes
  7.6. Were other factors accounted for (measured) that could affect outcomes? Yes
  7.7. Were the measurements conducted consistently across groups? Yes
8. Was the statistical analysis appropriate for the study design and type of outcome indicators? Yes
  8.1. Were statistical analyses adequately described and the results reported appropriately? Yes
  8.2. Were correct statistical tests used and assumptions of test not violated? Yes
  8.3. Were statistics reported with levels of significance and/or confidence intervals? Yes
  8.4. Was "intent to treat" analysis of outcomes done (and as appropriate, was there an analysis of outcomes for those maximally exposed or a dose-response analysis)? N/A
  8.5. Were adequate adjustments made for effects of confounding factors that might have affected the outcomes (e.g., multivariate analyses)? N/A
  8.6. Was clinical significance as well as statistical significance reported? No
  8.7. If negative findings, was a power calculation reported to address type 2 error? N/A
9. Are conclusions supported by results with biases and limitations taken into consideration? Yes
  9.1. Is there a discussion of findings? Yes
  9.2. Are biases and study limitations identified and discussed? No
10. Is bias due to study's funding or sponsorship unlikely? Yes
  10.1. Were sources of funding and investigators' affiliations described? Yes
  10.2. Was the study free from apparent conflict of interest? Yes