NSCR: Quick and Easy Nutrition Screening Tools: Adult Patients in Acute and Ambulatory Care Settings (2009)
Study 1: The purpose of this study was to conduct a construct validation study that compared the index score from SCREEN II to a dietitian's rating of nutritional risk based on medical and nutritional history, dietary intake and anthropometry. An abbreviated version of SCREEN II was also developed.
Study 2: The purpose of this study was to assess the test-retest reliability of SCREEN II.
Study 3: The purpose of this study was to assess the inter-rater reliability associated with SCREEN II.
Study 1
- 55 years of age or older
- Lived in the community (Guelph area of Ontario)
- No cognitive impairment as determined by Standardized Mini Mental State Exam; scores greater than or equal to 26.
Study 2
- Seniors from the community subsample from Study 1, OR
- Seniors who lived in the community (Guelph area of Ontario).
Study 3
- Seniors who required assistance from others to live independently in the community (Guelph area of Ontario)
- No cognitive impairment.
Study 1
- Less than 55 years of age
- Did not live in the community
- Cognitive impairment.
Study 2
- Did not participate in Study 1, OR
- Did not live in the community.
Study 3
- Did not require assistance from others to live independently in the community
- Cognitive impairment.
Recruitment
Study 1:
- Participants recruited from the community in the Guelph area of Ontario via newspaper ads, posters, invitational letters and displays at various community programs and clubs
- Participants also recruited from a geriatrician's clinic. Clinic staff identified patients who were eligible and asked them to participate.
Study 2:
- 45 seniors from the community subsample from Study 1
- 100 seniors from the community using displays at a recreation center, a mall and places of worship.
Study 3:
Participants were recruited from malls, recreation and day programs in the Guelph area.
Design
Study 1:
- Geriatrician's Clinic
- SCREEN II, weight and blood work collected at clinic visit
- Demographic and medical questionnaire, anthropometry and 24-hour dietary recall completed in person's home (time period from clinic visit not specified)
- Two telephone calls within a week following completion of home visit to obtain further dietary information.
- Community
- SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
- Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms
- Two telephone calls within week following assessment appointment to collect two more days of dietary intake data.
- Development of abbreviated version of SCREEN II
- Development used mixed methods
- 21 dietitians with expertise in senior nutrition reviewed SCREEN II and ranked each item on its importance to the construct of nutrition risk
- Items ranked highest were candidates for inclusion in the abbreviated version.
- Development used mixed methods
Study 2:
- Community sample from Study 1
- SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
- Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms; a second SCREEN II was form completed during assessment appointment.
- Additional community sample
- First administration of SCREEN II and demographic information completed in presence of research assistant
- Second SCREEN II mailed, completed and returned within two weeks of first administration.
Study 3:
- Telephone administration of SCREEN II and a demographic and functional assessment questionnaire
- Second telephone administration completed by second trained researcher (time from first administration not specified).
Statistical Analysis
Study 1:
- Descriptive statistics for all samples and subgroups for comparison purposes on demographics, health and functional measures
- Receiver operating characteristics (ROC) curves were created using the median of the dietitian's risk rating as the cut-point for risk
- Area under the curve (AUC) equals consistent with scoring of the criterion (risk/no risk)
- Higher AUC equals increased consistency
- Development of abbreviated SCREEN II: Using dietitian ranking as dependent variable, individual SCREEN II items were regressed to determine those most predictive of the risk rating.
Study 2:
- Intra-class correlation (ICC) of scores for the entire index and individual index items.
Study 3:
- Intra-class correlation (ICC) of scores for the entire index and individual index items.
Timing of Measurements
Study 1:
- Geriatrician's Clinic:
- SCREEN II, weight and blood work collected at clinic visit
- Demographic and medical questionnaire, anthropometry and 24-hour dietary recall completed in person's home (time period from clinic visit not specified)
- Two telephone calls within week following completion of home visit to obtain further dietary information
- Community:
- SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
- Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms
- Two telephone calls within week following assessment appointment to collect two more days of dietary intake data.
Study 2:
- Community sample from Study 1:
- SCREEN II, demographic and health questionnaires mailed to participants who qualified (as determined during telephone call)
- Anthropometry, physical measures and 24-hour dietary recall completed at assessment appointment within two weeks of completion of forms. Second SCREEN II form completed during assessment appointment.
- Additional community sample:
- First administration of SCREEN II and demographic information completed in presence of research assistant
- Second SCREEN II mailed, completed and returned within two weeks of first administration.
Study 3:
- Telephone administration of SCREEN II and a demographic and functional assessment questionnaire
- Second telephone administration completed by second trained researcher (time from first administration not specified).
Dependent Variables
- Study 1: SCREEN II index; development of abbreviated SCREEN II index used the dietitian ranking as the dependent variable
- Study 2: Second completion of SCREEN II index
- Study 3: Second telephone interview completion of SCREEN II index.
Independent Variables
Study 1: Dietitian's rating of nutritional risk based on medical and nutritional history, dietary intake and anthropometry (one- point scale; one = low risk, 10 = high risk).
- Medical and Nutritional History questionnaire
- Demographics
- List of medical problems, including vision and hearing
- Use of medications, vitamins, minerals, herbal remedies, meal supplements and alcohol
- Perceived health status
- Quality of life using SF-20
- Doctor visits and hospitalizations in the past six months
- Dentition
- Swallowing problems
- Smoking
- Gastrointestinal problems
- Appetite, taste and smell changes
- Specialized diets
- Activities of daily living
- Use of community services (e.g., Meals on Wheels, home care, nursing, etc.)
- Income
- Anthropometry
- Triceps and subscapular skinfolds
- Knee height
- Weight
- Mid-arm, wrist and calf circumferences
- Dietary intake: 24-hour recalls (two weekdays, one weekend day)
- Laboratory data
- Prealbumin, N=44
- C-reactive protein, N=50
- Hematocrit and hemoglobin, N=57
- Development of abbreviated SCREEN II
- individual SCREEN II items.
Study 2: First completion of SCREEN II index.
Study 3: First telephone interview completion of SCREEN II index.
Initial N
Study 1:
- 61 participants from geriatrician's clinic
- 132 participants from community.
Study 2:
- 45 participants from community subsample of Study 1
- 100 participants (additional) from community.
Study 3:
97 participants: 69 for inter-rater reliability, 28 for intra-rater reliability.
Attrition (final N):
Study 1:
- 61 participants from geriatrician's clinic
- 132 participants from community.
Study 2:
- 45 participants from community subsample of Study 1
- 100 participants (additional) from community
- Nine additional participants, not specified from which group.
Study 3:
97 participants: 69 for inter-rater reliability, 28 for intra-rater reliability.
Age
Study 1:
- N=193
- Less than 64 years, 1.6% (N=3)
- 65 to 74 years, 48.4% (N=93)
- 75 to 84 years, 43.8% (N=85)
- 85+ years, 6.3% (N=12).
Study 2:
- N=149
- Less than 64 years, 16.1% (N=24)
- 65 to 74 years, 44.3% (N=66)
- 75 to 84 years, 32.2% (N=48)
- 85+ years, 7.4% (N=11).
Study 3:
- N=97
- Less than 64 years, 3.1% (N=3)
- 65 to 74 years, 17.5% (N=17)
- 75 to 84 years, 53.6% (N=52)
- 85+ years, 25.8% (N=25)
Other Relevant Demographics
Study 1 N=193 |
Study 2 N=149 |
Study 3 N=97 |
|
Characteristic | % | % | % |
Gender (female) | 62.2 | 69.1 | 85.5 |
Living situation Lives alone Lives with spouse |
33.7 56.5 |
37.8 48.6 |
NA NA |
Marital status Married Widowed |
60.1 26.9 |
NA NA |
28.9 58.8 |
Education level Primary school or less High school graduate Post-secondary degree |
5.2 26.4 37.8 |
8.7 30.9 34.2 |
17.5 22.7 33.0 |
Income less than $20,000/year | 26.9 | 14 | NA |
General health rating Good to excellent Fair to poor |
76.7 23.3 |
87.9 12.1 |
NA NA |
Health-related QOL Health worse than last year Limited in vigorous activity Limited in moderate activity Limited walking stairs Limited bending/stooping Limited walking one block Limited ADL Receives help with ADL/IADL |
20.8 76.7 44.1 46.1 42.7 26.9 9.3 43.5 |
NA 61.0 22.3 29.3 21.8 10.9 4.7 NA |
NA NA NA NA NA NA NA 64.9 |
Dietitian rating Risk level Low (rank one to four) Moderate (rank five to seven) High (rank eight to 10) |
14.5 66.8 18.7 |
NA NA NA |
NA NA NA |
Location:
Guelph area of Ontario, Canada
Proportion of Participants with Specific SCREEN II Item Scores that Indicate Increased Risk (Item Score <2), Study 1-3
Question |
Study 1 (%) |
Study 2 (%) |
Study 3 (%) |
More than 5-lb weight change |
23 | 25.5 | 18.6 |
Unintentional weight change |
4.7 |
4.7 |
14.4 |
Think excess weight |
60.2 |
57.7 |
60.4 |
Think weight less than should be | 8.3 | 4.7 | 4.2 |
Often/always skip meals | 4.2 | 5.4 | 8.2 |
Limits foods | 38.7 | 36.9 | 30.9 |
Fair/poor appetite | 12.6 | 7.4 | 19.6 |
Less than or equal to three fruits and vegetables per day |
33.0 | 43.0 | 8.2 |
Less than or equal to one meat and alternatives per day |
22.5 | 39.6 | 12.4 |
Less than or equal to two meat and protein per day | 43.9 | 55.0 | 28.9 |
Tree to four cups fluid per day | 23.8 | 28.4 | 4.1 |
Sometimes/often swallow difficulty | 17.5 | 15.4 | 13.4 |
Sometimes/often chewing difficulty | 19.6 | 10.7 | 19.6 |
Sometimes/often use meal replacements | 14.7 | 24.8 | 3.0 |
Sometimes/never eat with others | 33.0 | 42.3 | 55.7 |
Prepares own meal | 59.2 | 70.5 | 67.0 |
Shares meal prep with others | 14.7 | 12.8 | 9.3 |
Other cooks | 26.2 | 16.8 | 23.7 |
Sometimes cooking is a chore | 29.6 | 30.0 | 23.7 |
Usually cooking a is a chore | 9.5 | 7.0 | 15.5 |
Dissatisfied with others' cooking | 0.5 | 0 | 1.0 |
Often/always difficulty with groceries | 12.6 | 15.4 | 4.1 |
TOTAL SCREEN II SCORE | |||
Median | 95% Confidence Interval | ||
Study 1 sample | 51.0 | 48.28, 50.37 | |
Study 2 sample | 49.0 | 47.27, 49.37 | |
Study 3 sample | 52.0 | 50.07, 52.41 |
Other Findings
Study 1:
- Seniors recruited from the geriatrician's clinic had significantly higher dietitian risk rating (7.3±6.8 vs. 5.7±1.5, T=-8.0, P=0.000) and lower SCREEN II scores (45.9±7.0 vs. 50.8±6.9, T=4.46, P=0.000) than seniors recruited from the community
- Participants who perceived their health to be excellent had significantly higher SCREEN II scores than those who considered their health to be poor (53.7±4.3 vs. 42.4±8.0, F=12.2, P=0.000)
- SCREEN II was significantly correlated with the dietitian nutritional risk rating (P=-0.62, P=0.01)
- The ROC curve using the median dietitian risk rating 96.0 on a scale of one to 10 indicated that SCREEN II was an adequate index for comparison to this criterion
- The AUC was 82% and a score less than 54 was considered a good cut-point with a sensitivity of 84%, specificity of 62%, positive predictive value of 85% and a negative predictive value of 61%
- The eight-item abbreviated SCREEN II was also an adequate index when compared with the criteria. The AUC was 79% and a score less than 43 out of a possible 48 total had a sensitivity of 84%, specificity of 58%, positive predictive value of 83% and a negative predictive value of 59%
- A 'high-risk' category (ranking eight to 10 by dietitian risk rating) was also determined. The AUC was 82% and the best cutoff was less than 50 (SE=86%, SP=66%)
- A similar lower cut-point (score less than 38) for the abbreviated version was also determined (SE=77%, SP=64%, AUC=78%).
Study 2:
- Total SCREEN II score for test and retest were significantly and highly correlated [ICC=0.83, 95% CI (0.75, 0.88)]
- The test-retest reliability of the abbreviated SCREEN II is similar [ICC=0.84, 95% CI (0.79, 0.89)].
Study 3:
- Intra-class correlations indicate that inter-rater [ICC=0.83, 95% CI (0.74, 0.89)] and intra-rater reliability [ICC 0.90, 95% CI (0.80, 0.95)] were adequate for the full index
- Abbreviated SCREEN II had slightly lower reliability [ICC=0.79, inter-rater, ICC=0.85 intra-rater)
- Individual SCREEN II items were compared for test-retest, and inter-rater and intra-rater reliability
- There was a range of ICC based on these analyses
- There was no consistency among the three types of reliability testing for items with low correlations (ICC<0.5) suggesting that no specific items warranted removal or rewriting.
- Based on dietitian rankings for the validation sample, nutritional risk based on a detailed nutritional assessment and clinical judgement is prevalent in community-living seniors
- SCREEN II has a higher sensitivity, specificity and reliability than SCREEN I
- Recommended cut-off of less than 54 on the SCREEN II be used for health promotion and nutrition education efforts
- Recommended cut-off of less than 50 on the SCREEN II be used for a screening process
- SCREEN II is valid and reliable.
Government: | Canadian Institutes of Health Research |
- A relatively well-conducted set of studies to construct validate and test the reliability of SCREEN II
- More information on exclusion criteria would be desired
- More information on the timing of events in Study 3 would be helpful
- Information on the ethnicity of the participants would enhance the validity, reliability and applicability of the data.
Quality Criteria Checklist: Primary Research
|
|||
Relevance Questions | |||
1. | Would implementing the studied intervention or procedure (if found successful) result in improved outcomes for the patients/clients/population group? (Not Applicable for some epidemiological studies) | Yes | |
2. | Did the authors study an outcome (dependent variable) or topic that the patients/clients/population group would care about? | Yes | |
3. | Is the focus of the intervention or procedure (independent variable) or topic of study a common issue of concern to dieteticspractice? | Yes | |
4. | Is the intervention or procedure feasible? (NA for some epidemiological studies) | Yes | |
Validity Questions | |||
1. | Was the research question clearly stated? | Yes | |
1.1. | Was (were) the specific intervention(s) or procedure(s) [independent variable(s)] identified? | Yes | |
1.2. | Was (were) the outcome(s) [dependent variable(s)] clearly indicated? | Yes | |
1.3. | Were the target population and setting specified? | Yes | |
2. | Was the selection of study subjects/patients free from bias? | ??? | |
2.1. | Were inclusion/exclusion criteria specified (e.g., risk, point in disease progression, diagnostic or prognosis criteria), and with sufficient detail and without omitting criteria critical to the study? | No | |
2.2. | Were criteria applied equally to all study groups? | ??? | |
2.3. | Were health, demographics, and other characteristics of subjects described? | Yes | |
2.4. | Were the subjects/patients a representative sample of the relevant population? | Yes | |
3. | Were study groups comparable? | Yes | |
3.1. | Was the method of assigning subjects/patients to groups described and unbiased? (Method of randomization identified if RCT) | N/A | |
3.2. | Were distribution of disease status, prognostic factors, and other factors (e.g., demographics) similar across study groups at baseline? | N/A | |
3.3. | Were concurrent controls or comparisons used? (Concurrent preferred over historical control or comparison groups.) | N/A | |
3.4. | If cohort study or cross-sectional study, were groups comparable on important confounding factors and/or were preexisting differences accounted for by using appropriate adjustments in statistical analysis? | Yes | |
3.5. | If case control study, were potential confounding factors comparable for cases and controls? (If case series or trial with subjects serving as own control, this criterion is not applicable.) | Yes | |
3.6. | If diagnostic test, was there an independent blind comparison with an appropriate reference standard (e.g., "gold standard")? | N/A | |
4. | Was method of handling withdrawals described? | No | |
4.1. | Were follow-up methods described and the same for all groups? | Yes | |
4.2. | Was the number, characteristics of withdrawals (i.e., dropouts, lost to follow up, attrition rate) and/or response rate (cross-sectional studies) described for each group? (Follow up goal for a strong study is 80%.) | No | |
4.3. | Were all enrolled subjects/patients (in the original sample) accounted for? | Yes | |
4.4. | Were reasons for withdrawals similar across groups? | N/A | |
4.5. | If diagnostic test, was decision to perform reference test not dependent on results of test under study? | N/A | |
5. | Was blinding used to prevent introduction of bias? | No | |
5.1. | In intervention study, were subjects, clinicians/practitioners, and investigators blinded to treatment group, as appropriate? | N/A | |
5.2. | Were data collectors blinded for outcomes assessment? (If outcome is measured using an objective test, such as a lab value, this criterion is assumed to be met.) | ??? | |
5.3. | In cohort study or cross-sectional study, were measurements of outcomes and risk factors blinded? | No | |
5.4. | In case control study, was case definition explicit and case ascertainment not influenced by exposure status? | N/A | |
5.5. | In diagnostic study, were test results blinded to patient history and other test results? | N/A | |
6. | Were intervention/therapeutic regimens/exposure factor or procedure and any comparison(s) described in detail? Were interveningfactors described? | Yes | |
6.1. | In RCT or other intervention trial, were protocols described for all regimens studied? | N/A | |
6.2. | In observational study, were interventions, study settings, and clinicians/provider described? | N/A | |
6.3. | Was the intensity and duration of the intervention or exposure factor sufficient to produce a meaningful effect? | Yes | |
6.4. | Was the amount of exposure and, if relevant, subject/patient compliance measured? | Yes | |
6.5. | Were co-interventions (e.g., ancillary treatments, other therapies) described? | No | |
6.6. | Were extra or unplanned treatments described? | No | |
6.7. | Was the information for 6.4, 6.5, and 6.6 assessed the same way for all groups? | Yes | |
6.8. | In diagnostic study, were details of test administration and replication sufficient? | N/A | |
7. | Were outcomes clearly defined and the measurements valid and reliable? | Yes | |
7.1. | Were primary and secondary endpoints described and relevant to the question? | Yes | |
7.2. | Were nutrition measures appropriate to question and outcomes of concern? | Yes | |
7.3. | Was the period of follow-up long enough for important outcome(s) to occur? | Yes | |
7.4. | Were the observations and measurements based on standard, valid, and reliable data collection instruments/tests/procedures? | Yes | |
7.5. | Was the measurement of effect at an appropriate level of precision? | Yes | |
7.6. | Were other factors accounted for (measured) that could affect outcomes? | Yes | |
7.7. | Were the measurements conducted consistently across groups? | Yes | |
8. | Was the statistical analysis appropriate for the study design and type of outcome indicators? | Yes | |
8.1. | Were statistical analyses adequately described and the results reported appropriately? | Yes | |
8.2. | Were correct statistical tests used and assumptions of test not violated? | Yes | |
8.3. | Were statistics reported with levels of significance and/or confidence intervals? | Yes | |
8.4. | Was "intent to treat" analysis of outcomes done (and as appropriate, was there an analysis of outcomes for those maximally exposed or a dose-response analysis)? | N/A | |
8.5. | Were adequate adjustments made for effects of confounding factors that might have affected the outcomes (e.g., multivariate analyses)? | N/A | |
8.6. | Was clinical significance as well as statistical significance reported? | No | |
8.7. | If negative findings, was a power calculation reported to address type 2 error? | N/A | |
9. | Are conclusions supported by results with biases and limitations taken into consideration? | Yes | |
9.1. | Is there a discussion of findings? | Yes | |
9.2. | Are biases and study limitations identified and discussed? | No | |
10. | Is bias due to study's funding or sponsorship unlikely? | Yes | |
10.1. | Were sources of funding and investigators' affiliations described? | Yes | |
10.2. | Was the study free from apparent conflict of interest? | Yes | |