Cognitive Test Analysis May Determine Reliable MS Change Over Time

A new analytic strategy may help determine whether change over time on commonly used neuropsychological tests reflects reliable improvement or decline in persons with multiple sclerosis (MS), according to a study report in the December issue of the International Journal of MS Care (Barker-Collo and Purdy, 2013).

“Repeated cognitive assessment—particularly of information processing speed—is a common method of tracking disease progression, whether or not treatments such as beta-interferons are being used,” wrote Suzanne L. Barker-Collo, Ph.D., and Suzanne C. Purdy, Ph.D., from the Department of Psychology, Faculty of Science, University of Auckland in New Zealand. “… The importance of accurately tracking change in cognitive functioning in MS is indicated by findings suggesting that cognitive functioning makes a significant and independent contribution to activities of daily living, quality of life, and work and social functioning.”

Therefore, it would be helpful to identify techniques that could determine whether a patient’s improvement or decline over time in neuropsychological test performance is reliable and clinically significant.

A particularly useful cognitive marker may be the Paced Auditory Serial Addition Test (PASAT), which is a measure of information processing speed and the only cognitive measure included in the MS Functional Composite index developed by the U.S. National MS Society Task Force on Clinical Outcomes Assessment.

In this study, 52 persons with MS underwent cognitive testing at baseline and at 6-month follow-up using the PASAT, the Simple Adjusting–Paced Serial Addition Test (A-PSAT), and the Victoria Stroop test.  

The investigators looked at two different methods for assessing the reliability of cognitive change over time. In the simple standard deviation (SD) method, performance is said to change reliably if the difference score exceeds the group mean of the difference scores by more than 1 SD. The Reliable Change Index (RCI) method accounts for practice effects and other sources of variance when determining change in performance over time and uses a more conservative 90% confidence level (±1.645 SD).

The RCI only identified a few persons with MS who declined reliably in cognitive test performance from baseline to 6-month follow-up. The SD method showed reliable declines on most measures in a small number of persons with MS and yielded a larger number of individuals who improved reliably.

Both calculation methods produced similar levels of reliable improvement for the 2.4-second PASAT and Stroop word trial. However, a greater proportion of participants improved reliably when using the SD method for the A-PSAT, Stroop dots, and Stroop colors trials.

The investigators concluded that RCI was a more stringent method than SD, as it identified fewer persons with MS who declined reliably over 6 months.

“Responsiveness of individuals to treatment effects can be lost in group analyses,” the study authors wrote. “The data presented here provide clinicians with an approach for determining whether an individual’s change over time on commonly used neuropsychological tests reflects reliable improvement or decline.“

Study limitations were relatively small sample size and inability of all participants to complete all assessments at each time period.

“Most randomized controlled trials rely on averaging or pooling data across individuals to determine mean change,” the study authors conclude. “The responsiveness of individuals to treatment effects can be lost in such analyses, yet such individual responsiveness is important to future efforts to develop and refine treatments.”

Key open questions

• How will use of the RCI and SD methods compare when tested in a larger validation sample?
• How will use of the RCI and SD methods affect design of and data analysis for large randomized controlled trials?

Disclosures

The New Zealand Multiple Sclerosis Society, Auckland branch, assisted with this project. The study authors reported having no financial disclosures.