INTRODUCTION

The prevalence of behavioural and psychological symptoms of dementia (BPSD) in residential aged care (RAC) is high, occurring in over 78% of people with dementia (1-3). Routine assessment is a key component in early detection and accurate identification of BPSD provides information from which a care plan can be developed and utilised to address and manage BPSD accordingly. Determining what assessment tools should be used, how often, and by whom is a critical part of care planning. There has been no policy to guide aged care providers about routine assessment of BPSD. Recently the Australian Government introduced the Aged Care Funding Instrument (ACFI) as the means of allocating Australian Government subsidies to residential aged care providers, replacing the former Resident Classification Scale (4). The ACFI is obtained within 4 to 6 weeks of admission to a RACF and consists of 12 care need questions under three domains of Activities of Daily Living (ACFI-ADL), Behaviour Supplement (ACFI-BEH) and Complex Health Care Supplement (ACFI-CHC), some of which have specified assessment tools (4). Such information concerning the resident’s functional, mental, behavioural and medical conditions is collected by Residential Aged Care Facilities (RACF) staff, which categorises the residents’ care needs as nil, low, medium or high (5).  

The research team were interested in finding out whether or not the ACFI-BEH, which has potential for a routine assessment of BPSD, can provide clinically appropriate information that can logically inform care of residents with BPSD. Limited evidence is available concerning the validity of the ACFI-BEH and the utility of the ACFI-BEH data in informing care practices related to BPSD. Three key issues need to be taken into account in considering the utility of the ACFI in driving care planning practice: 1) there are no publicly available psychometric analyses of the ACFI including the ACFI-BEH; 2) despite the time and resource intensive nature of the ACFI implementation processes the guidelines state that the ACFI is not designed to provide a comprehensive assessment of an individual, to inform care planning, or to monitor the quality of care provided; and 3) the ACFI assessment does not require routine, annual re-appraisal (5). Indeed, a re-appraisal of care needs using the ACFI can be conducted: “any time 12 months or more after the existing classification has taken effect; if there has been a major change in the resident’s care needs; and within two months of a resident transferring from another aged care home, or at any time when a resident is classified at the lowest classification level” (pp.27-28) (4). 

While ACFI is not intended as a care planning tool, it would be illogical not to use the results of the ACFI-BEH especially when there is often no other mechanism for a routine assessment of BPSD in RACFs. Further, the assumption would be that to receive adequate funding the ACFI should be an accurate reflection of a resident’s care needs. It would be also inefficient to use other assessment tools at the same time, which may provide similar information to that of the ACFI-BEH. For instance, the questions and language used to assess vocal and physical behaviours, wandering and depression are quite similar to many of the well-known and validated instruments for the same constructs such as Cohen Mansfield Agitation Inventory (CMAI) (6), Revised Algase Wandering Scale (RAWS) (7), and Cornell Scale for Depression in Dementia (CSDD) (8) respectively. Furthermore, the depression component of the ACFI-BEH contains the very same CSDD.

We conducted a study to test the feasibility and the effects of a multi-pronged education toolkit that utilised the ACFI-BEH to inform care planning for aged care residents with BPSD (9). This paper reports a secondary analysis of the ACFI scores examining the extent to which the existing ACFI scores reflected the resident’s present condition using the validated BPSD tools mentioned above (CMAI, RAWS-LTC and CSDD). RAWS-LTC (7) is a well validated tool to measure wandering among people with dementia and the other two assessment tools were recommended by the Dementia Outcome Measurement Suite (DOMS) review as valid and reliable measures of agitation and depression of individuals with dementia (10). It was not possible to synchronise assessments in this to the timing of the ACFI administration by participating RACFs, although it is well known that assessments taken further apart in time are generally less correlated (11). However, we worked under two assumptions, specifically that BPSD tend to be very persistent (3, 12) and that the ACFI-BEH would have been reviewed and re-conducted if there was a significant change in the resident’s condition. Therefore there should be a moderate to high association between RACF staff’s ACFI assessments and contemporaneous research assessments. This paper reports on the findings of the secondary analysis and its implications in timely assessment and management of BPSD in RACFs.

METHODS

Five residential aged care facilities (RACFs), ranging in size from 35 to 70 beds, from the Sydney metropolitan area, Australia, were recruited to participate in this study. These RACFs were selected because they covered a mix of four private for-profit facilities and one religious not-for-profit facility, with and without dementia specific care, and were interested in joining the study. The facilities were similar in terms of: management structure, staffing and standards; holding three year accreditation status granted in the last 12 months by the Australian Residential Care Accreditation Agency; services by General Practitioners and other specialist health staff; and providing similar levels of nursing care, therapy provision and recreation programs. Prior to recruitment, research ethics approvals were granted by all relevant institutional research ethics committees.

Recruitment occurred from August to December 2010. Written informed consent for all residents was obtained by proxy from a close family member or guardian, in order to access their ACFI data and other clinical records and to conduct BPSD assessment on residents. Inclusion criteria were that informed consent was provided by the resident’s proxy and that the resident of the participating RACF: 1) had a dementia diagnosis; and 2) had an ACFI score greater than ‘A’ on at least one of the following ACFI-BEH domains [ACFI 7 - Wandering, ACFI 8 - Verbal behaviour, ACFI 9 - Physical behaviour & ACFI 10 – Depression] meaning that the assessed behaviour occurred at least once per week. Each ACFI-BEH domain ranges from a score of ‘A’ meaning ‘behaviour does not occur or occurs less than once a week’, to a score of ‘D’ where ‘behaviour occurs twice a day or more, at least 6 days in a week’. Notably, unlike the assessment for wandering and verbal/physical behaviours, depression in the ACFI-BEH domain requires additional information such as a medical practitioner’s diagnosis of depression to qualify scores of ‘C’ or ‘D’. This means despite a resident scoring highest in CSDD as part of the ACFI-BEH if there is no diagnosis made by a medical practitioner or a formal record of depression diagnosis, a rating for depression cannot be ‘C’ or ‘D’ (5). See Table 1 for further details on the ACFI-BEH.

Table 1

Summary of the ACFI-BEH Assessment Procedures (pp.27-33) (5)

Residents were excluded if they: 1) had serious co-morbidities complicating or masking dementia; 2) were receiving palliative care; or 3) were on a respite placement. Of 109 eligible residents approached to participate in the study, 56 (51.4%) provided consent.

After two days of training an experienced aged care Registered Nurse in aged care (Research Nurse) collected data between October 2010 and January 2011, using the following assessment measures.

The Revised Algase Wandering Scale (RAWS)-Long-Term Care version (7): 19 items that measure the frequency (from 1=never/unable through to 4=usually) of wandering based on RAC staff interviews (usually care staff). The RAWS-LTC version is derived from longer earlier versions of the Algase Wandering Scale (13-15) and has been shown to have good internal consistency reliability (Cronbach’s alpha 0.93) (16).
Cohen-Mansfield Agitation Inventory (CMAI) (6): 29 items that measure the frequency (from 1=never through to 7=several times an hour) of agitation during the past two weeks (range 29 to 203); higher scores reflect worse agitation. ‘Agitation’ in CMAI consists of aggressive, physically non-aggressive, and verbally agitated behaviours. The CMAI has good interval consistency reliability (Cronbach’s alpha between 0.86 - 0.91) and is significantly correlated with the Behavioral Pathology in Alzheimer’s Disease (Behave-AD) (17).
Cornell Scale for Depression in Dementia (CSDD) (8): 19 items measuring the following domains of depression - Mood Related Signs, Behavioural Disturbance, Physical Signs, Cyclic Functions, Ideational Disturbance (0=absent, 1=mild or intermittent and 2=severe). While it is recommended that a total score above 10 (> 10) indicates probable major depression and above 18 (> 18) definite major depression (8), the ACFI specifies that a score of 9 or more (≥ 9) indicates depressive symptoms sufficient to interfere with the person’s ability to participate in their regular activities. The ACFI cut off score of ≥ 9 was used in this study. The CSDD’s internal consistency reliability ranges from 0.84 to 0.98 (Cronbach’s alpha) and inter-rater reliability from 0.67 to 0.74 (10). The CSDD requires both staff and resident interviews and observations. In this study where discrepancies were observed between the two interviews the lower score was used as a final score.
Chart audit: Demographics and clinical information including type of dementia, current co-morbidities, length of stay; and the most recent ACFI-BEH scores – measuring levels of care need for wandering, verbal behaviour, physical behaviour and depression.

Data were entered and analysed using SPSS (Statistical Package for the Social Sciences) version 18 (18). Spearman rank order was utilised to assess the correlation between the ACFI-BEH domains and the outcome measures – RAWS, CMAI and CSDD. Pearson product moment correlation was utilised to examine correlation between CSDD scores rated by the Research Nurse through a resident interview and observation and proxy assessment by a RAC staff member. Although for consistency reasons the final ratings of the CSDD items should be based on the Research Nurse’s clinical impression with best available information, it was opportune to conduct further analysis of CSDD scoring to explore the level of agreement between the CSDD proxy assessment score answered by a RAC staff and the CSDD resident response/Research Nurse rated score. Kappa analysis was utilised to determine the level of agreement between these scores.

RESULTS

Age of the participants ranged from 69-98 years (M=86.69 years; SD=6.47), and of the 52 participants recruited, 45 (86.5%) were female. As identified by the RAWS, CMAI and CSDD scores 71.2%of participants presented wandering behaviour, 76.9% verbal aggression, 88.5% physical aggression and 51.9% depressive symptoms. The ACFI-BEH scores were on average 12 months old (ranging between 2-31 months).

As shown in Table 2, a significant correlation was identified between ACFI verbal and CMAI verbal score (rs=0.360, p=0.009). However, no significant correlations were found for wandering, physical aggression and depression.

Table 2

Spearmans Rank Correlation- ACFI-BEH and validated dementia outcome measure for BPSD (pre-test data)

The difference between the ACFI-depression (old scores) and the Research Nurse rated CSDD (current scores) was explored further. It was found that 24% of people who did not have any depression at the time they were assessed using the ACFI (assigned a score of A) now appeared to have potential depression according to the current CSDD (≥ 9). There were no other assessment results for depression available for the participating residents, which means those 24% of the residents did not have depression recorded anywhere in their record.

We also compared the correlation of the results of two different sources within the CSDD score, bearing in mind that the CSDD is designed to consider the best available sources of information including staff interview, resident interview and observation. We compared the Research Nurse’s interview with the resident and direct resident observation, and the CSDD score based on a Research Nurse’s interview with the RAC staff (proxy). A statistically significant positive relationship was identified (r=0.60, n=52, p=0.0005) between CSDD scores rated by the Research Nurse and by a RAC staff member. The Kappa measure of agreement was Kappa= 0.44, p=0.001 between the classification of residents as having no depression, probable depression and major depression using the total score rated by the Research Nurse and a RAC staff member (proxy). Kappa coefficients of 0.40 – 0.75 have been characterised as fair to good (19).

There was a high proportion of ‘a’ - ‘unable to evaluate’ responses on CSDD questions 16 to 19 measuring Suicidal ideation, Self-deprecation, Pessimism and Mood-congruent delusions. The frequency of non-completion for these four items (i.e. unable to rate) was 40% and 48% when implemented by RAC staff and the trained Research Nurse respectively. Field notes and observations during the study suggest the missing items were associated with the lengthy time required to implement the tool and the difficulty of scoring these items for residents with dementia.

DISCUSSION AND CONCLUSION

This study has investigated one of the most pressing issues that the Australian Government and the aged care industry face: that is having a reliable regular assessment mechanism for ensuring the provision of quality residential aged care services.  While our findings are instructive they should be interpreted with caution for two reasons: their generalisability is limited given that the age of the ACFI data used in the study may not be a true reflection of the current behavioural status of the residents participating in the study; and that results reported in this paper are of a secondary analysis and therefore the sample size for this analysis was dictated by the primary analysis (9).

The study findings question the accuracy of some of the ACFI-BEH data that is routinely collected by RACF staff, in particular the data on wandering, physical behaviours and depression. The existing ACFI-BEH scores did not accurately reflect the resident’s present condition when compared with the scores obtained by the Research Nurse using validated BPSD tools. This study cannot confirm if the result is due to an inappropriate rating of the ACFI-BEH by staff from RACFs, to the age of the data when compared with more recent assessments, or if there is a fundamental flaw in the ACFI-BEH instrument. One explanation might be the way the degree or severity of some of the ACFI-BEH is determined. For example, for depression, the CSDD scores of 19 or higher (i.e. definite major depression) can only guarantee the category of B (mild care needs) in the ACFI because the categories of moderate (C) or high (D) level care needs require a diagnosis or provisional diagnosis of depression made by a medical practitioner (5). Anecdotally it is well known in the aged care industry that GPs appear reluctant to make a formal diagnosis of depression in RACFs. Furthermore, in the ACFI Wandering scale only problematic wandering behaviour attracts scores for B, C or D (5), whereas the RAWS-LTC does not make a distinction between problematic and non-problematic wandering.

Nevertheless, the question is asked if adequate care can be provided for residents when there is no up-to-date ACFI information available for assessment and management of BPSD. The main findings of the study which are reported elsewhere (9) confirm that the overall quality of the care plans, measured as how well they addressed behaviour, was poor for all behavioural domains, in particular for depression where high proportions of residents with depression as assessed by the Research Nurse (pre educational intervention-=73.9%, post educational intervention-=75%) had care plans that did not identify or address their depression in any way. This result may not be surprising as the allocation of the funding for the ACFI-BEH domain is approximately a third of the funding allocated for the ACFI-ADL and approximately a half of the ACFI-CHC (See Table 3 for subsidy rates for three domains of the ACFI). As the ACFI-BEH domain attracts a proportionally small amount of subsidies compared to other domains, aged care providers have little incentive to reappraise the ACFI-BEH. This was reflected in the age of the ACFI scores in this study (2-30 months). Our chart review of the participating residents clearly showed that whilst the ACFI is the funding allocation tool, there was no other mechanism to ensure appropriate assessment of BPSD occurring in the participating RACFs.

Table 3

Daily ACFI subsidy rates, 2010-11 (p.31) (4)

Our study findings show that ACFI-BEH data, in particular wandering, depression and physical behaviours, cannot be relied on in reporting BPSD in aged care residents. We also question whether or not the current arrangement of a significantly smaller amount of subsidy rates allocated for the ACFI-BEH, compared to the rates allocated for other domains of the ACFI, is appropriate and useful to guide care services. Given that there is no other mechanism for a regular assessment of BPSD in RACFs and that the current funding tool (the ACFI) does not support, or to some extent discourages the on-going BPSD assessment, the study raises an important question as to how quality care is ensured in the current environment. The findings suggest that it would be difficult for RAC staff to provide effective person-centred care for residents with BPSD, as there is no up-to-date information concerning the resident’s BPSD. A person-centred care approach requires a comprehensive assessment of the resident’s past history and present context and is critical to producing effective resident outcomes (20, 21). Quality behaviour assessment requires a significant amount of staff time, and it is unclear as to what funding allocation is proportioned to behaviour assessment with the ACFI. Consequently, further research is warranted to examine the construct validity of the ACFI-BEH data using a larger sample and recent ACFI data, the consequences of the ACFI subsidy rates on the care provided to residents with BPSD, and the association between the ACFI subsidy rates and resident well-being and quality of life.

Despite having good reliability and validity (10), rating of the CSDD appears to be particularly problematic for residents with moderate to severe dementia.  The fact that 20% of the CSDD items (items 16-19) were not answered in 40-48% of these assessments raises an important question as to how useful the instrument is in everyday RAC practice. The same issues, albeit less frequently, were identified in Snowdon’s study where 14% of participating residents did not have complete CSDD ratings due to their severe cognitive impairment and inability to communicate in a meaningful manner (22, 23). Snowdon’s study (n=162) found “items 16–19 were often not rated, even if other items were, because ideational disturbance was too difficult to rate if a subject could not converse intelligibly or convey meaning” (p.34) (23). Whilst CSDD should be based on best available sources of information including medical records, observation and talking to close family member(s), not just interviews with RAC staff and resident, our study findings and those by Snowdon’s (22, 23) indicate they are not necessarily relied upon when using CSDD.

Notably, most published studies of CSDD psychometric testing are based on CSDD ratings by specialist mental health clinicians or specifically trained researchers, often psychologists. Furthermore, 20 of 98 submissions made to the first national review of the ACFI raised specific concerns about the use of the CSDD, most of which related to its complexity and time consuming nature, as well as its unpopularity among GPs (4). It is not known how reliable the CSDD is when used in RACFs where the workforce may not be fully trained to administer CSDD, since there is no standardised process in place to assess their knowledge and skill base prior to administering the CSDD, a condition required by the instrument developers.  A primary goal of comprehensive assessment is to inform the resident’s care plan and to monitor the quality of care provided. This is best achieved when using valid and reliable assessment instruments that reflect the person’s current health status. Based on the results of this study we are conducting a large scale multi-site study that examines the clinical utility of RACF staff completed CSDD assessments, against clinical diagnosis of depression made by a specialist psychogeriatic clinician. Further research is needed to develop a suite of tools that can be reliably and easily used for assessment of BPSD in RACFs, as well as establishing evidence for strategies that facilitate the routine assessment of BPSD in RACFs.