Introduction

Nearly everyone working in research fields of psychology and neuropsychiatric diseases is acquainted with the Likert-type scale. The Likert scale was invented by a psychologist called Likert Rensis ¹. Although general applicability of Likert scales has been often questioned, this type of measurement remains an extremely popular methodology in the fields of psychology, public health and nursing research^2-6. In fact, it is so widely used in scaling responses in surveys that it is sometimes used interchangeably with rating scales, even though the two are not synonymous. Analysis of data measured on Likert scales is another issue requiring special notation. A variable measured using Likert type questions exists on an ordinal scale- generally one limited to a few levels. This poses the question of whether we can safely apply statistical methods that rely on assumptions of normality. The issue gets even more problematic when the sample size is small and a complex model needs to be developed to assess a latent variable representative of a health-related phenomenon.

Structural equation modelling (SEM) is a statistical methodology that takes a confirmatory (i.e. hypothesis-testing) approach to the analysis of a structural theory bearing on some phenomenon. Typically, this theory represents “causal” processes that generate observations on multiple variables⁷.

Review of SEM applications during the past 15 years (in psychological research, at least) reveals most measurements to be based on Likert scaled data with estimation of parameters done using maximum likelihood (ML) procedures⁸.

When the number of categories is large and the data approximate a normal distribution, failure to address the ordinal form of the data is likely to have negligible consequences; however, this may not be the case in many studies⁹. In psychology and other social sciences, data are often collected through questionnaireswhich use a Likert scale. Multivariate normality is an essential assumption that may not hold for this kind of data.

In an experiment on assessing spiritual coping, the aim of our study was to illustrate the problems of using the common maximum likelihood (ML) method, and to assess a way for dealing with structural equation models when variables are in categorical form and don’t follow a normal and continuous distribution.

Material and methods

The study population where the data come from contained 120 adolescents in State Welfare Organizations of Tehran database. Data was derived from institutionalized orphan adolescents between 14-20 years of age enrolled in nineteen protector centres of Tehran.

A structural model was developed and specified for spiritual coping. The developed scale was called “Institutionalized adolescents spiritual coping scale”. The structural statistical model was fitted and investigated using common fit indices of ML, and Sattora-bentler estimators for small samples and non-normal Likert scale data.

Sattora-bentler method ()

ML methods produce parameter estimators to ensure that observed sample probability is maximized. This method assumes that observed variables have multi-normal distribution. The likelihood function will be:

Figure 1

In which:

Figure 2

Quantity of A cause estimator function minimized.

Assuming that π shows parameters, we have:

Figure 3

In which:

Figure 4

When mean structure was unknown, assume µ̑=i̅ and then we have:

Figure 5

An alternative hypothesis under this offered model is µ̑ = y̑ and ∑̑ = S which ∑̑ get from ML function:

Figure 6

Regarding that:

Figure 7

Assuming that π̑ showed the ML estimator value given the null hypothesis, the chi square likelihood proportion for fitting model H₀ against H₁ equals 2nF_ML(π̑).

Bentler (2005) noted that adjusting test statistics for small samples may help to achieve better estimators when analysing Likert data for the purpose of hypothesis testing.

When observed variables that do not have multivariate normality, SB-χ² goodness of fit test benefits from the following adjustment to chi square:

Figure 8

In which c is adjusting constant and computed from:

Figure 9

In which d is degree of freedom for model and:

Figure 10

In which W is a weighted matrix and its optimum value is S, also:

Figure 11

This study was approved as a thesis research project for a degree of master in biostatistics registered in Tarbiat Modarres University in Tehran, Iran.

Results

The model for spiritual coping, offered by researchers, contained seven variables. They were spiritual coping, spiritual attitude and subjective norm, which were taken as latent variables; and hopefulness, self-respect, self-efficacy and age, which were taken as observational variables. Observational variable scores were computed from the mean of their items. Because of being three latent variables in the structural model, there were three observational models. After reducing nonrelated items, spiritual attitude, subjective norms and spiritual coping were computed by 11, 10 and 16 items by respectively. Exploratory factor analyses had given the best SEM model. Figure 1 shows the structural model, which describes spiritual coping. As shown in Fig. 1, the measurement model related to subjective norms used second order factor analyses.

Figure 12

Figure 1: Structural model for spiritual coping

Table 1 shows the results of confirmatory factor analyses for measurement models. Results show that fit indices and parameters encountered the under-estimation problem by using the common ML estimator method. Indices show that application of a robust SB-estimator produced a better fit.

Figure 13

Table 1: Confirmatory factor analysis fit indices for the measurement models

The Mardia index shows that none of the variables had normal distribution. Results show that when variables aren’t normal, it is better to use Robust SB-xi square method. Figure 2 shows the measurement model of subjective norms drawn by EQS software package.

Figure 14

Figure 2: second order CFA model for measuring subjective norm latent variable

Measurement models evaluation show the efficiency of Robust Sattora-Bentler correction. Fit indices for total SEM model confirm the same results. Table 2 compares SEM model fit indices in both continuous and categorical status. Study shows that fit indices and parameters encounter the under-estimation problem by using the common ML estimator. Using a Robust SB-chi square method produces a model which better fits data of this type.

Figure 15

Table 2: Fit indexes for Structural Equation Modeling of spiritual coping

Figure 16

Figure 3 shows total the SEM model for spiritual coping, when analysed with common ML method.

Chart 3:Total SEM model for spiritual coping

Discussion

An important disadvantage of all normal theory methods, including ML, is that they can yield distorted conclusions about model adequacy when there are violations of the requisite distributional assumption- here, multivariate normality. That is, when data are not normal, the test statistic Maximum Likelihood is not distributed as a chi-square variate, and the standard errors obtained are generally not correct. Assuming that each categorical variable has an underlying continues and normal distributed scale is undoubtedly a difficult criterion to meet and, in fact, may be totally unrealistic in some cases. As such, in psychological studies, care should be taken in using SEM models on Likert scales.

This study was an example of SEM with small sample size and non-normal distribution. Many studies use Likert scale data with a small sample size in psychology, psychiatry, and social sciences. As found in our experience, in case of non- normal distribution and small sample size, fit indices get problematic and under-estimation problem is encountered. This study used the robust Sattora-bentler method for adjusting under-estimated fit indices. Results show that using this method could be beneficial for researches who need to handle categorical Likert scale data similar to that used here in spiritual coping assessment.

Crawford and Henry evaluated the Positive and Negative Affect Schedule(PANAS) in a large sample of 1003 cases. Data had high positive skew. PANAS is measured with 20 Positive and Negative items, offered by Watson and Clark. Results produced a χ² = 689.8 (156) using the ML method and SB - χ² = 508.3 (156) ¹⁰ using the Sattora-Bentler method.

Consistent with our results, indices garnered from another study by Crawford and Henry (2003) on the Depression Anxiety Stress Scales (DASS) demonstrated that the SB-χ² index provided a better fit than the χ² index¹¹.

The Personal Disturbance Scale was evaluated by Henry (2002) with regards to anxiety and depression scores being computed from 14 Delusions-Symptoms-States items. Data was derived from 758 adolescents between the ages of 16 and 19. Based on the Kolmogorov-Smirnoff test, the normality hypothesis was rejected, and the SB-χ² index was used by for comparing deferent models although with a quite large dataset.

Beck Depression Inventory (BDI) has also been evaluated by fitting second order factor analysis model. Results of the study showed very different values between the ML method and SB estimators. CFI index for the unadjusted ML was equal to 0.76, while it was equal to 0.93 for SB robust method¹².

In the present study, the CFA model fit indices for spiritual coping with 11 items showed evidence of moderate fitting of the model to the data. The reason was that all the variables used for measuring spiritual coping were categorical variables, yet were applied as continues variables ( CFI = .836 ). Fitting the model to the data again using a SB robust model provided acceptable results ((SB-χ² )/df) = 2.16, CFI = .908 ).

Consistent with previous research on other types of psychological scales, we found in our study on spiritual coping that, when variables are categorical, using the SB-χ² robust method produces better estimates than the common ML method. The method is recommended to be employed preferentally to ML when employing both Likert scales and other similar psychological scales.