Introduction

India launched Expanded Programme on Immunization (EPI) in India in 1978 to control Vaccine Preventable Diseases (VPD). In 1978, EPI coverage was included for six diseases: diphtheria, peruses, tetanus, poliomyelitis, typhoid and childhood tuberculosis. The aim of EIP was to cover 80% of all infants. Subsequently, the programme was universalized and renamed as Universal Immunization Programme (UIP) in 1985. Measles vaccine was included in the programme and typhoid vaccine was discontinued. The UIP was phased in from 1985 to cover all districts in the country by 1990, targeting all infants with the primary Immunization schedule and all pregnant women with Tetanus Toxic Immunization._{1 2}

Figure 1

Table 1: National Immunization Schedule-India

Earlier evaluations of routine immunization in India have shown wide differences between reported coverage by local health agencies compared to evaluated coverage by external agencies. _{3 4 5 6} Such differences are often ascribed to attempts by local health agencies to meet with targets set by themselves or higher agencies. Our study aims at evaluating the quality of sources of data and the reasons for the large variations in a high-risk district in Karnataka, India. The district of Bellary was classified high-risk district because it had 18 confirmed cases of Poliomyelitis in the year 2003 and failure to implement routine immunization services was been given as the predominant reason. Our objective is to evaluate the coverage of immunization in the district of Bellary and further identify data source with highest reliability against a golden standard.

Subjects and Methods

A community-based study of children aged 0–2 years was carried out in Bellary district during the month of September 2007. We used multistage cluster sampling for the selection of sample. We collected complete list of talks (administrative blocks in district) and villages in Bellary district. After considering different sample designs such as simple random sampling, Probability Proportion to Size (PPS) and EPI 30 X 7 cluster method, we chose systematic random sampling as it assured objectivity of houses selection and helpful for planning service provision. _{7 8 9 10 11 12} The study used a multi stage random cluster sample of children in the age group of 0-12 months for collection of data.

All the taluks (administrative divisions) in the district were included for the study. In the first stage, two primary health centers (PHC) were selected in each Taluk based on randomly selected number from random table. In the second stage, in each of the PHC, two villages were randomly selected from the list of villages using random number table. In the third stage, the surveyors would pick up the first house randomly and then would select every 3 ^rd house and conduct interview in 20 houses. First house will be selected based on the random method of picking up houses. The guideline for picking the first house was that pick any house randomly from the micro plan prepared for the purposes of implementing polio special immunization rounds (SIAs). The micro plans for SIAs are updated every round and are expected to be complete for all the villages.

The eligibility criterion for the selection was any house having at least one childbirth in the last two years. Thus the study period will comprise of calendar years starting from 1st April 2005 till 31st March 2005. From the first house, every 3rd house visited in the entire village adding up to 20 houses. If any house does not contain any live births in the past two years, the next house will be selected based on the eligibility criterion.

Figure 2

Figure 1: Scheme followed for selection of samples – Immunization survey

After data check and validation, a total of 1632 parent’s response was included in the survey data. The data corresponds to information about 1632 children residing in Bellary district, Karnataka.

The participation in the study was voluntary and informed consent was taken from the subjects. The analysis was done at University of California, Los Angeles with permission of IRB from University of California Los Angeles for data analysis. (IRB# 007-06-084-02)

Sources of Data: First, the collection of information about immunization history was sought from interviews of parents. Second, The information regarding immunization was also obtained independent of the information through interview, by cross checking the details on immunization cards of children. Third, in the event where immunization cards are not available, the same details were obtained by immunization register maintained in each village by the ICDS worker present in the village. Finally, the information for BCG scar was obtained by cross checking the BCG scar (Gold standard) present generally on the lateral side of the left arm. This too was obtained independent of information obtained from the interviews. The information obtained by first above was classified as parental recall, second and third were classified as card and fourth was classified as scar.

Data entry and analysis:All the information obtained was entered in a master sheet corresponding to the village by the interviewer. The coded information was entered village wise in Microsoft excel. The names and all other personal identification measures were removed from the data before data analysis. Initial data analysis was performed using SPSS for Windows (Rel. 11.0.1. 2001. 17.0, R 2.11. Chicago: SPSS Inc). The output for this paper was generated using SAS software. (Copyright, SAS Institute Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc., Cary, NC, USA.)

Results

Out of the 1630 children were surveyed, we included only 1110 children between 9-24 months of age for our study. This was because we wanted to check complete immunization status in these children and this could have been done only if they have completed nine months of age.

Figure 3

Figure 2: Percentage of Completely Immunized children based on type of Data source

We assessed coverage of complete immunization (immunization against all antigens in UIP) according to the source of data; on the basis of card alone, on the basis of BCG Scar and card and on the basis of parental recall. On analyzing information in cards, complete immunization was found to be 96%, where as on the basis of parents recall alone, the coverage of complete immunization was 87%.

Figure 4

Table 2: Status of Complete Immunization – Bellary district

The difference between two sources of data for complete immunization is significant since the confidence intervals are non-overlapping. To test the reliability of data source further, we wanted to compare the properties of two sources of data (parental recall and immunization card) with that of Gold standard (BCG Scar). Administration of BCG to children provides useful insight since the BCG scar is permanent. Hence the information obtained regarding BCG scar was independently plotted along with information from parental recall and cards. (Figure.3)

Figure 5

Figure 3: Variations in coverage of BCG (against Tuberculosis) based on data source

The coverage of BCG according to presence of BCG scar in the children was 92.4%. From fig 3, we can infer that information from cards overestimates the information compared to gold standard (BCG scar), whereas parental recall is more reliable.

We found that the sensitivity of Card for BCG dose history is similar to that of parental recall (overlapping confidence intervals, Card: 98.9% (97.9-99.5), Parental recall: 97.1% (95.7-98.1). Our results show that parental recall has higher specificity for capturing history of BCG dose. (Card: 11.7 % (3.8-28.4); Parental recall: 39.1% (27.8-51.6))

Figure 6

Table 3: Properties of Parental recall regarding Immunization history (of BCG card and parental recall compared to BCG Scar as Gold standard)

Figure 7

Table 4: Cohen’s Kappa Coefficients (for BCG card and Parental recall compared to BCG Scar as Gold standard)

Discussion

In their analysis of validity of reported coverage in 45 countries, ₁₃Christopher Murray et al have shown that the officially reported data could be misleading in assessing immunization coverage in developing countries. Hence coverage evaluation surveys by external agencies are important for determining immunization coverage. Selection of reliable data source becomes very important in such surveys. There is considerable literature available on evaluation of immunization coverage in different areas of India. _{14 15 16 17 18 19 20 21 22 23} In the recent years, there has been a greater emphasis to rely on the information from immunization cards in developing countries like India. This can be supported by some studies that have inferred that parental recall may not be a very good tool for evaluating vaccination history. _{24 25} However, all such studies were done in developed nations and hence need not apply to settings in developing nations.

Accepting results from any one source of data with wide difference can offer challenge to decision makers as the decisions taken can be completely different. Our study shows that the information from parental recall is close to the data from BCG scar for BCG, and has higher specificity compared to immunization cards. The overestimation of immunization coverage by cards can be due to overestimation are errors due to multiple sources of registration, errors due to duplication of entries, lack of crosschecks, possible errors in data collection and management and exaggerated coverage reports by local health authorities. _{26 27}

Our study compares sources of data with BCG scar as gold standard for vaccination against Tuberculosis, whereas earlier studies have used either immunization cards ¹¹ or prospective history ₂₈ as gold standard. Developing countries like India may not consider either immunization card or prospective history as gold standard to compare other data sources since these countries have ineffective immunization card utilization absence of any reliable registry data. BCG vaccination offers unique opportunity to cross check reliability of other data sources by permanent scarring.

For vaccines other than BCG used in UIP, determining a reliable data source poses greater challenge. This challenge is based on assumption that the vaccine distribution system is efficient and health workers have administered the recommended vaccines. The absence of gold standards such as BCG scar for other antigens makes it difficult for such comparisons. In the absence of gold standards, use of modern epidemiological methods can be made for estimation of immunization coverage_{29 30 31 32 33} .

We had checked B.C.G scar prior to (on the first contact of child) and independently of parental recall for BCG vaccination. Hence, we think that recall bias due to association of scar and parental recall might have significantly reduced.

We infer that in addition to the existing literature, there is a need of re-examining the stand several agencies of endorsing immunization cards for evaluation of immunization coverage in Developing Countries. Most importantly, in the absence of gold standards for other antigens and absence of reliable system for use of immunization cards, parental recall might be the best available option for nearly reliable source of information in developing countries like India.