SciELO - Scientific Electronic Library Online

 
vol.39 número118DESIGUALDAD EDUCACIONAL EN CHILE: GEOGRAFIA Y DEPENDENCIAINDICE ALFABETICO POR AUTORES DE ARTICULOS PUBLICADOS EN CUADERNOS DE ECONOMIA DURANTE EL AÑO 2002 índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

Compartir


Cuadernos de economía

versión On-line ISSN 0717-6821

Cuad. econ. v.39 n.118 Santiago dic. 2002

http://dx.doi.org/10.4067/S0717-68212002011800006 

Cuadernos de Economía, Año 39, Nº 118, pp.423-454 (Diciembre 2002)

 

THE PERFORMANCE OF PRIVATE AND PUBLIC
SCHOOLS IN THE CHILEAN VOUCHER SYSTEM

Claudio Sapelli*
Bernardita Vial**

ABSTRACT

We estimate the average treatment effect (ATE) parameter and treatment on the treated (TT) for those students effectively choosing a private voucher school. With detailed individual data for both the school and the family of the student we are able to control for factors not used before in the literature. Controlling for the selection bias caused by school choice, we estimate a small ATE but a substantial TT. The paper concludes that the voucher system works to provide a better education to those that use it.

RESUMEN

En este trabajo estimamos el efecto de tratamiento promedio (average treatment effect parameter, ATE) y el efecto del tratamiento en los tratados (treatment on the treated parameter, TT) para aquellos alumnos que escogieron establecimientos subvencionados privados. Debido a que contamos con información a nivel individual de los establecimientos y de la familia de origen de los estudiantes, podemos controlar por características que no habían sido utilizadas en la literatura previa. Controlando por la autoselección de los alumnos en los establecimientos educacionales, encontramos un ATE pequeño y un alto TT. Este último es el parámetro de interés convencional utilizado para evaluar este tipo de programas. Se concluye que el sistema de subvenciones es efectivo en proveer una mejor educación para aquellos alumnos que escogen los establecimientos privados subvencionados.

1. Introduction

The introduction of vouchers in the financing of social services such as education or health continues to be an area of much debate. The advantages and disadvantages of vouchers have been analyzed in detail at a theoretical level. The lack of sufficient (in both number and length) empirical evidence on the effects of vouchers has hindered the advance of knowledge in this area. In Chile a voucher system was introduced in education in 1982. This is the only education voucher system established at a national scale and that has data for more than 15 years. Its evaluation is of great interest to evaluate the arguments of the theoretical literature on the advantages and disadvantages of the voucher system (and on the proper design of a voucher system, an issue whose importance is not sufficiently emphasized in the literature).

The literature that has been developed on the Chilean case is small, and in many cases faces methodological and/or data limitations. In the discussion of the literature found in section 3 below, we describe the methodological limitations. Regarding the data, up until late 1999, there was no individual data available. Papers had to use the school as a unit of study, because the results of the SIMCE test, the test used to measure achievement, were made public as the average per school and the individual data was not available. Additionally, these studies lacked good information on the socioeconomic characteristics of the students (and their families). Proxies were used that implied averages at the school or county level (averages of socioeconomic characteristics of the county in which the school is located). Hence each school implied one observation. We are now able to analyze how appropriate these proxies were.

In this paper we evaluate the performance of private and public schools in the Chilean voucher system, using the new individual data available for the test taken in 1998 to the second grade of secondary school. A parallel survey produced the data on individual socioeconomic characteristics that can be matched to all test results. We also use information on the characteristics of the schools and the centrally designed programs in which they participate, and on the transfers they receive from municipalities.

2. Description of the Chilean Education System

Education in Chile is provided by three types of schools: municipal schools (MUN), private subsidized schools (PS), and private non subsidized (PP, for its Spanish name, particulares pagados). MUN schools have little more than one half of total enrollment, PS schools have about one third, and PP schools have about one tenth. These percentages vary in Santiago, where PS and PP schools have a larger share.

In 1988 the System of Measurement of the Quality of Education (SIMCE for its Spanish Sistema de Medición de la Calidad de la Educación) was established to generate information on the quality of education in different schools. The system consists of tests in the areas of language, mathematics, social sciences, and science. For language and mathematics, all schools are tested in fourth and eighth grades (primary school). In 1993 a test started to be taken to students in the second grade of high school. The raw scores show that PP schools have the highest test scores, while MUN schools have the lowest.

Although the evolution of the test since its inception would be interesting, and many analysts make much of its evolution, that comparison is meaningless since the test is not designed to be comparable1. The test was not equated until 1997. The test was administered by the Universidad Católica up to 1992 and then was transferred to the Ministry of Education.

MUN schools may be of two types: i) those dependant of a department within the Municipality (called Departamentos de Administración de la Educación Municipal (DAEM)); or ii) autonomous, constituted as a Corporation, where the Mayor is the head of the Board of the Corporation. The Corporation does not belong to the municipality and hence the statute of municipal public servants does not bind its personnel. Most MUN schools are controlled by the DAEM. Since 1988 the creation of new corporations was prohibited. The PS schools include: i) religious schools; ii) for profit lay schools; and iii) not for profit lay schools.

PP schools finance themselves 100% from tuition payments, while PS and MUN schools were usually free and financed themselves 100% from the fiscal budget at the beginning of the voucher system. The fiscal funds were distributed according to a per capita subsidy. The purpose of this subsidy is to promote competition among schools financed from the fiscal budget to attract and retain students, since the amount of money they receive depends directly on enrollment (the subsidy is paid every month to the school according to average attendance during the previous three months). The subsidy is expressed as a multiple of a unit called Unidad de Subvención Educacional (USE). The multiple varies according to different characteristics of the school (see Table 1).

The per capita subsidy does not depend on student's characteristics. If the education of students with lower initial human capital is costly, it implies that they receive a lower net subsidy. In other words, poorer families generally receive a lower net subsidy. This design of the subsidy implies that poorer families are the less benefited from the voucher system.

Some current characteristics of the Chilean voucher system imply that we are not facing the typical voucher design, in which schools compete for a similar per capita subsidy and then have to operate with the budget the sum of the subsidies provides. Though it is not the purpose of this paper to analyze these issues, they are relevant to the design of the empirical strategy and to interpret the results we obtain from the evaluation of the system. We will discuss briefly those issues.

On the regulatory side, an important problem is that since 1990 there is a "Teachers Statute". The statute imposes the centralized negotiation of wages, that implies schools lose control over their wage budget. The statute is pertinent for municipal schools, and partially to particular subsidized schools, since it imposes a minimum standard.

Concerning the budget constraint the schools face, most MUN schools receive subsidies from the municipalities if the voucher is not enough to cover the entire budget. The average monthly transfer was US$9.5 per student in 1998, but some municipalities did not transfer funds at all to the schools, while other transferred a large amount of money (the highest was US$298.3). That is the reason why we claim that some municipal schools face a "soft budget constraint".

In addition, at present education policy is directed to increase the quality of output by increasing inputs, through centrally designed and financed programs. It includes several specific programs in different areas (the most important of which is the mandatory extension of the school day). This strategy involves the construction of a structure of incentives parallel (and sometimes opposed) to that of the voucher system. These incentives in many respects may neutralize the effects of the voucher system, since resources are allocated to the schools and not to students2. The money channeled through these programs also increases the school's budget, beyond the voucher funding.

Since 1993 there are two new sources of financing for the PS schools: the system of shared financing or financiamiento compartido and the possibility of deducting educational donations from taxes. The financiamiento compartido system consists of the possibility of parents making direct payments to the school, without the school losing 100% of the subsidy, as was the case until then. Once a school chooses to participate in the system it can charge up to four USEs (approximately US$88), and the per capita subsidy is reduced according to the average fee paid by the student. The discount corresponds to: 10% of the portion of the fee between one half and one USE, 20% of the portion of the fee between one and two USEs, and 35% of the portion of the fee between two and four USEs. As B. Vial (1998) shows, this system unambiguously increases total financing to education, reducing the distortions created by the "either/or" option between fully paid and free schools. The system of financiamiento compartido was a huge success and about 40% of PS schools (representing about 65% of total PS enrollment) had shifted to the system by 1996.

Enrollment has proved that people find the system is successful in providing what they want for their children. Not only one quarter of the students are currently in PS schools with financiamiento compartido, but if one goes a bit further back, the changes in the structure of the system are startling. In 1981, 80% of the students were in MUN schools and by 1997, only 55% of the students where in MUN schools.

Certainly, while the introduction of centrally financed programs may neutralize the positive effects of the voucher system, during the last few years the design and operation of the voucher system has also been improved: the increase in the real value of the subsidy, the development of financiamiento compartido, the publicity of the SIMCE results, are all developments that improved the operation of the voucher system.

Concluding, several problems in the operation of the Chilean voucher system need to be taken into account in the evaluation of the Chilean voucher system. We are not facing the typical voucher design in which schools compete for a similar per capita subsidy and then have to operate with the budget the sum of the subsidies provides. That is only true for PS schools without financiamiento compartido (FC). MUN schools receive extra budget from the municipalities and the Ministry. PS schools with FC receive extra budget from the parents. Though the latter are directed to those schools parents think deserve support, the first are sometimes directed to funding schools that have lost many students to competition, therefore negating the objectives of the system. Currently, schools that fail due to lack of demand or bad administration are subsidized.

Our empirical strategy consists in the estimation of treatment effects associated to school dependence (private or public). We try to estimate the gain associated with the attendance to private subsidized schools, versus municipal (or public) schools, for students with the same characteristics. We are not interested in finding the reason why there is a gain (or not) associated with school dependence, but on the existence of that gain. Therefore, in the estimation we control by student characteristics3, but we do not control by school characteristics. We do not attempt to estimate an education production function, and we are not interested in answering if students who attend to schools with the same characteristics (inputs, incentives and technology) obtain the different results. We are interested in answering if students who attend to schools with different dependence (private or public) obtain different results, precisely because the schools have different characteristics.

3. The Empirical Literature

The literature up to 2000

Up until 2000, the number of studies was small and used different independent variables that complicates extracting overall conclusions. Tessada (1998) reviews the five production function studies done in Chile (Rounds (1996), Aedo and Larrañaga (1995), Aedo (1988), Mizala and Romaguera (1998), Gallego (1997, 2001)), up until then. Of all these the one that had the most influence is Mizala and Romaguera, later published in the JHR (Journal of Human Resources) (2000).

The variables that are consistently significant are those related to the children's environment, such as parental education, the vulnerability index (an index constructed by Junta de Auxilio Escolar y Becas, JUNAEB and principally related to family income), and dummies that represent the different regulatory frameworks in which schools work: municipal schools (organized as corporations or directly dependant on the municipality), private subsidized schools, private subsidized schools with financiamiento compartido, religious schools, and non subsidized private schools. The results indicate that the poorest performance is that of municipal schools that are directly dependant of the municipality (DAEMs), and the best that of PP schools (and this is still true once one extracts the influence of parental characteristics and controls for selection bias). Aedo (1997), in particular, provides evidence of the importance of appropriately modeling the supply side, i.e. that there is great heterogeneity of incentives within groups of schools usually analyzed as a homogeneous. The differences also include different budgets, as mentioned above.

Are private subsidized schools more effective?

Up until 2000, the literature concluded that, if properly controlled by pupil's background, SIMCE scores did not differ between municipal and private subsidized schools. These result influenced educational policy for many years. For example, policy papers such as Cox and Lemaitre assert that, if pupils background and the conditions in which the schools operate are controlled for, SIMCE results are the same. If confirmed, the result that municipal and PS schools do not differ in the quality of their product would be earth shattering, since municipal schools face a very different incentive framework than PS schools (such as soft budget constraints). If they effectively produce the same product, then it implies that the additional funds received by municipal schools really matter, or this incentive framework is basically irrelevant. This is the reason why many analysts take this result as implying that the voucher system is an inadequate system to achieve better education.

Mizala and Romaguera (1997) argue against their results necessarily being read as implying that the voucher system had failed. They argue that freedom of choice should be factored in as an advantage of the voucher system, even if SIMCE results where not changed. Also, the fact that SIMCE results are the same may imply that both MUN and PS school results may have increased pari passu as a result of competition.

Moreover, it is not clear whether the conclusion that "if pupils background and the conditions in which the schools operate are controlled for, SIMCE results are the same" is a consistent result in the literature. McEwan and Carnoy (1999), and Bravo, Contreras and Sanhueza (1999) found that the difference in scores between PS schools and MUN schools is statistically significant, even controlling for socioeconomic background (for 1997 data). However, they argue this is due to the performance of catholic schools (or schools that were in operation before the introduction of the voucher system). McEwan and Carnoy argue that, since most catholic schools were in operation before the voucher system was introduced, and in the US it has been found that catholic schools are more efficient (even though no voucher system exists), they should not be considered when evaluating the merits of the voucher system. Once one drops these schools, they find that the other PS schools perform worse than MUN schools.

Bravo, Contreras and Sanhueza (1999) argue along the same lines, but use a dummy to symbolize schools that existed in 1982 when the voucher system started, and those that did not. Schools that existed before 1982 have better scores. They conclude, "the superiority of the private subsidized schools is due rather to pre-reform or religious subsidized schools." The table that follows this sentence actually says something different. Pre reform schools are significantly better than post reform schools in 1989, but the difference halves in 1993 and disappears in 1997. In 1997 both pre and post reform schools are significantly better than municipal schools. Hence the results in the literature were converging into finding small but significant differences in scores between PS and MUN schools.

The most recent literature

The literature reviewed above had the following limitations: 1) the papers did not control for selection bias with the possibility of a biased estimation of the effects if the selection of school was endogenous and depended on variables included in the regressions; 2) the effect that was estimated is an homogenous treatment effect independent of student characteristics, when there is the possibility that different types of schools are more effective for certain students, and hence the average treatment effect may be misleading; and 3) there is an implicit assumption that all voucher schools operate with the same budget, when, as was described above the budget of the voucher schools differs significantly (due to centralized programs, transfers from the municipality or fees), hence there is a need to give adequate structure to the supply side4.

Selection bias

All empirical work in the area before 2000 is subject to the criticism of not taking into account the existence of selection bias. Data to correct for it was not available until 2000. The issue is present, for example, when attempting to interpret the difference in raw test data between different types of schools. Two effects can be confused: that of a better quality entering student due to more "in-house" production of human capital, and that of a better student-school match due to parents with higher human capital to find a better match. There is a need to use econometric methods to separate the value added by the school from that due to the better quality of the input (the student).

The literature up to 2000 regresses the test results on family characteristics and type of school. If the differences in raw scores is all explained by family characteristics, then the conclusion was that schools do not differ much in the quality of education. However, since the decision of type of school is endogenous, if one does not control for its endogeneity, one may make inferential mistakes (Card and Krueger 1992 actually argue, for the US, that when adequate controls for previous human capital are used, they explain most of the difference in performance). Family characteristics influence test scores through two channels, the selection of the type of school, and the learning ability of the child. If one regresses test scores on family characteristics, one is not able to determine which channel is most important. It could be possible to explain 100% of raw scores by parental education and that would not mean the quality of the schools is similar. If all the influence of parental education is channeled through the choice of school, then what makes the difference is the school. Only if most of the influence were through the learning abilities of the child, would the conclusions in the Chilean empirical literature be correct. Mizala and Romaguera and Reinaga (1999) apply the methodology to separate these two effects to Bolivia. When they do not correct for sample selection, they find that parental characteristics explain two thirds of the raw test difference between public and private school test scores. When they correct for sample selection, they conclude that half of the difference explained by parental or family characteristics is through school selection, hence implying the quality of the student explains one third and not two thirds of the raw score differences. The other third is due to better selection of schools by highly educated parents.

In the Chilean literature, Contreras (2001) and Tokman (2001) introduce selection bias and find significant differences in scores between MUN and PS schools. Contreras estimates a LATE with an equivalent of the SAT (the Prueba de Aptitud Académica or PAA), which both due to its methodology (instrumental variables) and the use of the PAA instead of the SIMCE, is unique and cannot be compared to the rest of the literature. Mizala and Romaguera (2001) use the same data set as we use in this paper, but report they are unable to control for selection bias (due to lack of convergence). However, even when not controlling for selection bias they find that the SIMCE scores do differ significantly between MUN and PS schools. They attribute the differences between these results (for secondary education) and their previous study (for primary education) to differences in the type of schools and students at both levels.

Absolute advantage versus comparative advantage (one or two abilities?)

The literature up to 2000 included the estimate of an average treatment effect in a model where schools had absolute advantages to teach all students. Tokman (2001) includes an estimate of the average treatment effect (ATE)5 for different categories of students, and finds that there could be evidence for comparative advantage. A positive ATE onaverage could be composed of a positive ATE for some students and a negative ATE for others. However, she does not take into account the effects of differences in the supply side that could also explain her results (for example, different incentives or budgets of private subsidized schools with and without FC, of MUN schools with and without subsidies, or MUN schools functioning in a regime of competition or monopoly).

How much structure to the supply side?

All the studies in the literature implicitly assume that MUN and PS schools are an homogeneous pool that function subject to similar incentives and similar budgets. However, this is not true, and there are several differences that should be controlled for, or at least taken into account when interpreting the data. These include: the differences in the budget of MUN schools according to the municipality where they are located, or their inclusion in Ministry of Education programs; the extra subsidy received by rural schools; the extra budget received through financiamiento compartido; and whether schools compete with others in the same municipality, or have monopoly power due to their location.

Recently, Gallego (2001) has provided evidence of the importance of the industrial organization aspects of the problem, showing evidence that MUN schools perform better when they face competition from PS schools.

4. Treatment Effects: Private Subsidized versus Municipal Schools

Previous evaluations of the Chilean education voucher system (for example, see Mizala and Romaguera, 2000) commonly use a model like the following:

Y= Xb + aD + u

Where Y is the test score, X is a vector of exogenous personal characteristics that affect test scores (including family background)6, and D is a vector of dummy variables indicating attendance to private subsidized or public schools.

The test score gains associated with different school types are measured by the estimate of the dummy's coefficients. This general model had two important problems: (a) it was very restrictive, because it assumed that the score gain is constant across individuals with different observable characteristics, and (b) it does not consider self selection. We attempt to correct both problems, using a more general model and correcting for selection bias. We also attempt to better model the supply side, by including the transfers to municipal and PS schools.

Selection Model 7

We use a generalized Roy model to characterize the self selection process. Given school characteristics, potential test scores are determined by student's characteristics (including family background). However, municipal and private subsidized schools exploit differently those characteristics, so potential test scores in each type of school are different. We assume that parents choose the type of school that maximize their utility, which includes the student's test score as an argument.

Potential test scores are:

We define a dummy variable: D=1 if the student chooses a private subsidized school, and D=0 if the student chooses a public school. The selection rule is defined as follows:

We assume that the error terms are normally distributed, so

We only observe test scores in each type of school for students who actually attend this school. The mean of test scores observed in private subsidized schools is:

The mean of test scores observed in municipal schools is:

In the previous expressions l corresponds to
f(.)
 

, where f (.) and F(.) are the density and the cumulative
1 - F(.)  

distribution function of a standard normal variable respectively. We estimate b0, b1, s0, s1, r0 and r1 by Maximum Likelihood.

We estimate the Average Treatment Effect (ATE) and the effect of Treatment on the Treated (TT) associated with private schooling on language achievement8 9. When we consider the gain associated with private subsidized schooling versus public schooling, the ATE is defined as the expected gain in test scores associated with attendance to a private subsidized school (versus a public one) for a randomly chosen individual. In turn, the estimated effect of TT corresponds to the expected gain in test scores for those students who actually attend private subsidized schools (versus their expected test scores in a public one). Standard errors are computed using parametric bootstrapping10.

ATE corresponds to:

Where n is the sample size. In turn, TT corresponds to:

Where D=1 if Y1 > Y0, and nt is the number of observations with D=1.

To evaluate the policy in question, the conventional parameter of interest is TT11.

For the estimation of treatment effects, we run separate regressions for Private Subsidized and Municipal schools, controlling by income group, mother and father education, and a dummy indicating that the child comes from an indigenous family12. In the selection equation we include the previous variables, and in addition we include the number of municipal and PS schools by km2, and the ratio of the number of students in PS and PS+MUN schools13, by geographical area. These variables are included to capture the effect of school availability on the school selection. For a description of the variables, see Appendix 2.

Exclusion restrictions

As we noted before, in the selection equation we include the number of municipal and PS schools by km2, and the ratio of the number of students in PS and MUN+PS schools by geographical area as exclusion restrictions. We define geographical areas according to the province where the student attend to school, except for Santiago. In this case we separate by electoral district, due to the large size and population density of the province. The average size of provinces is 17.232 km2 (and the standard deviation is 21.219).

The crucial assumptions behind those exclusion restrictions are that mobility of students across provinces (and electoral district in Santiago) is low, and that families do not take into account the school quality in the province (or electoral district in Santiago) when they choose area of residence14. The first assumption means that the student almost always attend to a school inside the area of residence, i.e., the area where they attend to school is almost always the area of residence. This assumption is not opposed to the evidence about mobility of students across municipalities, since provinces (and electoral districts) are groups of municipalities.

As to the second assumption, Coloma and Edwards (1997) argue that social housing reduces residential mobility. According to the 1998 Encuesta de Caracterización Socioeconómica Nacional, CASEN, 30% of the second grade students in High School who attend subsidized schools, live in social-program houses. Thus, according to Coloma and Edwards, for this 30% of the students, residential mobility is low. This issue, added to the low residential mobility due to occupational reasons, justify our second assumption.

5. Description of the Data

In this paper we evaluate the performance of the Chilean voucher system, using the new individual data available for the SIMCE test taken in 1998 to the second grade of secondary school. We also use a parallel survey with data on individual socioeconomic characteristics that can be matched to test results, and information on the characteristics of the schools and the centrally designed programs in which they participate. On the individual characteristics, we have some measures for family background, like parental education and family income, and student specific variables, such as if he/she have repeated a course. Table 2 presents a summary description of the variables used in the estimation.

6. Results and Discussion

The results on treatment effects for language test scores are summarized in Tables 3 to 6 (estimated standard errors are in parenthesis). In the tables we report the estimated unconditional average treatment effect (ATE), and the unconditional effect of treatment on the treated (TT). To approximate unconditional treatment effects, we average the estimated individual effects for the complete sample (first column) and for each income group (in columns that follow)15. Regression results are shown in Appendix 1 16.

The regression results show that the family background variables are important in both explaining the SIMCE test results and the selection of PS or Municipal schools. An important result is that self selection is an important issue in the estimation. If we do not take into account the issue of self selection and perform an OLS estimation of the difference in SIMCE scores between MUN and PS schools (see Table 3), controlling by family characteristics, we obtain a difference of about 9.6 points in favor of PS schools17. This Average Treatment Effect (ATE) corresponds to 0.2 Standard Deviations (of Test Scores).

When taking into account self selection (see Table 4) we obtain an estimate of ATE that is negative (ATE=-2.3) and a lower estimate of the effect of TT (TT=6.9). As we noted before, the conventional parameter of interest for the evaluation of social programs is the effect of TT. The estimated TT is lower (than the OLS result), but still statistically significant. It corresponds to 0.15 standard deviations, an effect considered slight to moderate in the literature (effects of 0.1 standard deviations are considered slight; 0.2 to 0.4 are moderate; and 0.5 or higher are large; see Greene, Peterson and Du, 1997).

As we can see in Table A1.2 (Appendix 1), in the estimation of the selection model we obtain positive and significant selection coefficients in the private subsidized sector. Those results are consistent with absolute advantages.

An issue that is not considered in this estimation, is that some schools receive financial assistance from the government above and beyond the value of the voucher. Thus, not all the schools considered in the estimation are working with the same per capita subsidy. To try an solve this problem, we separate the data set according to the geographical area. The criteria for separating geographical areas is the per capita subsidy received.

Taking into account the differences in per capita subsidies: estimation by transfer quintile

We run 10 separate regressions, according to the level of local and central government per capita funds received by subsidized schools in addition to the voucher. For this purpose we use SUBDERE (Subsecretaría de Desarrollo Regional) information on municipal funds transferred to municipal schools, and MINEDUC (Ministerio de Educación) information on school participation in the two most important central government programs for high school (Montegrande and PME, see Annex 1). We separate the data set by province of residence, except for Santiago. In this case we separate by electoral district, due to the large size of the province. Then we proceed as follows:

i) We estimate the average municipal funds transferred to municipal schools per student in each geographical area (using the information on total funds transferred, and the total number of students by municipality).

ii) We use information on the percentage of students attending Montegrande and PME schools by geographical area, to estimate the funds received by municipal and PS schools per student in each area (using the average per-student transfer in Montegrande and PME programs).

iii) We construct an index according to the additional funds received by municipal schools.

iv) We sort the geographical areas according to the index described above, and construct quintiles (each quintile with approximately 20% of high school students).18

In this classification, the first quintile gives us the most pure voucher comparison, since it includes schools that work approximately within the voucher value19. Those in the fifth quintile are those municipal schools that receive the most transfers (on average 71% more than the value of the voucher, ranging from 50 to 200% of the voucher, see Table A2 in Appendix 2).

The weighted average (by number of students) of annual transfers per student, is US$48 in the first quintile; US$64.9 in the second; US$95.6 in the third; US$135.4 in the fourth; and US$278.8 in the fifth quintile. This implies, using US$393.5 as the annual standard voucher transfer in 1998, an increase that ranges from 12% in the first quintile and of 71% in the fifth20.

When we run regressions by quintile (see Tables 5 and 6), we obtain the result that students in the first 3 quintiles have TT results that are positive and statistically significant (i.e. the students that attend PS schools have better scores in PS schools). Even ATE estimates are statistically significant in the first 3 quintiles on average. However, for the fifth quintile we obtain ATE and TT estimates that are substantially negative. That is, the municipal schools that receive the most transfers perform substantially better than PS schools. This implies that the results shown previously are a combination of TT that are positive with this large and negative TT, hence we are underestimating the TT effect when schools have similar per capita subsidies.

The case that most approximates a pure voucher system, that of the first quintile, shows a TT that is larger than when no consideration was made of transfer differences. For language tests we obtain a difference of 23.7 (0.5 SDs, an effect considered large in the literature). This result shows the importance of attempting to appropriately incorporate the differences in the supply side. In the second and third quintile we still find an effect of TT that is considered moderate in the literature.

It is also interesting to note that when including the differences in per capita subsidies for municipal schools, the result that we are in the presence of "absolute advantages" holds. The most notable case is possibly that of quintile 5, in which there is absolute advantage but the best schools (the municipal schools in this case) receive the worst students. This result needs further research and could possibly be linked to the selection criteria used by these municipal schools that have queues to enter.

As Figure 1 shows, for municipal schools, the distribution of test results in the fifth quintile are substantially different from the rest even when not controlling for anything. This is the effect we are capturing when we separate by transfer quintiles in the estimation. It is important to note, however, that this result do not imply that more resources unambiguously increases test scores: in the fifth transfer quintile not only the average test score is higher (in municipal schools), but also the variance. The coefficient of variation goes from 2.3 in the first quintile, to 9.0 in the fifth quintile. Therefore, some schools in the fifth quintile obtain higher tests scores than those in the first quintile, but other schools in the fifth quintile obtain lower test scores.

7. Summary and Conclusions

The results of the empirical work show that there are important gains in test scores associated with attendance to private subsidized schools. When we do not take into account differences in per capita subsidies, those differences are significant when we consider the expected gain for those students who actually attend private subsidized schools (TT), the conventional parameter of interest in programme evaluation. In contrast with some previous estimations, those gains do not disappear when we control for personal and family characteristics of the tested students. Also, the estimates of the TT effect are very similar across income groups.

Since these results do not address the problem of modeling the supply structure, in the paper we attempt to partially control for these differences by incorporating the supplements that municipal schools receive from municipalities or the Ministry. When we do so, we obtain a higher ATE, and a higher TT in the first three quintiles (with low supplementary budget), and a lower ATE and TT in the fourth and fifth quintiles. In the first quintile, where municipal and PS schools operate with a similar per capita subsidy, we obtain an estimated TT in the order of 0.4 to 0.5 standard deviations, wich is a large effect.

This result leaves us with an evaluation issue that will have to be taken up in the future. If municipal schools receive a budget that is on average 70% larger, and fluctuates from being 50% to 200% larger, they perform on average substantially better than PS schools (a gain of 80-90 points in results that are on average of 250 points). However, when municipal schools receive less than 27% supplementary budget (i.e., for 60% of the students), the results of PS schools are significantly better. In the case of pure budget schools, that is, with low budgetary supplements, PS schools provide a gain of approximately 23 points, for those attending PS schools. This gain goes from 21 points for low income students, to 35 points for high income students.

The voucher system in Chile is not a pure version since voucher schools face differences in their per capita budgets and the relationship these budgets have with attendance, implying differences in the incentive structure they face. Municipal schools have several alternative sources of financing to the voucher. Also, the proliferation of non-portable funding to schools diminishes competition. This is aggravated by the fact that the budget per student does not vary according to any accepted criterion. For example, if it varied inversely with parental education, the way the value of the voucher is set would not limit the choices of the group of students that is less endowed with human capital from the household, as it does now. These issues represent a problem for the evaluation of the system, since the differences in incentives and/or budget need to be taken into account, but have not yet been studied. We hope our paper is a first step in this direction.

Keywords: Education System, Nonparametric Methods, Program Evaluation and Vouchers
JEL Classification: C14, I22, I28

1 Quoting Eyzaguirre and Fontaine (1999): "...la comparación de las pruebas anteriores a 1997 no sería metodológicamente correcta".

2 The strategy that is currently being followed is based in part on a negative evaluation of Chile's experience with vouchers. This evaluation is quite clear in the paper by Cox and Lemaitre (1999) "Market and State Principles of Reform in Chilean Education: Policies and Results", in which there is a strong and well-structured defense of the current strategy.

3 We control by family income, parental education and indigenous family.

4 Other problems include use of inadequate controls or of the market structure in which schools compete, Gallego (this issue) looks at the second issue.

5 The treatment here would be to attend a private subsidized school.

6 Some studies also control for school characteristics, but it is not correct, because the realization of D determines school characteristics (for an interpretable definition of the treatment effect we need the "no feedback condition"). See Heckman, LaLonde and Smith (1999).

7 We also estimated a multinomial-OLS selection model, considering PP, PS and Municipal schools, and we obtained similar results. However, we present the results for the standard normal selection model because the multinomial-OLS selection model requires distributional assumptions that differ from those of the Roy Model, and requires the Independence of Irrelevant Alternatives. Moreover, the distribution of characteristics of students attending PP schools differs substantially from those of students attending PS and MUN schools (there is no common support).

8 We obtained similar results when we estimated treatment parameters for Math test scores.

9 Quote from Heckman and Vytlacil (2000 a), University of Chicago, December. "Accounting for individual level heterogeneity in the response to treatment is a major development in the econometric literature on program evaluation. A substantial body of empirical evidence demonstrates that econometric models fit on individual-level data manifest heterogeneity in treatment effects that is present even after conditioning on observables. An important distinction is the one between evaluation models where participation in the program being evaluated is based, at least in part, on unobserved idiosyncratic responses to treatment and models where participation is not based on unobserved idiosyncratic responses. This is the distinction between selection on unobservables and selection on observables. The validity of entire classes of evaluation estimators hinges on whether or not they allow agents to act on unobserved idiosyncratic responses. In a wide variety of applications, the available evidence suggests that not only are ex post (post enrollment) responses heterogeneous, but that ex ante decisions to participate in programs are based, in part, on these heterogeneous responses. (James Heckman and Edward Vytlacil, 2000b). An important consequence of these findings is that in the presence of selection on idiosyncratic treatment effects, no single "effect" describes a program or intervention. A variety of treatment effects can be defined that depend on the conditioning sets used to define "the" effect. Picking persons at random and entering them into a program and comparing their mean outcomes with those of persons denied access produces the Average Treatment Effect (ATE). Picking persons at random who go into the program and comparing their average outcomes with those of the same type of people denied access to the programs defines the parameter Treatment on the Treated (TT).

10 See Heckman, Tobias and Vytlacil (2000).

11 See, for example, Heckman et al. (1997) and Heckman et al. (1998).

12 Children from indigenous families may be disadvantaged because of their use of Mapudungun, an indigenous language very different from Spanish.

13 To estimate the school size, we used information on the number of students in first grade (of high school), in order to avoid a bias due to different dropout rates.

14 Variables used as exclusion restrictions should influence school choice, but not the test scores. Therefore, the crucial assumption is that those variables are not correlated with unobservables that affect test scores.

15 These income groups are defined in Chilean pesos, and correspond approximately to the following brackets in US$: less than 217; 217-434; 434-651; 651-1302; more than 1302.

16 Note that in tables A1.2 and A1.3 the coefficients for the selection equations for the PS and MUN columns are the same in absolute value, but with the opposite sign. This is so because for PS, selection occurs when the individual chooses a PS school (as opposed to a Municipal school), and for MUN, selection occurs when he or she chooses a municipal school (as opposed to a PS school). Therefore, the PS column shows the results for the estimation of the probability of choosing a PS school, and the MUN column shows the opposite.

17 Since for OLS estimation we assume there is no selection bias, ATE and TT are identical.

18 We also construct an index according to the difference between additional funds received by municipal schools and those received by PS schools. The results obtained with this index are similar to those presented here.

19 When we aggregate across geographical areas, we obtain that municipal schools always receive more additional funds from the government than PS schools. Therefore, in the first quintile, we are not making a comparison between municipal schools receiving low additional funds, with PS schools receiving a lot of them.

20 The transfers increase the budget by 12% in quintile 1, 16% in quintile 2, 24% in quintile 3, 34% in quintile 4 and 71% in quintile 5.

21 The central government programs not included to construct the index, such as MECE and ENLACES, have an extensive coverage in the country, and the funds received per student are very low. Hence, their inclusion does not alter the results.

REFERENCES

Aedo, C. (1998), "Diferencias entre escuelas y rendimiento estudiantil en Chile", cap. 2 en "La Organización Marca la Diferencia: Educación y Salud en América Latina", Red de Centros del BID, BID.         [ Links ]

Aedo, C. (1996), "Calidad de la Educación y Elementos de Mercado", capítulo 2 en Educación en Chile: Un Desafío de Calidad, ENERSIS.         [ Links ]

Aedo, C. and Larrañaga, O. (1995), "Educación privada versus pública en Chile: calidad y sesgo de selección", mimeo, Programa de Postgrado en Economía, ILADES/Georgetown University.         [ Links ]

Bravo, D., Contreras, D., and C. Sanhueza (1999), "Educational Achievement, Inequalities and Private/Public Gap: Chile 1982-1997". Working Paper, Department of Economics, Universidad de Chile, July.         [ Links ]

Card, D. and A. B. Krueger (1996), "School Resources and Student Outcomes: An Over-view of the Literature and New Evidence from North and South Carolina", Journal of Economic Perspectives, Fall, Vol. 10 (4).         [ Links ]

Card, D. and Krueger, A. B. (1996), "Labor Market Effects of School Quality: Theory and Evidence", in Does Money Matter? The Effect of School Resources on Student Achievement and Success, Brookings Institution Press, edited by Gary Burtless.         [ Links ]

Carnoy, M. (1997), "Is Privatization Through Education Vouchers Really the Answer? A Comment of West", WBRO February.         [ Links ]

Coloma, F. and G. Edwards (1997), "Análisis Económico de la Localización de Viviendas Sociales", Revista Estudios Públicos, Nº 68.         [ Links ]

Contreras, Dante (2001), "Evaluating a Voucher System in Chile. Individual, Family and School Characteristics", Documento de Trabajo No. 175, Facultad de Ciencias Económicas y Administrativas, Universidad de Chile, marzo.         [ Links ]

Cox, C. and M.J. Lemaitre (1999), "Market and State Principles of Reform in Chilean Education" chapter 4, in Chile: Recent Policy Lessons and Emerging Challenges, WBI Development Studies, World Bank.         [ Links ]

Eyzaguirre, B. y L. Fontaine (1999), "¿Qué nos dice el SIMCE de Octavo Básico?: Análisis y Perspectivas" en Serie Documentos de Trabajo Nº 294, Centro de Estudios Públicos, mayo.         [ Links ]

Gallego, F. (1997), "La Economía de los Resultados del SIMCE: Un Análisis de la Educa-ción Subvencionada en Chile", Seminario de Título, Instituto de Economía, PUC.         [ Links ]

Gallego, F. (2001), "Incentivos, Recursos y Calidad de la Educación: Teoría y Evidencia para Chile". Tesis para optar al grado de Magister en Ciencias de la Economía, Instituto de Economía, PUC.         [ Links ]

Greene, Peterson and Du (1997), "Effectiveness of School Choice: The Milwaukee Experiment", Occasional Paper 97-1, Department of Government, Harvard University.         [ Links ]

Grogger, J. and D. Neal (1999), "Further Evidence on the Effects of Catholic Secondary Schooling", Working Paper.         [ Links ]

Hanushek, E. (1986), "The Economics of Schooling: Production and Efficiency in Public Schools", Journal of Economic Literature, 24, 1141-1177.         [ Links ]

Hanushek, E. (1995), "Interpreting Recent Research on Schooling in Developing Countries", World Bank Research Observer, August.         [ Links ]

Hanushek, E., S. Rivkin and L. Taylor (1996), "Aggregation and the estimated effects of school resources", The Review of Economics and Statistics, 78, 4, November, pp. 611-27.         [ Links ]

Hanushek, E. (1996), "School Resources and Student Performance" en Does Money Matter? The Effect of School Resources on Student Achievement and Success, Brookings Institution Press, edited by Gary Burtless.         [ Links ]

Hanushek, E. (1998), "Conclusions and Controversies about the effectiveness of School Resources", en Economic Policy Review, Federal Reserve Bank of New York, March, Vol. 4 (1).         [ Links ]

Hanushek, E. (1998), "The Evidence on Class Size", Occasional Paper 98-1, Wallis Institute of Political Economy, University of Rochester, February.         [ Links ]

Heckman, J. Layne-Farrar, A. and P. Todd (1996), "Does Measured School Quality Really Matter? An Examination of the Earnings Quality Relationship", Chapter 7 in Does Money Matter?, G. Burtless Ed., Brookings Institution Press, Washington D. C.         [ Links ]

Heckman, J., J, Smith and N. Clements (1997), "Making the most out of programme evaluations and social experiment: accounting for heterogeneity in programme impacts", Review of Economics Studies 64, 487-535.         [ Links ]

Heckman, J., H. Ichimura, J. Smith and P. Todd (1998), "Characterizing Selection bias Using Experimental Data", Econometrica, 66, Nº 5, 1017-1098.         [ Links ]

Heckman, J., Lalonde, R., and Smith, J. (1999), "The Economics and Econometrics of Active Labor Market Programs", Handbook of Labor Economics, Volume 3, Ashenfelter, A. and D. Card, eds., Amsterdam: Elsevier Science.        [ Links ]

Heckman, J. and E. Vytlacyl (2000a), "Policy Relevant Treatment Effects", mimeo, University of Chicago, December (published in American Economic Review, Papers and Proceedings, May 2001, 91(2): 107-111).         [ Links ]

Heckman, J. and E. Vytlacyl (2000b), "Structural Equations, Treatment Effects and Econometric Policy Evaluation", unpublished manuscript (Fisher Schultz Lecture) presented at the World Congress of the Econometric Society, Seattle, WA.         [ Links ]

Heckman, J., J. Tobias and Vytlacil (2000), "Simple Estimators for Treatment Parameters in a latent variable framework with an application to estimating the returns to schooling", NBER Working Paper 7950.         [ Links ]

Hoxby, C. M. (1998), "What Do America's Traditional" Forms of School Choice Teach Us about School Choice Reforms?" in Economic Policy Review, Federal Reserve Bank of New York, March, Vol. 4 (1).         [ Links ]

Hoxby, C. M. (1994), "Do private schools provide competition for public schools?", Working Paper Nº 4978, National Bureau of Economic Research.         [ Links ]

Jiménez, E. (1986), "The Public Subsidization of Education and Health in Developing Countries: A review of equity and efficiency" World Bank Research Observer, 1: 111-129, January.         [ Links ]

Jiménez, Lockhead and Wattamaha (1988), "Relative Efficiency of Public and Private Schools: the Case of Thailand" WBER, May.         [ Links ]

Kremer, M. (1995), "Research on Schooling: What we Know and What we Don't. A Comment on Hanushek", World Bank Research Observer, 10(2): 247-54, August.         [ Links ]

Lee, L. "Generalized Models with selectivity", Econometrica, March, 51(2) pp. 507-12.         [ Links ]

Maddala, G. S. (1983), Limited dependent and Qualitative Variables in Econometrics, New York: Cambridge University Press.         [ Links ]

Mc Ewan, P. and M. Carnoy (1998), "Competition and Sorting in Chile's Voucher System", mimeo Stanford University.         [ Links ]

Mc Ewan, P. and M. Carnoy (1999), "The Effectiveness and Efficiency of Private Schools in Chile's Voucher System", processed, World Bank.         [ Links ]

Mizala, A. y P. Romaguera (1998), "Desempeño Escolar y Elección de Colegios: La Experiencia Chilena", Documento de Trabajo Nº 36, Centro de Economía Aplicada, DII, Universidad de Chile. junio.         [ Links ]

Mizala, A., P. Romaguera y D. Farren (1998), "Eficiencia Técnica de los Estableci- mientos Educacionales en Chile", Documento de Trabajo Nº 38, Centro de Economía Aplicada, DII, Universidad de Chile, julio.         [ Links ]

Mizala, A., P. Romaguera, and T. Reinaga (1999), "Factores que inciden en el rendimiento escolar en Bolivia", Documento de Trabajo Nº 61, Serie Economía, Departamento de Ingeniería Industrial, Universidad de Chile.         [ Links ]

Mizala, A. and P. Romaguera (2000), "School Performance and Choice: The Chilean Experience". Journal of Human Resources, 35(2): 392-417.         [ Links ]

Mizala, A. y P. Romaguera (2000), "Determinación de Factores Explicativos de los Resultados Escolares en Educación Media en Chile", Serie Economía Nº 85, Centro de Economía Aplicada, Departamento de Ingeniería Industrial, Facultad de Ciencias Físicas y Matemáticas, Universidad de Chile, agosto.         [ Links ]

Neal, D. (1998), "What have we Learned about the Benefits of Private Schooling?" en Economic Policy Review, Federal Reserve Bank of New York, March, 4 (1).         [ Links ]

Neal, D. (1997), "The Effects of Catholic Secondary Schooling on Educational Achievement", Journal of Labor Economics, 15, Nº 1.         [ Links ]

Rounds, P. (1996), "Will pursuit of higher quality sacrifice equal opportunity in education? An analysis of the education voucher system in Santiago", Social Science Quarterly, 77(4).         [ Links ]

Rouse, C.E. (1997), "Private School vouchers and Student Achievement: an evaluation of the Milwaukee Parental Choice Program", NBER Working Paper 5964.         [ Links ]

Sapelli, C. y Torche, A. (1999), "Determinantes de la Elección entre Colegios Públicos y Privados", Mimeo, Pontificia Universidad Católica de Chile, diciembre,         [ Links ]

Tessada, J. A. (1998), "La Educación en Chile: Nuevas Reformas y Revisión de la Situación Actual", Seminario de Título, Instituto de Economía, PUC.         [ Links ]

Todd, P. and K. Wolpin (2000), "On the Specification and Estimation of the Production Function for Cognitive Achievement", Department of Economics, University of Pennsylvania.         [ Links ]

Tokman, A. (2001), "Is Private Education Better? Evidence from Chile", mimeo, University of California, Berkeley, April.         [ Links ]

Vial, B. (1998), "Financiamiento Compartido de la Educación", Cuadernos de Economía 106, pp. 325-342, diciembre.         [ Links ]

West, E. G. (1997), "Education Vouchers in Principle and Practice: A Survey", World Bank Research Observer, 12:83-103, February.         [ Links ]

* Instituto de Economía, Pontificia Universidad Católica de Chile. csapelli@faceapuc.cl

** Instituto de Economía, Pontificia Universidad Católica de Chile. bvial@faceapuc.cl

 

APPENDIX 1: REGRESSION RESULTS

TABLE A1.3. SELECTION MODEL RESULTS, BY TRANSFER QUINTILES
(GEOGRAPHICAL AREAS). LANGUAGE TEST SCORE

TABLE A1.3. (CONT.)

TABLE A1.3. (CONT.)

APPENDIX 2

Description of the variables

In the test scores equations we included the following variables:

Income group: we included 9 dummies in the regression, the lowest income group excluded. Income groups were constructed according to total family income.

Father and mother education: we included 7 dummies for father's and mother's educational level in the regression, the lowest educational level excluded. Dummies were constructed according to years of education completed.

Indigenous family: we included a dummy for children who come from indigenous families.

In the selection equations we included the previous variables, in addition to the following:

Municipal and PS Schools by km2 in the geographical area (GA): we included the number of schools per km2 in the geographical area, by type of school. Geographical areas are defined by the province where the student attends school, and by electoral district in Santiago Province (due to the large size and population density of this province).

Ratio of the number of students in PS school and number of students in subsidized (MUN+PS) schools in the GA: To estimate the number of students in each school, we used information on the number of students in first grade (of high school), in order to avoid a bias due to different dropout rates.

The construction of quintiles according to additional funds received by municipal schools

In order to compare schools with similar budget, we run separate regressions by geographical areas, according to the level of local and central government funds received by municipal schools in addition to the voucher. For this purpose, we used external information on municipal funds transferred to municipal schools, and on school participation in two important central government programs.

We first separated the data set by province of residence, and by electoral district in Santiago Province (due to the large size of this province). Then we estimated the average municipal funds transferred to municipal schools per student in each area, using the information on total funds transferred, and the total number of students by municipality. To estimate central government funds transferred to the schools in addition to the voucher, we used information on Montegrande and PME programs. Montegrande program assigned US$34.760.000 to 51 schools for five years, or US$173.8 per student each year. In 1998, PME (Programa de Mejoramiento Educativo) program assigned US$4.544.000 to 287 schools (considering only high school), or US$28.2 per student approximately. We used the previous information and information on the percentage of students attending Montegrande and PME schools by geographical area, to estimate the funds received by municipal schools per student in each area, and we constructed an index. Finally, we ordered geographical areas according to the index, and constructed quintiles (each quintile with approximately 20% of High School students)21.

Table A2 shows the number of students, average test scores, and upper and lower bound of government funds received by municipal schools in addition to the voucher for each quintile.

Creative Commons License Todo el contenido de esta revista, excepto dónde está identificado, está bajo una Licencia Creative Commons