An Environmental Impact Assessment System for Agricultural Research and Development II : Institutional Learning Experience at Embrapa

“An environmental impact assessment system for agricultural R&D” has been implemented by the Brazilian Agricultural Research Agency (Embrapa) aiming at fulfilling the institutional mission statement of ‘introducing sustainable development objectives in all steps of agricultural research’. The impact assessment platform emphasizes close interaction between R&D teams and technology-adopting producers, under actual field contexts, in order to improve both the technology development and the demand probing processes. The proposed integrated environmental indicators system (AmbitecAgro) has been routinely applied in technology appraisals by all of Embrapa’s Research Units, as one of the criteria of its institutional evaluation system, and toward the formulation of the Social Balance Reports, annually published since 1997. The present paper describes the social and environmental dimensions of this integrated impact assessment platform, its roles in the institutional learning process for technology research management, and an impact analysis of proposed agricultural innovations.


Introduction
As continuation to a proposed "environmental impact assessment system for agricultural R&D" (Rodrigues et al., 2003a), this paper aims to describe the general procedures for technology appraisal developed at the Brazilian Agricultural Research Agency (Embrapa).Based on a review of the current context of 'agricultural R&D for sustainable development' (EIARD, 2003), the text (i) presents the premises upon which the institutional impact assessment platform has been constructed; (ii) describes the background and methodological bases, following with a (iii) technology innovation typology for allowing the (iv) analysis of impact assessment results obtained to date, a (v) conclusion regarding the institutional learning process for technology research management, and a set of (vi) recommendations for strengthening the impact assessment network.
All these subjects stem from a current institutional orientation in the agricultural R&D sector toward the organization of innovation systems that may, on the one hand, target technology advancements for productive intensification, and on the other hand include a wider agenda for poverty alleviation, environmental management, and social inclusion (Horton and Mackay, 2003).In order to instruct these objectives, agricultural research organizations seek strategies to plan their resources allocation, the management of their capabilities, and the transfer of their results.Such strategies require viewing innovations in systemic terms, with flows of knowledge between research teams and external parties -technology adopting farmers foremost -as key for organizational performance (Hall et al., 2000;2003).
The proposed wider agenda for agricultural R&D implies viewing innovations as contributions to sustainable livelihoods, recognizing that prosperity (the goal of technology adoption) depends on the satisfaction of diverse and multiple interests of farmers, their families and communities (Adato and Meinzen-Dick, 2002).This is not a novel standpoint, having been brought up decades ago with demands for change in R&D evaluation methods, in order to account for institutional development, sustainability, and environmental impact (Conroy and Litvinoff, 1988).
The focus on sustainable development, environmental management and social inclusion poses a challenge to technology innovation decision making, be this related to what technology to develop, for application where, and for adoption by whom, entailing questions on how to make the innovations available and involve all people concerned in these decisions (Haque, 1991).Actor-oriented, integrative, and participatory approaches are increasingly employed to address the multiple sustainability objectives of different groups of interest.Also, farmer involvement in agricultural technology R&D contributes to identification of appropriate applications, adoption mechanisms, and transfer to larger numbers of users, providing valuable information feedback to research teams (van de Fliert and Braun, 2002).
In order to pursue sustainable livelihoods as the goal for technology innovation, and to exercise integrative, participatory approaches for research, adoption and transfer, agricultural R&D organizations must promote a veritable institutional learning transformation (Watts et al., 2003).This learning and change process has been associated with evaluation procedures (Horton and Mackay, 2003), which purpose it is to help managers and scientists to better understand how their research activities generate effectively impacting results, and to draw lessons about how to improve future research programs (Raina, 2003).
Institutions undertake evaluations in order to organize their (i) needs assessment, (ii) priority setting, (iii) research proposals review, (iv) ongoing research monitoring, (v) completed research appraisal, (vi) research output verification, (vii) impact assessment, (viii) programs and management review, and (ix) staff performance assessment (Horton, 1998).To meet these intents, evaluations must be organized in credible procedures, with little room for speculative convergence of cases (Nelson and Meredia, 2007), or mere plausible links between research and impacts (EIARD, 2003).
Impact assessment methodologies comprise the toolkit for consolidation of innovation systems, to orient innovation demand probing, research priorities setting, research formulation and agricultural practices recommendations (Mackay and Horton, 2003).Given such broad sense benefits and wide expectations for management, institutions must gauge a compromise between how much impact assessment they should do, and how sophisticated the assessments should be (Collison and Tollens, 1994).
Attentive to these institutional tendencies and in concerted action with PROCISUR4 (Puignau, 1998) and other institutional partners, Embrapa has been committed to the systematic assessment of impacts on agricultural technological innovations, as attested by scientific meetings held (Puignau, 1998;Embrapa, 2008) and cooperative research developed on the subject (Rodrigues et al., 1998;Bin et al., 2003;Bonacelli et al., 2003).The accorded approach has been to foster Environmental (and Social) Impact Assessments (EIA) with the declared objective of "promoting solutions for the sustainable development of rural spaces by generating, adapting and transferring knowledge and technology for the benefit of society" (Embrapa, 2004).

Methodological Considerations
Impact assessments were implemented at Embrapa in the early eighties, after an initial organization period  when the institution obtained generous public funding, supported by international loans aimed at creating its infrastructure and training its human capital (Neinke et al., 2001).Following this initial phase, a series of financial crises affected most public institutions of the country in the mid 1980s and 1990s, when Embrapa underwent severe budget cuts (Avila and Souza, 2002).The new scenario motivated the initiative of the Administration Board to encourage impact assessment studies, as to improve the visibility of agricultural technologies' role in rural development and better justify public investments.
The mono-disciplinary (economic) view was then complemented by an impact evaluation platform covering economic, social and environment aspects (Avila et al., 2008), systematized as an integrated impact assessment system (Bosshard, 2000).Indicators were selected according to sustainability objectives (Lewandowski et al., 1999), for estimation by scores in a multi-criteria procedure formulated as to avoid the ambiguous step of monetization of environmental goods and social effects (van Pelt, 1994).The proposed impact indicators (Esterhuizen and Liebenberg, 2001) were integrated in scaling checklists, in which original quantitative data obtained in field surveys (measures of area, quantities, concentrations, proportions, numbers) are normalized into a non-dimensional unit for allowing cross-scale and multivariable aggregation (Bisset, 1987;Rodrigues, 1998).
Aggregation is a much debated step in impact assessment practice.On the one hand it consents integration and accounting for heterogeneous sets of characteristics and effects, while on the other hand it may cause a loss of meaning for some of the criteria in the dataset, if expression of integrated results is not appropriately organized (Andreoli and Tellarini, 2000).The necessity of normalization and aggregation is easier to justify in multidimensional assessment systems that include both environmental (impact and performance) and social (health, food security, welfare) considerations (Pinho and Pires, 1991).Upstream, this means accounting for resources required for technology development (e.g., raw materials, habitats affected).Downstream, it means consideration of the residuals and effects of technology application on environmental quality and social welfare (Porter, 1995).
Compounding on these methodological premises, a system for environmental impact assessment of agricultural technology innovations has been formulated, based on objective indicators, constructed on a flexible platform, acceptable for application on the large diversity of rural activities, environmental situations and their combinations typically included in Embrapa's agricultural R&D program.

A System for Environmental Impact Assessment of Agricultural Technology Innovations -Ambitec-Agro 5
An integrated impact assessment procedure has been developed for motivating farmers to promote agricultural technology conversion and adoption, facilitating technology development project appraisal and socio-environmental impact assessment at the institutional R&D level (Rodrigues and Rodrigues, 2007).The so-called Ambitec-Agro system (Rodrigues et al., 2002;2003a) has been installed in the impact assessment platform employed yearly by all Embrapa Research Centers to (i) evaluate their technological contributions (Avila et al., 2005), (ii) assist in research project appraisals (Rodrigues et al., 2000) and (iii) assess social and environmental impacts of technology innovations made available through R&D (e.g., Lanna et al., 2004;Rodigheri et al., 2006;Tosto et al., 2006a;Rodrigues et al., 2006;Carvalho et al., 2007;Duarte et al., 2007;Mori et al., 2007;Almeida et al., 2007;Sá et al., 2008;Silva et al., 2008;Almeida et al., 2009).
The Ambitec-Agro structure relies on a series of Principles of technology and rural activity performance, composed by social and environmental compliance Criteria, integrated by series of sustainability Indicators selected from prior experience and field trials (Irias et al., 2004a;Magalhães et al., 2006).The indicators are scored in field surveys / interviews with farmers / administrators, to obtain change coefficients according to technology or rural activity effects observed in the studied contexts.The change coefficients are weighted by factors related to each indicator's relevance toward effecting socio-environmental impacts and its scale of occurrence (Rodrigues et al., 2003b;Monteiro and Rodrigues, 2006).Finally, impact indices are calculated for each indicator and criterion, and aggregated as a technology innovation socio-environmental impact index.
The Ambitec-Agro system comprises four modules, focused on the productive sectors of Agriculture, Animal husbandry, and Agro-industry environmental impact assessment (Irias et al., 2004b) and a specific module for social impact assessment (Rodrigues et al., 2005a), encompassing 24 criteria and 125 indicators (Table 1).
The impact assessment of a given technology innovation with the Ambitec-Agro system is carried-out in three steps: 1. definition of technology innovation use magnitude, geographical area delimitation and users; 2. field survey / interview at the rural establishment scale, applied with innovation-adopting farmers and data filling out in the scaling checklists; and 3. analysis, interpretation, and reporting of impact indices (in formatted templates), with proposition of alternative management practices and technology adaptation, focused on minimizing negative impacts and promoting positive ones.
A stimulated practice is to return these technology assessment reports to farmers and rural establishment administrators, favoring technology use improvement and contributing to local sustainable development.(SeeTable 1 at the end of the article)

Ambitec-Agro scaling checklists
The Ambitec-Agro system consists of integrated indicator scaling checklists, in which change coefficients checked in field surveys / interviews are related to quantitative measures of area, quantities, proportions, etc., then standardized as varying from -3 (meaning a major decrease in the indicator) to +3 (meaning a major increase in the indicator), reflecting the effects of the studied technology or rural activity, contingent to each particular assessment context.
The indicators are then weighted according to their defined relevance to conform the assessment criterion and their scale of occurrence.The weighing factors related to the relevance of each indicator are defined on an ad hoc basis according to user criteria in order to better reflect specific situations and add up to ±1 (according to the indicator impact direction, either positive or negative).Hence, the relevance weighting factors consist of a normalization step to equalize the different number of indicators that make up each assessment criterion.
A second series of weighting factors are related to the geographic scale in which the indicator change coefficient occurs in any studied case, as follows: i. near environment -when the innovation / rural activity effect on the indicator is restricted to the crop area, productive field or facility where the studied innovation is being adopted / activity is being conducted; ii. proximate environment -when the innovation / rural activity effect on the indicator extends beyond the productive unit, but within the limits of the rural establishment; iii. surrounding environment -when the innovation / rural activity affects the indicator in an area or environment beyond the limits of the rural establishment.
Once the change coefficients resulting from the field survey / interview are introduced in the scaling checklists, the impact index for each indicator is calculated, according to the given relevance values and scale of occurrence, and then combined to express the impact index for the criterion (resulting range ±15, Figure 1).
The given example for the water quality criterion represents a field observation of moderate reduction in BOD at the proximate environment scale, a major decrease in turbidity also at the proximate environment, a major reduction in the presence of floating materials / oil / scum in the surrounding environment; and no-effect in siltation (note that the weighting factor for siltation is zeroed, with corresponding weighting factor being transferred, in the given example, to BOD).Twenty four such scaling checklists, with a total of 125 indicators, comprise the Ambitec-Agro system.Once all indicator change coefficients are inserted into the scaling checklists, a Technological Innovation Impact Index is calculated for the specific conditions studied, by averaging all the normalized impact indices for the criteria considered.Similarly to the weighting factors included in each indicator scaling checklist, this normalization step allows a new adjustment of relevance values, this time for the different criteria considered in the impact assessment system.With this definition of relevance weights for indicators and criteria (Figure 2), assessments may be better adapted to specific evaluation contexts, by emphasizing relevant local aspects or evaluation objectives, or even by excluding certain aspects that may not appropriately represent meaningful consideration for particular cases (no-effect).(See Figure 2 at the end of the article)

Ambitec-Agro insertion in the institutional impact assessment platform
The aim of Ambitec-Agro is to provide a practical, expeditious, low cost, and reproducible socio-environmental impact assessment procedure for the wide range of agricultural technologies and rural activities concerned in Embrapa's research program.These particular technology innovations, made available through the numerous R&D projects in the decentralized Research Centers, comprise the basic units of Embrapa's impact assessment platform.The institutional coordinating body (the Secretariat for Management and Strategy, SGE) recommends for each of the 37 Research Centers to proceed evaluations on three selected innovations identified as in initial adoption phase, by three consecutive years.
Evaluations are carried out by appointed teams in the Research Centers, normally comprised by dedicated socioenvironmental researchers and the members of the group responsible for the research project in which the selected technology innovation development has taken place.A budgetary allowance is made available through SGE for this task, estimated as a minimal amount sufficient to fund interviews / field surveys in ten technology-adopting rural establishments.Results of these field studies are entered in formatted datasheets and report templates, available through intranet access.These templates facilitate calculations of economic impacts and internal rates of return, as well as discussion regarding social and environmental impact indices (Avila et al., 2008).Lastly, the final integrated impact assessment report composed for each technology innovation is verified for consistency and formulation quality at SGE, which issues a statement for each Research Center regarding completion of the evaluation process.Technologies addressing agro-industrial processes and post-harvest practices may be simpler mono-disciplinary, or extremely complex multidisciplinary, inter-institutional endeavors; aimed either at quite specific or very broad applications, to simpler incremental gains in input and energy use efficiencies or quite innovative products and processes.Typically these technologies are related with semi-industrial, work-intensive processing phases of production, so affecting social indicators more importantly.
Innovations classified as improved agricultural practices are in general efficiency related, normally aimed at adapting or integrating proven practices to new specific uses.In this sense these innovations may involve either a few or many branches of knowledge, basic or advanced engineering solutions, and simple or complex integration of resources and production means.In all cases, however, they are aimed at obtaining incremental gains for specific applications, products or production practices.Integrated management technologies, for instance 'Integrated cotton production system for the Cerrados', 'Management of native açai stands in floodplains and estuaries of the Amazon', 'Sisplan -computational system for forest management', 'No-till management for soybean', 'Crop rotation system with winter pastures', and the like.
Based on these integrative levels for agricultural technology innovations, an analysis of the socioenvironmental impact assessments carried out to date at Embrapa has been performed, allowing a better understanding of the contributions of research to agricultural development, as well as an analysis of the institutional learning process regarding the impacts of innovations made available by research.
To date, as included in the Social Balance Reports 2005-2009 (published the following years), 123 innovations selected by Embrapa's Research Centers have been evaluated, corresponding to 180 matched social and environmental assessments 7. The aggregated environmental and social impact indices and their frequency distribution, according to the proposed technological integrative levels, can be seen in Figure 3.The insert graph shows that the frequency distribution of the untransformed impact indices tend to normality, despite the one-tailed distribution observed for social impact indices, given that no negative observations occurred for this impact dimension in the whole sample.
No definite relationship has been observed in the dataset between environmental and social impacts (r2=0.052),that is, one can not say whether a more positive impact on the environment will result in increased social benefits.In effect, there is no reason to expect this association to show any predetermined pattern, given the trade-offs between short-term income objectives and long-term environmental ones (van Pelt, 1994), and the very diverse possible interactions and adoption contexts included in the large set of technology innovations studied.
For example, highly positive environmental performance improvements, such as savings in energy expenditure, habitat restoration, or even water quality or biodiversity conservation may well be achieved without immediately impacting peoples' quality of life.Conversely, intensification in natural resources or input uses, which frequently entail negative environmental impacts, usually result in important gains in production scale, worker engagement and work formality, income generation, and social wellbeing improvements.
Several instances of negative environmental impact indices were observed in the dataset (Figure 3), related with agricultural intensification on the one hand, and expansion of agricultural areas onto new territories, resulting in natural habitats occupation, on the other hand.Crop and livestock breeding technologies have been associated with both increased demands on inputs in many cases, and expansion of crops and animal husbandry to new areas, thus frequently resulting in negative environmental impact indices (37.5%).Agroindustry / post harvest technologies (33.3% negative indices) have been linked with increased demand on energy and inputs, which also tend to impose negative environmental impacts due to resources demands and residue emissions.
Input use technologies followed in terms of negative environmental impact indices (30.8%) due to intensification in demand, while integrated management technologies (20% negative indices) were most frequently at the origin of negative environmental impacts due to agricultural expansion onto new, sometimes marginal areas, imposing pressures on natural habitats.Due to their incremental and adaptive nature, improved agricultural practices showed just 7.1% of cases with negative environmental impact indices.By contrast with environmental impacts, social impact indices obtained in the field studies were without exception positive (Figure 3), allowing to infer from the large dataset that, in general, agricultural technology innovations forwarded by the Research Centers contribute favorably toward sustainable livelihoods and rural development.
This prevalence of highly positive social impact indices has been shown to be especially true for those innovations classified as agro-industry / post harvest (mean impact indices for all years = 3.96) and integrated management technologies (mean impact indices for all years = 3.39; Table 2).These two sets of innovations showed significantly higher social impact indices than the other three types, for all years in which assessments were carried out.While no statistical differences were observed in the ANOVA among years (environmental indices' pvalue = 0.339; social indices' p-value = 0.104), specific contrasts showed that social impact indices were always significantly higher than environmental impact indices (See Table 2 at the end of the article).
On the other hand, besides showing fewer negative results, environmental impact indices were almost invariably higher for innovations classified as improved agricultural practices (except for year 2006, when agro-industry / post harvest technologies faired better) and integrated management technologies (Table 2).These two sets of innovations were shown to be significantly more effective for saving resources, managing emissions, and conserving / restoring ecosystems' functions, all explicitly considered criteria in Ambitec-Agro (see Table 1).
This kind of analysis and interpretation of results, focusing on aggregate impact indices and their tendencies (as positive or negative effects) represents the first step for technology research prioritization and transfer strategy.These are based on the norm enunciated for the assessment system, which states "recommendation of agricultural technology is conditioned to improvement of the environmental and social performances of the activity to which technology is applied, as measured by designated indicators" (Rodrigues et al., 2003a).This norm does not mean that negative impact indices will immediately impose that an innovation should be discontinued, but rather indicates major issues for research continuation, emphasizing trade-offs pointed out in the assessments.
Much improved detailing of these trade-offs is possible in the impact assessment procedure, with analysis of the impact indices at the disaggregated criteria and indicator levels.At these levels, aspects pertaining to use of inputs and resources, environmental quality (atmosphere, water, soils and biodiversity conservation), natural habitat restoration, as well as product quality and food security, employment generation and quality, income, health, among many other indicators can be pondered upon, with reference elements obtained in real field contexts, together with technology adopters.
It is actually at this level of detail that reports are presented in the templates provided in the impact assessment platform.By delving into these reports, technology research teams are better equipped to define new steps for technology development and transfer, while research administrators can better decide on priorities for new R&D calls.

Discussion -Institutional Learned Lessons
Hundreds of evaluations, of a myriad of technological innovations have been carried out and gathered since the inception of systematic impact assessment practice at Embrapa.The strategic objective of extending impact evaluations beyond the economic internal rate of return or benefit / cost analyses, introducing social and environmental evaluations, has been crucial for the critique of the institutional role in agricultural development.This process, however, has not been always smooth, and the provision of a basic methodological approach may have facilitated the internalization of the impact assessment practice (Andrade and Moreira, 2009).
In what concerns the proposed methodological approach, a recent independent study on the practice of impact assessment at Embrapa has been carried out.This study pointed out both "ameliorating and funneling tendencies, by which creativity and normativeness coexist… in the institutional learning experience and the quotidian of technical innovation" (Andrade, 2008).In addition to this institutional-level critique, the applicability of Ambitec-Agro has been included in a review of several impact assessment methods (Payraudeau et al., 2005).The study emphasized the need for methods to be transparent in order to facilitate farmer participation, simple to allow uncomplicated field application, and sufficient in number and scope of indicators to avoid gaps in the assessments.The results obtained to date at Embrapa, reviewed in the present study, seem to imply that these features have been satisfactorily met.
The early decision of involving actual field surveys and farmer participation in the assessments has also served valuable objectives.First, impact causality attribution to specific innovations can be more immediately identified, excluded or harmonized, based on clear definition of technology application context, case-by-case, with active contribution of the actual users, according to their effective practice in the field (Gottret and White, 2001).
Second, this context definition step of the evaluation corresponds to the 'pressure premise' in EIA science: be adaptable to local socio-economic contexts, physicochemical environments and production scales (OECD, 1993).Third, the next phase in the assessments, the field survey / interview, scoring of technology effects, and impact indices calculations corresponds to the 'state premise' of EIA science: express the effects of changes on the quality of the environment and natural resources, including social, economical and ecological concerns.And fourth, the reporting step (with the provided templates and forms) corresponds to the 'response premise' of EIA science: offer the basis for issuing recommendations for decision making (including for farmers), contributing toward local sustainable development (OECD, 1993).
The experience on Impact Assessment at Embrapa and the recognition received from the Brazilian and international R&D organizations have shown that the adoption of a multi-dimensional approach contributes to a better understanding of the impacts of agricultural research as compared to more conventional, disciplinary evaluations.Once an integrated multi-dimension approach can be exercised, a main lesson is that impact assessment in agricultural research has a stronger influence when it is also integrated through feedbacks to institutional management, favoring research priority setting, planning, award / promotion assignment and formal institutional accountability, as consolidated in the Social Balance Reports.

Figure 3 .
Figure 3. Distribution of environmental and social impact indices obtained in 180 evaluations, relative to 123 technological innovations classified as pertaining to five different integrative levels, and frequency distribution of indices' rankings, obtained with the Ambitec-Agro system.Embrapa Research Centers, 2005-2007.

Figure 2 .
Figure 2. Final environmental impact assessment display of the Ambitec-Agro system.
Different letters associated to the means represent statistically significant contrasts in the t-test (p<0.05).

Table of change coefficients for variable
Input use technologies, for instance 'use of gypsum in soy (also in coffee, corn) in Cerrado soils', 'Dosage of the growth regulator mepiquat chloride in cotton', 'Bioinsecticide against A. aegypti larvae', 'Use of sewage sludge in agriculture', and the like; levels', as follows (the innovations mentioned below are actual examples appearing within Embrapa's portfolio, in one of the cited Social Balance Reports): Improved agricultural practices, for instance 'Integrated control of pests in apple', 'Use of gliricidia as live posts for fencing', 'Enriching the herb stratum in Caatinga for ovine raising', 'Nitrogen biological fixation in soybean', 'Integrated management of the wood-wasp (Sirex noctilio) in pine stands', and the like;

Table 1 .
Embrapa Pecuária Sudeste São Carlos.53 Journal of Technology Management & Innovation © Universidad Alberto Hurtado, Facultad de Economía y Negocios Integrated Principles, Criteria and Indicators included in the several modules of the System for Impact Assessment of Agricultural Technological Innovations (Ambitec-Agro).Source: Monteiro and Rodrigues (2006).

Table 2 .
Aggregated results for social and environmental impact assessments carried out on a set of 123 technology innovations evaluated by Embrapa's Research Centers to date