The first steps in the application of the evaluation methodology on the ground were the selection of target beneficiaries as well as the conduction of ad-hoc surveys in the identified areas. Since its inception, the IFPRI’s HarvestChoice team together with the USAID program design team have adopted a highly-structured approach to geographic targeting, which resulted in the selection of the three geographic areas for the program, namely the West African Guinea Savannah, the Ethiopian Highlands, and the maize- and rice-based systems of East and Southern Africa.6 In line with the AR’s mission, these three regions simultaneously satisfy the criteria of high levels of poverty, high concentration of cereal-based farming systems and low levels of productivity, therefore allowing to reach a large number of individuals in the target typology: poor cereal-based smallholder families.

Within each mega-site, geographical strata (or domains) were identified to represent relatively uniform farming systems where to implement specific sustainable intensification interventions. Given all the constraints, it was not possible to conduct specific research for each one of them so the domains where further classified in terms of the number of potential beneficiaries, infrastructure, environmental concerns and welfare-related characteristics in order to be able to prioritize certain strata on the base of the AR’s objectives. In particular, the stratification of project sites was based on the following attributes: farming system, rainfall, elevation (i.e. proxy for temperature), population density and access to markets.

In Northern Ghana three regions were chosen for the study: the Northern, the Upper-East and the Upper-West region. These areas cover both maize-based and rice-vegetables-based systems and therefore allow to address the production constraints characterizing both realities7. As IFPRI (2012) highlights, the northern regions of Ghana are characterized by small land holdings and low input - low output farming systems, which adversely impact food security. In particular, they are subject to a seasonal cycle of food insecurity of three to seven months for cereals (i.e., maize, millet and sorghum) and four to seven months for legumes (i.e., groundnuts, cowpeas, and soybeans). These crops in the savannahs are often produced in a continuous monoculture, steadily depleting soil natural resources and causing the yields per unit area to fall to very low levels. The poverty profile of Ghana identifies the three northern regions as the poorest and most hunger-stricken areas in the country. Gender inequalities are also apparent in these regions, since women have limited access to resources and therefore limited capacity to generate income on their own.

Guo and Azzarri (2013) reviewed available spatial biophysical and socio-economic data layers for northern Ghana and choose the appropriate layers for the stratification. They note that among the candidate layers on population density, Agro-Ecological Zones, precipitation, elevation, slope, farming system, market access, Length of Growth Period (LGP), and land cover, only some were appropriate to characterize and stratify districts in Northern Ghana. Given their spatial variability, Guo and Azzarri (2013) chose LGP and market access as proxies of agriculture potential and socio-economic integration in the food value chain, respectively. Combining these two layers, they derived six unique classes.8 Based on the stratification analysis and after consultation with local project partners, six action districts were initially identified. However, subsequent field work raised concerns over this first subdivision as for example the high density of rural population in some districts in the Upper East region that were not adequately sampled. As a result, there was a second round of field work that resulted in the identification of ten target districts.

As IFPRI’s Report (2014) highlights, to identify action and control sites the following steps were taken. First, all known villages within each district were mapped based on the digital locations provided by AR and on the digitization of printed maps. Also, new market access maps were prepared from the latest available digital maps of roads and tracks and were updated daily as the field work progressed, in order to eliminate inaccessible communities from the list of potential sites. Further, potential communities were selected ex-ante on the basis of a geographic framework ensuring an appropriate distance between action sites and counterfactuals (to avoid contamination), and paper and digital maps were prepared before each day of field work. Once obtained the list of potential beneficiary villages, all the selected communities were visited to check their suitability in terms of farming systems, accessibility and size. The team on the field was composed by the consultant, the project manager and other staff members from IITA, as well as the officers from the Ministry of Agriculture, which were familiar with the district’s features. Some pre-selected villages were abandoned, and other suitable sites were identified during the field work. Finally, the locations of all suggested action and counterfactual sites were presented during a planning workshop in Tamale at the end of October 2012.

During the above mentioned workshop, IFPRI raised concerns about the physical closeness of intervetion and counterfactual communities. Hence, some of the sites were abandoned and new ones were chosen as a replacement. The identification of suitable counterfactual communities has been a particularly difficult task, since to obtain a reliable impact assessment they have to present very similar properties as the action communities (i.e. population density, cropping system, market access, etc.), but should also be as far as possible from them to avoid being contaminated by spill-overs. Ideally, inhabitants of counterfactual communities should not meet inhabitants of action villages, and they should not share markets or other public facilities. These two main conditions –similarity and isolation- can very rarely be achieved at once. The best solution would be to have action and counterfactual sites located in different districts, but in northern Ghana this is rarely feasible because of the big differences in market accessibility and cropping systems across them. In addition, there are no major physical barriers to movement such as very large rivers, swamps or mountain ranges allowing to isolate control and treatment sites.

Overall, the evaluation design includes 25 counterfactual communities as well as 25 intervention communities. In particular, 18 communities were selected in the Upper West Region (8 counterfactual communities and 10 intervention communities), 11 communities in the Upper East Region (6 counterfactual communities and 5 intervention communities) and 18 communities in the Northern Region (8 counterfactual communities and 10 intervention communities) (Figure 1). However, during fieldwork, it was noted that an administrative reform of districts took place and there was therefore the need for re-arranging the geographical belonging of some communities. In particular, four communities (Namiyila, Arigu, Basigu, Karemiga) were moved from the Upper East administrative region to the Northern region jurisdiction. In light of this new categorization, Upper East Region finally includes 7 communities (5 intervention communities and 2 control communities) whereas Northern Region covers 25 communities (15 counterfactual and 10 intervention communities). Nevertheless, it is worth underlining that the physical proximity of such communities to the Upper East Region would rather suggest their inclusion in this geographical category for analytical purposes (e.g. similarity in characteristics of its population).

The households in the selected communities were divided into four groups:
1) Households in control communities;
2) Non-beneficiary households in AR intervention communities;
3) Africa RISING beneficiary households (2013) in AR intervention communities;
4) Africa RISING interested households (2014) in AR intervention communities;

The first step of the sampling strategy consisted in the stratification of the communities on the lines of the development domains at the district level. The second stage randomly selected households within each community. In particular, a constant number of control households (n=20) was randomly selected in each of the 25 control communities for a total of 500 control households. In regard to the 25 intervention communities, the sampling strategy was to randomly select a constant number of households (n=8) not directly benefitting from AR intervention and a constant number of 6 households interested in joining the program in 2014. Finally, 462 households that directly benefitted from the AR 2013 program were selected to participate to the survey. These guidelines were followed as closely as possible and only in a few cases the number of surveyed households in each group could not exactly match the target.The total sample size for GARBES is 1,284 households, of which 784 households in intervention communities and 500 in control communities. The households in target villages are further divided into 454 AR beneficiaries, 148 AR future beneficiaries and 182 AR non-beneficiaries.