Step 3: Develop a Research Methodology
Although you are collecting evidence to support community-led advocacy, it is important to be as unbiased as possible when conducting research. You will need to explain your methodology in your report in order to demonstrate that the results are credible. If you don’t, your advocacy targets may try to dismiss your findings as biased. It is therefore important to carefully design your methods to ensure that the data and information that you collect is accurate and objective.
There is a range of tools that you can use to gather information about the impacts of the project. It is best to use a combination of tools to ensure your data is comprehensive, accurate and reflects the experiences of all groups within the community. However, remember that if you use several tools, if your questionnaires are long, or if you interview a lot of people, you will end up with a large amount of information that you will need to organise and analyze.
Where your research has been limited in some way — for example, due to limited resources or time, or because some community members declined to participate — you need to state this clearly in your report. This helps to build credibility by being open and honest about the possible limitations of your research and prevents other actors from discrediting your results.
Consider your team’s skills and experience to collate the data when you select and design your research tools. For example, is there someone on your team experienced in using computer software, such as Microsoft Excel, to organise and analyze large amounts of quantitative, or measurable, data? If so, you may decide to collect measurable information about impacts by a significant proportion of households. For example, you might collect data on the amount of land lost to the company and the amount of income lost per household.
If your team does not have this capacity, you may decide to only collect qualitative, or descriptive, information and to keep your sample size, or the the number of people you interview, relatively small. For example, you may decide to ask key persons and a small sample of households that have experienced serious impacts to describe the effects on their livelihoods and food consumption. The most interesting reports contain a combination of quantitative and qualitative information.
Data collection tools
Some of the main types of data collection tools you could use include: participatory mapping, key informant interviews, household or individual surveys, and focus group discussions.
- Participatory community mapping can be a helpful exercise. You may wish to do this first, before conducting interviews. Key informants and others with knowledge about the village should be invited to participate. Ask participants to draw the main parts of the village, including its boundaries and main landmarks, on a flipchart. Ask them to draw different types of land uses on the appropriate parts of the map. Next, ask them to mark the boundaries of the project’s concession area to show where this overlaps with the village or community resources. This visual process can help both the research team and community members to understand the geography of the project and how it is affecting their village.
Below is an example of a participatory map drawn by a community. For a case study involving participatory mapping, please see Box 8.
- Key informant interviews are used to gather information about the community and impacts of the project. You might interview village elders, community leaders or organisers, local NGO staff working with the community, or even local government officials. These interviews are usually in-depth and generally aim to collect qualitative data — in other words, descriptive information that captures the project’s impacts and the community’s situation.
- Household or individual surveys are used to collect in-depth information about impacts as experienced by people in the community. Similarly to key informant interviews, these usually aim to collect qualitative data to describe the impact of the project on the community. However, depending on how you structure the questions, you might also be able to collect quantitative data. For example, if you have open-ended questions, respondents may give more detailed responses. If you have closed-ended questions, respondents will usually give a simple yes or no answer, or a number or ranking, such as severe, quite severe, positive, very positive. These closed-ended questions can be used to collect quantitative data.
You should pay special attention to vulnerable and disadvantaged groups within the community, such as the disabled, elderly or minorities that experience discrimination.
Make sure that you clearly explain the process you used to select the interviewees in your description of the data collection methodology in your report. The more people you interview from different households, community groups and demographic backgrounds, the greater the chance that your data will be regarded as credible and an accurate picture of what is happening on the ground. If you are analyzing whether human rights or other standards have been violated, it is important to find out about the worst issues and impacts experienced by people within the community. This means that you should intentionally identify and interview people who have suffered losses or negative impacts that may amount to violations.
- Focus group discussions are another way to gather information about a project’s impacts. Some impacts are communal in nature and are best described in a group setting using open-ended questions. The questions aim to facilitate discussion among the group about particular issues and impacts. Interesting points can arise in a group discussion that are not always gleaned from individual household interviews.
Focus group discussions are useful for ensuring that impacts on women and children are recorded. In communities where men tend to dominate, separate women’s group discussions are an effective way for ensuring that women’s voices are heard. Focus group discussions are also useful for gathering information about distinct impacts of the project on minorities or marginalised groups, or on people with disabilities.
Although you should prepare some questions that get people talking about the right topics, you should also be prepared to ask unplanned follow-up questions to probe into unexpected or interesting issues that arise during the discussion. Do your best to politely encourage quieter participants in the group to share their thoughts. It is best to keep the number of participants to less than 12, so that everyone has an opportunity to express their views and experiences. It is particularly important to designate a separate note-taker in your research team to record what is said during the focus group discussions. This will allow the interviewer to concentrate on facilitating the discussion.
Box 8: Case Study
Conducting a human rights impact assessment in Ratanakiri, Cambodia: Using participatory mapping
In several of the villages being interviewed, there was more than one large-scale plantation in the vicinity affecting the communities’ resources. In such cases, it was difficult to separate which impacts were caused by which companies.
To deal with this issue, the research team facilitated participatory community mapping before using the other tools. At least five people in the village who were familiar with local geography and were most knowledgeable about the company’s activities participated in the mapping exercise. They were asked to mark on the map all types of land-use patterns (farmland, forest, streams/rivers, grazing land, burial ground, sacred sites and residential areas) and infrastructure (wells, school, roads and the community center) in the village. They were also asked to point out each company-owned plantation and their boundaries.
Besides helping to understand the land-use patterns and infrastructure in each village, community mapping provided clarity about the location of the company’s plantations compared to the various parts of the village. It also helped participants and interviewers differentiate losses and impacts caused by each plantation. Interviewers used the maps to explain to villagers that they should focus on the Vietnamese company’s plantation when answering questions during interviews. Villagers were then able to attribute particular losses to the investment project assessed, as opposed to plantations owned by other companies. However, due to the cumulative nature of the impacts on food, livelihoods and culture from all the large-scale agribusiness activity in the area, it was not always possible for these to be clearly attributed to a single source.
To read more about this case, see chapter 3 of Human Rights Impact Assessment: Hoang Anh Gia Lai’s Economic Land Concessions in Ratanakiri, Cambodia.
Designing a Questionnaire
Your team will need to prepare a list of questions for each data collection tool being applied. Your questions should be designed to collect in-depth information about the main issues and impacts set within your assessment framework. When developing your list of questions, start with each part of your assessment framework and think of the key questions the research team will need to ask in order to find out whether specific human rights, laws or standards have been breached.
Although the issues that you are investigating will be the same across all data collection tools, the ways you ask the questions will be different. For example:
- When interviewing a key informant you might ask, “Has your community experienced a change in the amount and quality of food that it has access to because of the company’s project? If so, how has it changed, and why?”
- When interviewing individual households or families, you might ask, “Before the project began, how many meals did your household normally eat each day?” You might also ask, “Now, how many meals does your household normally eat each day? Has the quality of food changed? If so, how?”
- When conducting a focus group discussion, you might ask more open-ended questions, such as, “Describe any changes that your families and community have experienced in the food you eat since the company arrived.” As the discussion evolves you can help steer it in a particular direction, while also keeping the discussion to the point.
Questions should not be leading. You should not assume particular positive or negative impacts, but instead ask neutral questions. For example, start by asking whether and how things have changed, rather than what the negative impacts have been.
Conducting and Recording Interviews
In addition to taking written notes of respondents’ answers, it is important to record all of your interviews and discussions so you can check and clarify information that you have missed in your notes. It is also important to have recordings so that if anyone challenges your data, you have proof of what was said during the interviews. Audio recorders are inexpensive and easy to use and are a worthwhile investment for conducting research. Many smartphones also have a recording function. Recordings will also allow you to find good quotes from interview respondents to use in your report.
Make sure you ask the people participating for their consent to record conversations. Let them know that you will not publish their identities unless they consent. If they do not want to be publicly identified, you will record their names for your own verification purposes only, and you will not make them known to anyone outside of the research team. If someone does not agree to being recorded, you must respect their wishes.
During your visits to the community and to the project site, take photos if possible. Visual evidence is very effective at informing the audience and influencing your advocacy targets. If possible, you can also record interviews by video camera that show impacts of the project. Make sure you ask people for their consent to photograph or film them and explain to them how the photo or film will be used. People may be concerned that if their identities are revealed to governmental officials or the company, they will suffer reprisals. Err on the side of caution if this is a concern, and respect the wishes of those who do not want themselves, their families, or property filmed or photographed.
Box 9 below shows a table used to record answers regarding the human rights impact of a rubber plantation in Cambodia.